S3 Configuration

The task accepts one or more of the following configuration settings. Configuration settings are similar in functionality to properties, but they are rarely used. In order to avoid "polluting" the property namespace of the task, access to these internal properties is provided through the Other property.

S3 Configuration Settings

DirDelimiter:   The delimiter for subfolders on the S3 server that should not be maintained in the local directory.

By default, a downloaded S3 object name will be appended to the LocalDirectory when the object is written to the local filesystem. If the object name includes folder and/or subfolder delimiters, these will be treated as actual folders within LocalDirectory. To ignore these folders/subfolders and write the files directly in the LocalDirectory, set this config to the character or string delimiting folders/subfolders on the S3 server.

For example, if the object is stored in S3 as "folder1/folder2/file.txt", and neither "folder1" nor "folder2" should be present inside the LocalDirectory, this config should be set to /.

DownloadCacheFile:   When set, only new files will be downloaded from the server.

This may be set to the location of a file on disk that will hold the names of files that were previously downloaded from the server. On each PollingInterval the task will check to see if the file on the server exists in this list. If the file exists in this list, it is determined that the file was previously downloaded and is not downloaded on the current polling interval. The list is automatically updated on each polling interval.

DownloadCacheFileDuration:   The number of minutes that a file name will remain in the download cache.

If a file name is added to the DownloadCacheFile and is not seen on the server again within DownloadCacheFileDuration minutes, it will be removed from the cache. This can prevent the download cache file from growing too large in certain cases.

The default value is 0, meaning that files will never be removed.

This setting is only applicable if DownloadCacheFile is set.

EnableFileDetailsComparison:   Whether the receive task should consider metadata differences when deciding whether to download a file again.

This setting is used in conjunction with DownloadCacheFile, and is only available on the receive task. If DownloadCacheFile is enabled, this setting can be used to control how the receive task uses the download file cache information when deciding whether to download a file again. (Similar to the "Enable Timestamp Comparison" property on Microsoft's FTP BizTalk Adapter.)

If this is set to False, the receive task will not download a file again if the cache contains any information about it (that is, if it has been downloaded before).

If this is set to True, the receive task will check the metadata of the remote file against its cached metadata, and will download the file again if any of the metadata differs.

IgnoreBucketRestrictions:   Allows the user to ignore Amazon-defined bucket name restrictions.

Bucket name restrictions are enforced by default. This configuration settings allows you to override the task's validation, and set the Bucket property with any arbitrary string. The following restrictions are enforced:

  • Bucket names can contain lowercase letters, numbers, periods, underscores, and dashes.
  • Bucket names must start with a number or letter.
  • Bucket names must be between 3 and 63 characters long.
  • Bucket names should not end with a dash.
  • Bucket names cannot contain two adjacent periods.
  • Bucket names cannot contain dashes next to periods.
  • Bucket names must not be formatted as an IP address (eg: 192.168.5.4).
IncludeObjectMD5:   Includes the MD5 digest of the object data if set.

If set to True the component will include the MD5 digest of the object data when creating an object. The server will use this value to verify the data was not corrupted during transfer. The default value is False.

MaxObjects:   The maximum number of objects for the server to return.

The maximum number of objects to be returned at a time. The default value is -1, which causes the server's default limit (1000) to be used.

ProcessAllPages:   Whether or not to continue processing if results are paged.

If set to True (default), the task will continue processing if the server indicates that there are more objects to process during a ReceiveObject, ListObjects, or DeleteObjects operation. If set to False, the task will only process the first page of results returned from the server.

SessionToken:   The session token to send in the request when using temporary credentials.

When using temporary credentials, S3 requires you to send the session token provided with the temporary access and secret key in every request.

StorageClass:   Specify the storage class to be used for the uploaded object.

When ServiceProvider is set to one of the following, this setting specifies the storage class to use for newly-created objects:

  • spAmazonS3 (0)
  • spGoogleStorage (2)

When ServiceProvider is set to spAmazonS3 (0), valid values are:

Value Storage Class
1 (default) STANDARD
2 REDUCED_REDUNDANCY
3 STANDARD_IA
4 ONEZONE_IA
5 INTELLIGENT_TIERING
6 GLACIER
7 DEEP_ARCHIVE

When ServiceProvider is set to spGoogleStorage (2), valid values are:

Value Storage Class Notes
1 (default) multiple possible Standard storage; uses MULTI_REGIONAL for multi-regional buckets, and REGIONAL for regional buckets (refer to Region for more information).
2 MULTI_REGIONAL Only valid for objects created in multi-regional buckets.
3 REGIONAL Only valid for objects created in regional buckets.
4 NEARLINE
5 COLDLINE

Refer to the appropriate service provider's documentation for more information about each storage class.

URL:   The URL to use for Other service providers.

When ServiceProvider is set to spOther (255), this setting must be set to the URL of the service provider.

UseEC2RoleCredentials:   If set to true, authenticate requests with credentials obtained from the IAM role attached to the EC2 instance.

When ServiceProvider is set to spAmazonS3 (0), and the task is running within an Amazon EC2 instance, this setting can be set to true in order to automatically authenticate requests as the IAM role attached to the instance using temporary credentials obtained from the EC2 instance itself.

In order for the task to be able to auto-obtain authentication credentials, the EC2 instance must have an "instance profile" with an appropriate IAM role attached to it. Refer to the "Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances" page in the Amazon IAM documentation for more information.

By default, this setting is false. When set to true the task will do the following before each request:

  1. Make a request against the EC2 instance to get the name of the role attached to it.
  2. Make a request against the EC2 instance to obtain temporary security credentials for IAM role returned in the previous call.
  3. Parse the response, automatically populating the AccessKey and SecretKey properties.
  4. Execute the original request using the temporary security credentials that were acquired.

When using this setting, set the AccessKey and SecretKey properties to dummy values (e.g., "DUMMY").

UseServerSideEncryption:   If true, use server side encryption.

This setting specifies whether Amazon S3 should enable server-side encryption for newly-created objects.

By default, this setting is False. This setting only applies when ServiceProvider is set to spAmazonS3 (0).

SSL Configuration Settings

ReuseSSLSession:   Determines if the SSL session is reused.

If set to true, the task will reuse the context if and only if the following criteria are met:

  • The target host name is the same.
  • The system cache entry has not expired (default timeout is 10 hours).
  • The application process that calls the function is the same.
  • The logon session is the same.
  • The instance of the task is the same.

SSLCipherStrength:   The minimum cipher strength used for bulk encryption.

This minimum cipher strength largely dependent on the security modules installed on the system. If the cipher strength specified is not supported, an error will be returned when connections are initiated.

Please note that this setting contains the minimum cipher strength requested from the security library.

Use this setting with caution. Requesting a lower cipher strength than necessary could potentially cause serious security vulnerabilities in your application.

SSLEnabledCipherSuites:   The cipher suite to be used in an SSL negotiation.

The enabled cipher suites to be used in SSL negotiation.

By default, the enabled cipher suites will include all available ciphers ("*").

The special value "*" means that the task will pick all of the supported cipher suites. If SSLEnabledCipherSuites is set to any other value, only the specified cipher suites will be considered.

Multiple cipher suites are separated by semicolons.

Example values when UseInternalSecurityAPI is False (default):

// The "Other" property could contain ONE of the following lines:
SSLEnabledCipherSuites=*
SSLEnabledCipherSuites=CALG_AES_256
SSLEnabledCipherSuites=CALG_AES_256;CALG_3DES
Possible values when UseInternalSecurityAPI is False (default) include:
  • CALG_3DES
  • CALG_3DES_112
  • CALG_AES
  • CALG_AES_128
  • CALG_AES_192
  • CALG_AES_256
  • CALG_AGREEDKEY_ANY
  • CALG_CYLINK_MEK
  • CALG_DES
  • CALG_DESX
  • CALG_DH_EPHEM
  • CALG_DH_SF
  • CALG_DSS_SIGN
  • CALG_ECDH
  • CALG_ECDH_EPHEM
  • CALG_ECDSA
  • CALG_ECMQV
  • CALG_HASH_REPLACE_OWF
  • CALG_HUGHES_MD5
  • CALG_HMAC
  • CALG_KEA_KEYX
  • CALG_MAC
  • CALG_MD2
  • CALG_MD4
  • CALG_MD5
  • CALG_NO_SIGN
  • CALG_OID_INFO_CNG_ONLY
  • CALG_OID_INFO_PARAMETERS
  • CALG_PCT1_MASTER
  • CALG_RC2
  • CALG_RC4
  • CALG_RC5
  • CALG_RSA_KEYX
  • CALG_RSA_SIGN
  • CALG_SCHANNEL_ENC_KEY
  • CALG_SCHANNEL_MAC_KEY
  • CALG_SCHANNEL_MASTER_HASH
  • CALG_SEAL
  • CALG_SHA
  • CALG_SHA1
  • CALG_SHA_256
  • CALG_SHA_384
  • CALG_SHA_512
  • CALG_SKIPJACK
  • CALG_SSL2_MASTER
  • CALG_SSL3_MASTER
  • CALG_SSL3_SHAMD5
  • CALG_TEK
  • CALG_TLS1_MASTER
  • CALG_TLS1PRF
Example values when UseInternalSecurityAPI is True:
// The "Other" property could contain ONE of the following lines:
SSLEnabledCipherSuites=*
SSLEnabledCipherSuites=TLS_DHE_DSS_WITH_AES_128_CBC_SHA
SSLEnabledCipherSuites=TLS_DHE_DSS_WITH_AES_128_CBC_SHA;TLS_DH_ANON_WITH_AES_128_CBC_SHA
Possible values when UseInternalSecurityAPI is True include:
  • TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
  • TLS_DHE_DSS_WITH_AES_256_GCM_SHA384
  • TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
  • TLS_DHE_DSS_WITH_AES_128_CBC_SHA
  • TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
  • TLS_DHE_DSS_WITH_AES_256_CBC_SHA
  • TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
  • TLS_DHE_DSS_WITH_DES_CBC_SHA
  • TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
  • TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
  • TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
  • TLS_DHE_RSA_WITH_AES_128_CBC_SHA
  • TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
  • TLS_DHE_RSA_WITH_AES_256_CBC_SHA
  • TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
  • TLS_DHE_RSA_WITH_DES_CBC_SHA
  • TLS_RSA_WITH_AES_256_GCM_SHA384
  • TLS_RSA_WITH_AES_128_GCM_SHA256
  • TLS_RSA_WITH_3DES_EDE_CBC_SHA
  • TLS_RSA_WITH_AES_128_CBC_SHA
  • TLS_RSA_WITH_AES_128_CBC_SHA256
  • TLS_RSA_WITH_AES_256_CBC_SHA
  • TLS_RSA_WITH_AES_256_CBC_SHA256
  • TLS_RSA_WITH_DES_CBC_SHA
  • TLS_RSA_WITH_RC4_128_MD5
  • TLS_RSA_WITH_RC4_128_SHA

If SSLEnabledProtocols is configured to use TLS 1.3 the following values are supported:

  • TLS_AES_128_GCM_SHA256
  • TLS_AES_256_GCM_SHA384

SSLEnabledCipherSuites is used together with SSLCipherStrength.

SSLEnabledProtocols:   Used to enable/disable the supported security protocols.

Used to enable/disable the supported security protocols.

Not all supported protocols are enabled by default (the value of this setting is 4032). If you want more granular control over the enabled protocols, you can set this property to the binary 'OR' of one or more of the following values:

TLS1.312288 (Hex 3000) (Experimental)
TLS1.23072 (Hex C00) (Default)
TLS1.1768 (Hex 300) (Default)
TLS1 192 (Hex C0) (Default)
SSL3 48 (Hex 30)
SSL2 12 (Hex 0C)

Note: TLS 1.1 and TLS1.2 support are only available starting with Windows 7.

Note: Enabling TLS 1.3 will automatically set UseInternalSecurityAPI to True.

SSLIncludeCertChain:   Whether the entire certificate chain is included in the SSLServerAuthentication event.

This setting specifies whether the transport log contains the full certificate chain. By default this value is False and only the leaf certificate will be present.

If set to True all certificates returned by the server will be present in the transport log. This includes the leaf certificate, any intermediate certificate, and the root certificate.

Note: When UseInternalSecurityAPI is set to True this value is automatically set to True. This is needed for proper validation when using the internal provider.

SSLSecurityFlags:   Flags that control certificate verification.

The following flags are defined (specified in hexadecimal notation). They can be or-ed together to exclude multiple conditions:

0x00000001Ignore time validity status of certificate.
0x00000002Ignore time validity status of CTL.
0x00000004Ignore non-nested certificate times.
0x00000010Allow unknown Certificate Authority.
0x00000020Ignore wrong certificate usage.
0x00000100Ignore unknown certificate revocation status.
0x00000200Ignore unknown CTL signer revocation status.
0x00000400Ignore unknown Certificate Authority revocation status.
0x00000800Ignore unknown Root revocation status.
0x00008000Allow test Root certificate.
0x00004000Trust test Root certificate.
0x80000000Ignore non-matching CN (certificate CN not-matching server name).

TLS12SignatureAlgorithms:   Defines the allowed TLS 1.2 signature algorithms when UseInternalSecurityAPI is True.

This setting specifies the allowed server certificate signature algorithms when UseInternalSecurityAPI is True and SSLEnabledProtocols is set to allow TLS 1.2.

When specified the task will verify that the server certificate signature algorithm is among the values specified in this setting. If the server certificate signature algorithm is unsupported the task will fail with an error.

The format of this value is a comma separated list of hash-signature combinations. For instance:

// The "Other" could contain ALL of these lines:
UseInternalSecurityAPI=true
SSLEnabledProtocols=3072
TLS12SignatureAlgorithms=sha1-rsa,sha1-dsa,sha256-rsa,sha256-dsa
The default value for this setting is "sha1-rsa,sha1-dsa,sha224-rsa,sha224-dsa,sha256-rsa,sha256-dsa,sha384-rsa,sha384-dsa,sha512-rsa,sha512-dsa".

In order to not restrict the server's certificate signature algorithm, specify an empty string as the value for this setting, which will cause the signature_algorithms TLS 1.2 extension to not be sent.

TLS12SupportedGroups:   The supported groups for ECC.

This setting specifies a comma separated list of named groups used in TLS 1.2 for ECC.

The default value is ecdhe_secp256r1,ecdhe_secp384r1,ecdhe_secp521r1.

When using TLS 1.2 and UseInternalSecurityAPI is set to True, the values refer to the supported groups for ECC. The following values are supported:

  • "ecdhe_secp256r1" (default)
  • "ecdhe_secp384r1" (default)
  • "ecdhe_secp521r1" (default)

TLS13KeyShareGroups:   The groups for which to pregenerate key shares.

This setting specifies a comma separated list of named groups used in TLS 1.3 for key exchange. The groups specified here will have key share data pregenerated locally before establishing a connection. This can prevent an additional round trip during the handshake if the group is supported by the server.

The default value is set to balance common supported groups and the computational resources required to generate key shares. As a result only some groups are included by default in this setting.

Note: All supported groups can always be used during the handshake even if not listed here, but if a group is used which is not present in this list it will incur an additional round trip and time to generate the key share for that group.

In most cases this setting does not need to be modified. This should only be modified if there is a specific reason to do so.

The default value is ecdhe_x25519,ecdhe_secp256r1,ecdhe_secp384r1,ffdhe_2048,ffdhe_3072

The values are ordered from most preferred to least preferred. The following values are supported:

  • "ecdhe_x25519" (default)
  • "ecdhe_x448"
  • "ecdhe_secp256r1" (default)
  • "ecdhe_secp384r1" (default)
  • "ecdhe_secp521r1"
  • "ffdhe_2048" (default)
  • "ffdhe_3072" (default)
  • "ffdhe_4096"
  • "ffdhe_6144"
  • "ffdhe_8192"

TLS13SignatureAlgorithms:   The allowed certificate signature algorithms.

This setting holds a comma separated list of allowed signature algorithms. Possible values are:

  • "rsa_pkcs1_sha256" (default)
  • "rsa_pkcs1_sha384" (default)
  • "rsa_pkcs1_sha512" (default)
The default value is rsa_pkcs1_sha256,rsa_pkcs1_sha384,rsa_pkcs1_sha512. This setting is only applicable when SSLEnabledProtocols includes TLS 1.3.
TLS13SupportedGroups:   The supported groups for (EC)DHE key exchange.

This setting specifies a comma separated list of named groups used in TLS 1.3 for key exchange. This setting should only be modified if there is a specific reason to do so.

The default value is ecdhe_x25519,ecdhe_x448,ecdhe_secp256r1,ecdhe_secp384r1,ecdhe_secp521r1,ffdhe_2048,ffdhe_3072,ffdhe_4096,ffdhe_6144,ffdhe_8192

The values are ordered from most preferred to least preferred. The following values are supported:

  • "ecdhe_x25519" (default)
  • "ecdhe_x448" (default)
  • "ecdhe_secp256r1" (default)
  • "ecdhe_secp384r1" (default)
  • "ecdhe_secp521r1" (default)
  • "ffdhe_2048" (default)
  • "ffdhe_3072" (default)
  • "ffdhe_4096" (default)
  • "ffdhe_6144" (default)
  • "ffdhe_8192" (default)

General Configuration Settings

AbsoluteTimeout:   Determines whether timeouts are inactivity timeouts or absolute timeouts.

If AbsoluteTimeout is set to True, any method which does not complete within Timeout seconds will be aborted. By default, AbsoluteTimeout is False, and the timeout is an inactivity timeout.

LocalHost:   The name of the local host or user-assigned IP interface through which connections are initiated or accepted.

The LocalHost configuration contains the name of the local host as obtained by the Gethostname() system call, or if the user has assigned an IP address, the value of that address.

In multihomed hosts (machines with more than one IP interface) setting LocalHost to the value of an interface will make the task initiate connections (or accept in the case of server tasks) only through that interface.

If the task is connected, the LocalHost configuration shows the IP address of the interface through which the connection is made in internet dotted format (aaa.bbb.ccc.ddd). In most cases, this is the address of the local host, except for multihomed hosts (machines with more than one IP interface).

TcpNoDelay:   Whether or not to delay when sending packets.

When true, the socket will send all data that is ready to send at once. When false, the socket will send smaller buffered packets of data at small intervals. This is known as the Nagle algorithm.

By default, this config is set to false.

UseInternalSecurityAPI:   Tells the task whether or not to use the system security libraries or an internal implementation.

By default the task will use the system security libraries to perform cryptographic functions. When set to False calls to unmanaged code will be made. In certain environments this is not desirable. To use a completely managed security implementation set this setting to True. Setting this to True tells the task to use the internal implementation instead of using the system's security API.

Note: This setting is static. The value set is applicable to all tasks used in the application.

When this value is set the product's system DLL is no longer required as a reference, as all unmanaged code is stored in that file.

Copyright (c) 2022 /n software inc. - All rights reserved.
/n software Tasks for SSIS - Version 20.0 [Build 8319]