S3 Tasks
Properties Configuration Settings
The S3 Component provides access to Amazon's Simple Storage Service (S3), as well as other S3-like services.
Remarks
The S3 Task allows you to access Amazon's Simple Storage Service (S3), and other S3-like services, securely over an SSL connection. S3 services allow you to store arbitrary data objects in buckets and access them from anywhere using an Internet connection. Please refer to the Amazon S3 documentation (or the documentation for another S3-like service provider) for additional details.
To select which S3 service provider the S3 Task should communicate with, use the ServiceProvider property. Possible values are:
- spAmazonS3 (0) (default): Amazon S3
- spDigitalOcean (1): Digital Ocean Spaces
- spGoogleStorage (2): Google Cloud Storage
- spWasabi (3): Wasabi
- spBackblazeB2 (4): Backblaze B2
- spIBM (7): IBM Cloud Object Storage
- spOracle (8): Oracle Cloud Object Storage
- spLinode (9): Linode Object Storage
- spOther (255): Other Provider
Authentication
You will need to sign up for an account with the desired S3 service and obtain an access key and secret key for it. Once you have a valid access key and secret key, use them to set the AccessKey and SecretKey properties. Once this is done, you may begin uploading/downloading files to and from your buckets.
Note: AccessKey and SecretKey must be set before attempting to use the task, unless ServiceProvider is set to spGoogleStorage (2) and you wish to authenticate using OAuth 2.0; refer to the OAuthAuthorization and GoogleProjectId properties for more information.
S3 Receive Task
The S3 Receive Task will download objects specified by the ObjectMask property contained within the specified Bucket.
S3 Send Task
The S3 Send Task requires a Bucket and a ObjectKey to specify the location to upload files to.
Receiver Property List
The following is the full list of the properties of the receiver task with short descriptions. Click on the links for further details.
AccessKey | The access key to use for authentication. |
Bucket | The name of the bucket to make requests against. |
BucketList | String denoting the name of the variable which contains the last bucket listing entries. |
EncryptionAlgorithm | The encryption algorithm. |
EncryptionPassword | The encryption password. |
Firewall | A set of properties related to firewall access. |
GoogleProjectId | A Google Cloud Platform project Id string. |
HTTPProxy | A set of properties related to proxy access. |
LocalDirectory | The directory on the local machine to write downloaded objects to. |
LogFile | The file to write logging information to at runtime. |
LogMode | What information gets logged during component execution. |
OAuthAuthorization | OAuth Authorization Information. |
ObjectDelimiter | The delimiter string to use when listing objects. |
ObjectList | String denoting the name of the variable which contains the last object listing entries. |
ObjectMask | Indicates the type of files to download from the S3 server. |
ObjectPrefix | A prefix used to restrict the results returned when listing objects. |
Operation | The operation which will be executed by the S3 task. |
OracleNamespace | The Oracle Cloud Object Storage namespace string. |
Other | Defines a set of configuration settings to be used by the component. |
Overwrite | Whether or not the component should overwrite files during transfer. |
Region | The region the component will make requests against. |
RuntimeLicense | Specifies the component runtime license key. |
SecretKey | The secret key to use for authentication. |
ServiceProvider | The S3 service provider. |
SSLAcceptServerCert | Instructs the component to unconditionally accept the server certificate that matches the supplied certificate. |
SSLAcceptServerCertAcceptAny | Tells the component to accept any server certificate. |
SSLCert | The certificate to use for client authentication during the SSL handshake. |
Timeout | A timeout for the component. |
UseSSL | Whether to use SSL/TLS when connecting. |
UseVirtualHosting | Determines which style request to use. |
Sender Property List
The following is the full list of the properties of the sender task with short descriptions. Click on the links for further details.
AccessKey | The access key to use for authentication. |
AccessPolicy | The canned access policy to apply when uploading objects. |
Bucket | The name of the bucket to make requests against. |
ContentDisposition | Content disposition to send when uploading objects. |
ContentType | Content type to send when uploading objects. |
EncryptionAlgorithm | The encryption algorithm. |
EncryptionPassword | The encryption password. |
Firewall | A set of properties related to firewall access. |
GoogleProjectId | A Google Cloud Platform project Id string. |
HTTPProxy | A set of properties related to proxy access. |
LocalFile | The path to a local file for upload. |
LogFile | The file to write logging information to at runtime. |
LogMode | What information gets logged during component execution. |
OAuthAuthorization | OAuth Authorization Information. |
ObjectKey | This is the name of file being uploaded to the server. |
OracleNamespace | The Oracle Cloud Object Storage namespace string. |
Other | Defines a set of configuration settings to be used by the component. |
Region | The region the component will make requests against. |
RuntimeLicense | Specifies the component runtime license key. |
SecretKey | The secret key to use for authentication. |
ServiceProvider | The S3 service provider. |
SSLAcceptServerCert | Instructs the component to unconditionally accept the server certificate that matches the supplied certificate. |
SSLAcceptServerCertAcceptAny | Tells the component to accept any server certificate. |
SSLCert | The certificate to use for client authentication during the SSL handshake. |
Timeout | A timeout for the component. |
UseSSL | Whether to use SSL/TLS when connecting. |
UseVirtualHosting | Determines which style request to use. |
Configuration Settings
The following is a list of configuration settings for the task with short descriptions. Click on the links for further details.
DirDelimiter | The delimiter for subfolders on the S3 server that should not be maintained in the local directory. |
DownloadCacheFile | When set, only new files will be downloaded from the server. |
DownloadCacheFileDuration | The number of minutes that a file name will remain in the download cache. |
EnableFileDetailsComparison | Whether the receive component should consider metadata differences when deciding whether to download a file again. |
IgnoreBucketRestrictions | Allows the user to ignore Amazon-defined bucket name restrictions. |
IncludeObjectMD5 | Includes the MD5 digest of the object data if set. |
MaxObjects | The maximum number of objects for the server to return. |
ProcessAllPages | Whether or not to continue processing if results are paged. |
SessionToken | The session token to send in the request when using temporary credentials. |
StorageClass | Specify the storage class to be used for the uploaded object. |
URL | The URL to use for Other service providers. |
UseEC2RoleCredentials | If set to true, authenticate requests with credentials obtained from the IAM role attached to the EC2 instance. |
UseServerSideEncryption | If true, use server side encryption. |
ReuseSSLSession | Determines if the SSL session is reused. |
SSLCipherStrength | The minimum cipher strength used for bulk encryption. |
SSLEnabledCipherSuites | The cipher suite to be used in an SSL negotiation. |
SSLEnabledProtocols | Used to enable/disable the supported security protocols. |
SSLIncludeCertChain | Whether the entire certificate chain is included in the SSLServerAuthentication event. |
SSLSecurityFlags | Flags that control certificate verification. |
TLS12SignatureAlgorithms | Defines the allowed TLS 1.2 signature algorithms when UseInternalSecurityAPI is True. |
TLS12SupportedGroups | The supported groups for ECC. |
TLS13KeyShareGroups | The groups for which to pregenerate key shares. |
TLS13SignatureAlgorithms | The allowed certificate signature algorithms. |
TLS13SupportedGroups | The supported groups for (EC)DHE key exchange. |
AbsoluteTimeout | Determines whether timeouts are inactivity timeouts or absolute timeouts. |
LocalHost | The name of the local host or user-assigned IP interface through which connections are initiated or accepted. |
TcpNoDelay | Whether or not to delay when sending packets. |
UseInternalSecurityAPI | Tells the component whether or not to use the system security libraries or an internal implementation. |