CloudStorage Tasks
Properties Configuration Settings
The CloudStorage Task allows easy uploading and downloading to major cloud service providers.
Remarks
The CloudStorage Task provides a simple way to upload and download files from a variety of cloud storage providers. Supported services are:
- Amazon S3
- Azure Blob
- Box
- DigitalOcean
- Dropbox
- Google Drive
- OneDrive
- SharePoint Online
- Wasabi
Authorization
The following providers use OAuth 2.0 for authorization:
- Box
- Dropbox
- Google Drive
- Microsoft OneDrive
To perform OAuth authorization first open the OAuth Authorization dialog by clicking the ellipses (...) button of the OAuthAuthorization property at design time. This will open a dialog. Within the dialog specify the ClientId, ClientSecret, and CallbackURL values for the registered application.
After specifying the necessary values press the Authorize button to authorize the application. This will launch a browser and prompt the user to grant access. After access is granted the task will populate the following values:
All necessary values will be saved in the CacheLocation for use in subsequent requests by the task. The task will use the cached values and will automatically refresh the token if needed. In this way any user interaction is only required at design time, not at runtime.To authenticate with Amazon S3 set AmazonS3AccessKey and AmazonS3SecretKey.
To authenticate with Azure Blob set AzureBlobAccount and AzureBlobAccessKey.
To authenticate with DigitalOcean set DigitalOceanAccessKey and DigitalOceanSecretKey.
To authenticate with SharePoint Online set SharePointURL, SharePointUser, and SharePointPassword.
To authenticate with Wasabi set WasabiAccessKey and WasabiSecretKey.
CloudStorage Receive Task
At runtime the task will connect to the cloud storage provider and list the directory specified by RemotePath. All files that match the FileMask will be downloaded and submitted to the SSIS Package as individual messages.
In order to guarantee that files are not downloaded multiple times, you may set the DeleteMode to remove files from the server that have been downloaded. Alternatively DownloadCacheFile may be used to keep a list of files which have already been downloaded.
CloudStorage Send Task
The task will connect to the cloud storage provider whenever it has a message to send. The task will attempt to upload to the specified RemotePath. If Overwrite is set to True, the task will delete or overwrite remote files as necessary to complete the operation.
Receiver Property List
The following is the full list of the properties of the receiver task with short descriptions. Click on the links for further details.
| AmazonS3AccessKey | The Amazon S3 Access Key. |
| AmazonS3SecretKey | The Amazon S3 Secret Key. |
| AzureBlobAccessKey | The Azure Blob Access Key. |
| AzureBlobAccount | The Azure Blob Account. |
| DigitalOceanAccessKey | The DigitalOcean Access Key. |
| DigitalOceanSecretKey | The DigitalOcean Secret Key. |
| DirList | String denoting the name of the variable which contains the last directory listing entries. |
| EncryptionAlgorithm | The encryption algorithm. |
| EncryptionPassword | The encryption password. |
| FileMask | Indicates the type of files to download from the server. |
| Firewall | A set of properties related to firewall access. |
| HTTPProxy | A set of properties related to proxy access. |
| LocalDirectory | The directory on the local machine to write downloaded files to. |
| LogFile | The file to write logging information to at runtime. |
| LogMode | What information gets logged during task execution. |
| OAuthAuthorization | OAuth Authorization Information. |
| Operation | The operation which will be executed by the task. |
| Other | Defines a set of configuration settings to be used by the task. |
| Overwrite | Whether or not the task should overwrite files during transfer. |
| RemotePath | The current path on the server. |
| RuntimeLicense | Specifies the task runtime license key. |
| ServiceProvider | The cloud storage service provider. |
| SharePointPassword | The SharePoint user's password. |
| SharePointURL | The SharePoint site URL. |
| SharePointUser | The SharePoint user. |
| SSLAcceptServerCert | Instructs the task to unconditionally accept the server certificate that matches the supplied certificate. |
| SSLAcceptServerCertAcceptAny | Tells the task to accept any server certificate. |
| SSLCert | The certificate to use for client authentication during the SSL handshake. |
| Timeout | A timeout for the task. |
| WasabiAccessKey | The Wasabi Access Key. |
| WasabiSecretKey | The Wasabi Secret Key. |
Sender Property List
The following is the full list of the properties of the sender task with short descriptions. Click on the links for further details.
| AmazonS3AccessKey | The Amazon S3 Access Key. |
| AmazonS3SecretKey | The Amazon S3 Secret Key. |
| AzureBlobAccessKey | The Azure Blob Access Key. |
| AzureBlobAccount | The Azure Blob Account. |
| DigitalOceanAccessKey | The DigitalOcean Access Key. |
| DigitalOceanSecretKey | The DigitalOcean Secret Key. |
| EncryptionAlgorithm | The encryption algorithm. |
| EncryptionPassword | The encryption password. |
| Firewall | A set of properties related to firewall access. |
| HTTPProxy | A set of properties related to proxy access. |
| LocalFile | The path to a local file for upload. |
| LogFile | The file to write logging information to at runtime. |
| LogMode | What information gets logged during task execution. |
| OAuthAuthorization | OAuth Authorization Information. |
| Other | Defines a set of configuration settings to be used by the task. |
| Overwrite | Whether or not the task should overwrite files during transfer. |
| RemoteFile | The name of the remote file. |
| RemotePath | The current path on the server. |
| RuntimeLicense | Specifies the task runtime license key. |
| ServiceProvider | The cloud storage service provider. |
| SharePointPassword | The SharePoint user's password. |
| SharePointURL | The SharePoint site URL. |
| SharePointUser | The SharePoint user. |
| SSLAcceptServerCert | Instructs the task to unconditionally accept the server certificate that matches the supplied certificate. |
| SSLAcceptServerCertAcceptAny | Tells the task to accept any server certificate. |
| SSLCert | The certificate to use for client authentication during the SSL handshake. |
| Timeout | A timeout for the task. |
| WasabiAccessKey | The Wasabi Access Key. |
| WasabiSecretKey | The Wasabi Secret Key. |
Configuration Settings
The following is a list of configuration settings for the task with short descriptions. Click on the links for further details.
| DateFormat | The date format of the file time. |
| DownloadCacheFile | When set, only new files will be downloaded from the server. |
| DownloadCacheFileDuration | The number of minutes that a file name will remain in the download cache. |
| EnableFileDetailsComparison | Whether the receive task should consider metadata differences when deciding whether to download a file again. |
| LocalFile | The name of the LocalFile to which the remote file will be saved. |
| DocsExportFormat | The file format to export Google Docs files as when downloading. |
| DrawingsExportFormat | The file format to export Google Drawings files as when downloading. |
| SheetsExportFormat | The file format to export Google Sheets files as when downloading. |
| SlidesExportFormat | The file format to export Google Slides files as when downloading. |
| ReuseSSLSession | Determines if the SSL session is reused. |
| SSLCipherStrength | The minimum cipher strength used for bulk encryption. |
| SSLEnabledProtocols | Used to enable/disable the supported security protocols. |
| SSLIncludeCertChain | Whether the entire certificate chain is included in the SSLServerAuthentication event. |
| SSLSecurityFlags | Flags that control certificate verification. |
| SSLEnabledCipherSuites | The cipher suite to be used in an SSL negotiation. |
| TLS12SignatureAlgorithms | Defines the allowed TLS 1.2 signature algorithms when UseInternalSecurityAPI is True. |
| TLSNamedGroups | The supported (EC)DHE groups. |
| TLS13SignatureAlgorithms | The allowed certificate signature algorithms. |
| AbsoluteTimeout | Determines whether timeouts are inactivity timeouts or absolute timeouts. |
| LocalHost | The name of the local host or user-assigned IP interface through which connections are initiated or accepted. |
| TcpNoDelay | Whether or not to delay when sending packets. |
| UseInternalSecurityAPI | Tells the task whether or not to use the system security libraries or an internal implementation. |
| PersistedConnectionTimeout | How long to leave a persisted connection alive. |