Get-CloudStorage Cmdlet

Parameters   Output Objects   Configuration Settings  

The Get-CloudStorage cmdlet is used to download files and retrieve directory listings from Cloud Storage providers.


Get-CloudStorage [parameters]


To list a directory, set the List parameter to the remote path of the target directory. Passing "/" as the List parameter will list the root directory of the remote host.

To download files or entire directories, set the RemoteFile parameter to the name of the target file or directory. Wildcards (*) can be used to download multiple files. When downloading multiple files or a directory, the LocalFile parameter must be set to the local directory where the target files/directory should be stored. When downloading a single file, the LocalFile parameter must be set to the local path and file name to the file where the downloaded content will be stored. If the local file already exists, the Overwrite switch must be passed to overwrite the local file.

The following providers use OAuth 2.0 for authentication:

  • DropBox
  • Google Drive
  • Google Cloud Storage*
  • Microsoft OneDrive

For each of these providers, OAuthClientId and OAuthClientSecret are required to generate the OAuth token. The cmdlet will automatically fetch and cache OAuth tokens, which requires user-interaction the first time.

Once the token has been cached the cmdlet will automatically use and refresh this token without additional interaction. The OAuthCacheDir parameter governs the location of the OAuth cache files, and passing the empty string to this parameter will disable OAuth caching entirely.

To authenticate with the remaining providers, use the parameters shown in the table below:

Provider Relevant Parameters
Amazon S3 AccessKey and SecretKey
Azure Blob Account and AccessKey
Azure File Account and AccessKey
Digital Ocean Spaces AccessKey and SecretKey
Google Cloud Storage* ProjectId (if using OAuthClientId and OAuthClientSecret for OAuth); or, AccessKey and SecretKey
Hadoop DFS Multiple HTTP authentication schemes are supported; refer to AuthMechanism for more information. HadoopURL must always be set.
Wasabi AccessKey and SecretKey

*Google Cloud Storage supports two authentication methods. To authenticate with OAuth 2.0 (preferred) set the OAuthClientId, OAuthClientSecret, and ProjectId parameters. To authenticate with an access and secret key, set the AccessKey and SecretKey parameters instead.

The cmdlets support pipeline input for some of their parameters. Prebuilding an object and piping it to the cmdlet is very useful, but should be used with caution to prevent security conflicts. Steps have been taken to decrease the risk of a possibly accidental pipe to the cmdlet, for instance, the Credential parameter cannot be piped to the cmdlet and must be specified manually.

#list all buckets (the root directory) for Amazon S3
Get-CloudStorage -ServiceProvider Amazons3 -AccessKey $accesskey -SecretKey $secretkey -List "/"

#download all text files from a subfolder of Box
Get-CloudStorage -ServiceProvider Box -OAuthClientId $boxid -OAuthClientSecret $boxsecret -RemoteFile "test/*.txt" -LocalFile "C:/test"

#download a single file from Wasabi
Get-CloudStorage -ServiceProvider Wasabi -AccessKey $accesskey -SecretKey $secretkey -RemoteFile "Apple.txt" -LocalFile "C:/test/downloaded_apple.txt"

#download an entire directory from DropBox
Get-CloudStorage -ServiceProvider DropBox -OAuthClientId $dropboxid -OAuthClientSecret $dropboxsecret -RemoteFile "folder/subfolder" -LocalFile "C:/test"

Connection Handling

This cmdlet supports persistent connections through the Connection parameter. To establish a new CloudStorage connection, use the Connect-CloudStorage cmdlet. To close the connection, use the Disconnect-CloudStorage cmdlet.

Parameter List

The following is the full list of the parameters of the cmdlet with short descriptions. Click on the links for further details.

ConnectionAn already established connection.
LogFileThe location of a file to which debug information is written.
AccessKeyA storage service provider access key.
AccountA storage service provider account string.
AuthMechanismThe HTTP authentication scheme to use (for storage service providers that support multiple).
CertPasswordThe password to the certificate store.
CertStoreThe name of the certificate store for the client certificate.
CertStoreTypeThe type of certificate store for the client certificate.
CertSubjectThe subject of the certificate used for client authentication.
ConfigSpecifies one or more configuration settings.
CredentialThe PSCredential object to use for user/password authentication.
EncryptionAlgorithmThe encryption algorithm.
EncryptionPasswordThe encryption password.
ForceForces the cmdlet to accept the default behavior instead of querying the user.
HadoopURLThe URL of the Hadoop WebHDFS server.
ListGet a directory listing for the path and file mask specified.
LocalFileThe local file to download to.
LogFileThe location of a file to which debug information is written.
OAuthCacheDirThe directory where OAuth settings will be stored.
OAuthClientIdAccount credential for services that require OAuth authentication.
OAuthClientSecretAccount credential for services that require OAuth authentication.
OAuthWebServerPortThe port on the local machine where the OAuth web server will listen.
OverwriteIndicates whether the LocalFile may be overwritten.
PasswordA storage service provider password.
ProjectIdA Google Cloud Platform project Id string.
RemoteFileThe name of the remote file(s)/directory to download.
RemotePathThe path on the server.
SecretKeyA storage service provider access key.
ServiceProviderThe storage service provider.
TimeoutThe maximum time allowed for the operation.
UserA storage service provider user name.

Output Objects

The following is the full list of the output objects returned by the cmdlet with short descriptions. Click on the links for further details.

EntryInfoThis object contains details about a file returned from the Cloud Storage provider.

Configuration Settings

The following is a list of configuration settings for the cmdlet with short descriptions. Click on the links for further details.

AWSProfileThe name of the AWS CLI profile that the cmdlet should use to obtain authentication and region information.
AWSProfileDirThe location of the AWS CLI credentials and config files.
ContentDispositionThe Content-Disposition to send when uploading an file to an S3-like storage provider.
ContentTypeThe Content-Type to send when uploading an file to an S3-like storage provider.
DateFormatThe date format of the file time.
DocsExportFormatThe file format to export Google Docs files as when downloading.
DrawingsExportFormatThe file format to export Google Drawings files as when downloading.
EncryptionIVThe initialization vector to be used for encryption/decryption.
EncryptionKeyThe key to use during encryption/decryption.
FragmentSizeThe fragment size that the cmdlet should use when uploading files in multiple fragments.
ListResourcesScopeControls the scope of the results when listing GoogleDrive resources.
NewVersionOnOverwriteWhether to upload new file versions instead of performing true overwrites.
NormalizedFileName[i]The normalized file name.
RegionFor applicable providers, specifies the region to use.
SharedDriveIdThe Id of the Google Drive shared drive to list resources for.
SheetsExportFormatThe file format to export Google Sheets files as when downloading.
SimpleUploadLimitThe threshold above which the cmdlet will choose to perform a multipart upload, for applicable storage service providers.
SlidesExportFormatThe file format to export Google Slides files as when downloading.
UseVirtualHostingDetermines which URL style S3-like storage service providers should use.
AcceptEncodingUsed to tell the server which types of content encodings the client supports.
AllowHTTPCompressionThis property enables HTTP compression for receiving data.
AllowHTTPFallbackWhether HTTP/2 connections are permitted to fallback to HTTP/1.1.
AppendWhether to append data to LocalFile.
AuthorizationThe Authorization string to be sent to the server.
BytesTransferredContains the number of bytes transferred in the response data.
ChunkSizeSpecifies the chunk size in bytes when using chunked encoding.
CompressHTTPRequestSet to true to compress the body of a PUT or POST request.
EncodeURLIf set to true the URL will be encoded by the cmdlet.
FollowRedirectsDetermines what happens when the server issues a redirect.
GetOn302RedirectIf set to true the cmdlet will perform a GET on the new location.
HTTP2HeadersWithoutIndexingHTTP2 headers that should not update the dynamic header table with incremental indexing.
HTTPVersionThe version of HTTP used by the cmdlet.
IfModifiedSinceA date determining the maximum age of the desired document.
KeepAliveDetermines whether the HTTP connection is closed after completion of the request.
KerberosSPNThe Service Principal Name for the Kerberos Domain Controller.
LogLevelThe level of detail that is logged.
MaxRedirectAttemptsLimits the number of redirects that are followed in a request.
NegotiatedHTTPVersionThe negotiated HTTP version.
OtherHeadersOther headers as determined by the user (optional).
ProxyAuthorizationThe authorization string to be sent to the proxy server.
ProxyAuthSchemeThe authorization scheme to be used for the proxy.
ProxyPasswordA password if authentication is to be used for the proxy.
ProxyPortPort for the proxy server (default 80).
ProxyServerName or IP address of a proxy server (optional).
ProxyUserA user name if authentication is to be used for the proxy.
SentHeadersThe full set of headers as sent by the client.
StatusLineThe first line of the last response from the server.
TransferredDataThe contents of the last response from the server.
TransferredDataLimitThe maximum number of incoming bytes to be stored by the cmdlet.
TransferredHeadersThe full set of headers as received from the server.
TransferredRequestThe full request as sent by the client.
UseChunkedEncodingEnables or Disables HTTP chunked encoding for transfers.
UseIDNsWhether to encode hostnames to internationalized domain names.
UserAgentInformation about the user agent (browser).
ConnectionTimeoutSets a separate timeout value for establishing a connection.
FirewallAutoDetectTells the cmdlet whether or not to automatically detect and use firewall system settings, if available.
FirewallHostName or IP address of firewall (optional).
FirewallPasswordPassword to be used if authentication is to be used when connecting through the firewall.
FirewallPortThe TCP port for the FirewallHost;.
FirewallTypeDetermines the type of firewall to connect through.
FirewallUserA user name if authentication is to be used connecting through a firewall.
KeepAliveIntervalThe retry interval, in milliseconds, to be used when a TCP keep-alive packet is sent and no response is received.
KeepAliveTimeThe inactivity time in milliseconds before a TCP keep-alive packet is sent.
LingerWhen set to True, connections are terminated gracefully.
LingerTimeTime in seconds to have the connection linger.
LocalHostThe name of the local host through which connections are initiated or accepted.
LocalPortThe port in the local host where the cmdlet binds.
MaxLineLengthThe maximum amount of data to accumulate when no EOL is found.
MaxTransferRateThe transfer rate limit in bytes per second.
ProxyExceptionsListA semicolon separated list of hosts and IPs to bypass when using a proxy.
TCPKeepAliveDetermines whether or not the keep alive socket option is enabled.
TcpNoDelayWhether or not to delay when sending packets.
UseIPv6Whether to use IPv6.
AbsoluteTimeoutDetermines whether timeouts are inactivity timeouts or absolute timeouts.
FirewallDataUsed to send extra data to the firewall.
InBufferSizeThe size in bytes of the incoming queue of the socket.
OutBufferSizeThe size in bytes of the outgoing queue of the socket.
UseInternalSecurityAPITells the cmdlet whether or not to use the system security libraries or an internal implementation.

Copyright (c) 2021 /n software inc. - All rights reserved.
NetCmdlets 2020 - Version 20.0 [Build 7877]