HadoopDFS Class

Properties   Methods   Events   Config Settings   Errors  

The HadoopDFS class provides easy access to files stored in HDFS clusters.

Syntax

HadoopDFS

Remarks

The HadoopDFS class offers an easy-to-use API compatible with any Hadoop distributed file system (HDFS) cluster that exposes Hadoop's standard WebHDFS REST API. Capabilities include uploading and downloading files, strong encryption support, creating folders, file manipulation and organization, and more.

Authentication

First, set the URL property to the base WebHDFS URL of the server (see URL for more details).

Depending on how the server is configured, there are a few different authentication mechanisms that might be used; or, the server might not require authentication at all). Refer to the AuthMechanism property for more information about configuring the class to authenticate correctly.

Addressing Resources

HDFS addresses resources (files, directories, and symlinks) using Linux-style absolute paths. Unless otherwise specified, the class always works in terms of absolute paths, and will always prepend a forward slash (/) to any path passed to it that does not already start with one.

Listing Directory Contents

ListResources lists resources (files, directories, and symlinks) within the specified directory. Calling this method will fire the ResourceList event once for each resource, and will also populate the Resources properties. // ResourceList event handler. hdfs.OnResourceList += (s, e) => { Console.WriteLine(e.Name); }; hdfs.ListResources("/work_files/serious_business/cats"); for (int i = 0; i < hdfs.Resources.Count; i++) { // Process resources here. }

Downloading Files

The DownloadFile method downloads files.

If a stream has been specified using SetDownloadStream, the file data will be sent through it. If a stream is not specified, and LocalFile is set, the file will be saved to the specified location; otherwise, the file data will be held by ResourceData.

To download and decrypt an encrypted file, set EncryptionAlgorithm and EncryptionPassword before calling this method.

Download Notes

In the simplest use-case, downloading a file looks like this: hdfs.LocalFile = "../MyFile.zip"; hdfs.DownloadFile(hdfs.Resources[0].Path);

Resuming Downloads

The class also supports resuming failed downloads by using the StartByte property. If a download is interrupted, set StartByte to the appropriate offset before calling this method to resume the download. string downloadFile = "../MyFile.zip"; hdfs.LocalFile = downloadFile; hdfs.DownloadFile(hdfs.Resources[0].Path); //The transfer is interrupted and DownloadFile() above fails. Later, resume the download: //Get the size of the partially downloaded file hdfs.StartByte = new FileInfo(downloadFile).Length; hdfs.DownloadFile(hdfs.Resources[0].Path);

Resuming Encrypted File Downloads

Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt.

If LocalFile is set when beginning an encrypted download, the class creates a temporary file in TempPath to hold the encrypted data until the download is complete. If the download is interrupted, DownloadTempFile will be populated with the path of the temporary file that holds the partial data.

To resume, DownloadTempFile must be populated, along with StartByte, to allow the remainder of the encrypted data to be downloaded. Once the encrypted data is downloaded it will be decrypted and written to LocalFile. hdfs.LocalFile = "../MyFile.zip"; hdfs.EncryptionPassword = "password"; hdfs.DownloadFile(hdfs.Resources[0].Path); //The transfer is interrupted and DownloadFile() above fails. Later, resume the download: //Get the size of the partially download temp file hdfs.StartByte = new FileInfo(hdfs.Config("DownloadTempFile")).Length; hdfs.DownloadFile(hdfs.Resources[0].Path);

Uploading Files

The UploadFile method uploads new files.

If SetUploadStream has been used to set an upload stream, it will take priority as the file data source. If LocalFile is set the file will be uploaded from the specified path. If LocalFile is not set the data in ResourceData will be used.

To encrypt the file before uploading it, set EncryptionAlgorithm and EncryptionPassword. hdfs.LocalFile = "../MyFile.zip"; hdfs.UploadFile("/MyFile.zip");

Additional Functionality

The HadoopDFS class offers advanced functionality beyond simple uploads and downloads. For instance:

• Encrypt and decrypt files using the EncryptionAlgorithm and EncryptionPassword properties.
• Basic file and folder manipulation and organization using methods such as AppendFile, DeleteResource, MakeDirectory, MoveResource, and TruncateFile.
• Advanced file and directory manipulation with SetFileReplication, SetOwner, SetPermission, and SetTimes.
• Retrieval of both general file/directory information, as well as directory quota information, using GetResourceInfo and GetDirSummary.
• Execute any arbitrary WebHDFS operation with ease using the DoCustomOp method.
• And more!

Property List

The following is the full list of the properties of the class with short descriptions. Click on the links for further details.

Method List

The following is the full list of the methods of the class with short descriptions. Click on the links for further details.

Event List

The following is the full list of the events fired by the class with short descriptions. Click on the links for further details.

Config Settings

The following is a list of config settings for the class with short descriptions. Click on the links for further details.

AuthMechanism Property (HadoopDFS Class)

The authentication mechanism to use when connecting to the server.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetAuthMechanism();
int SetAuthMechanism(int iAuthMechanism);

Unicode (Windows)
INT GetAuthMechanism();
INT SetAuthMechanism(INT iAuthMechanism);

AM_NONE(0), 
AM_SIMPLE(1), 
AM_BASIC(2), 
AM_NTLM(3), 
AM_NEGOTIATE(4), 
AM_OAUTH(5)

int cloudstorage_hadoopdfs_getauthmechanism(void* lpObj);
int cloudstorage_hadoopdfs_setauthmechanism(void* lpObj, int iAuthMechanism);

int GetAuthMechanism();
int SetAuthMechanism(int iAuthMechanism);

Default Value

0

Remarks

This property controls what authentication mechanism the class should use when connecting to the server.

Possible values are:

When set to 1 (amSimple), the value of the User property is automatically sent in every request using the user.name query parameter.

When set to 2 (amBasic), 3 (amNTLM), or 4 (amNegotiate), the values held by the User and Password properties will be used to perform Basic, NTLM, or Negotiate (e.g., Kerberos SPNEGO) authentication.

When set to 5 (amOAuth), the value of the Authorization property is automatically sent in every request using the Authorization HTTP header.

Data Type

Integer

Authorization Property (HadoopDFS Class)

OAuth 2.0 Authorization Token.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetAuthorization();
int SetAuthorization(const char* lpszAuthorization);

Unicode (Windows)
LPWSTR GetAuthorization();
INT SetAuthorization(LPCWSTR lpszAuthorization);

char* cloudstorage_hadoopdfs_getauthorization(void* lpObj);
int cloudstorage_hadoopdfs_setauthorization(void* lpObj, const char* lpszAuthorization);

QString GetAuthorization();
int SetAuthorization(QString qsAuthorization);

Default Value

""

Remarks

This class supports authentication via OAuth 2.0. First, perform OAuth authentication using the OAuth property, using the OAuth class or a separate process. If using the OAuth property, then the Authorization property will not be used.

Bearer ACCESS_TOKEN

Refer to AuthMechanism for more information.

Data Type

String

DirSummary Property (HadoopDFS Class)

Directory content summary information.

Syntax

• C++
• C
• Qt

CloudStorageHDFSDirSummary* GetDirSummary();

int64 cloudstorage_hadoopdfs_getdirsummarydircount(void* lpObj);

int64 cloudstorage_hadoopdfs_getdirsummaryfilecount(void* lpObj);

int64 cloudstorage_hadoopdfs_getdirsummarynamequota(void* lpObj);

int64 cloudstorage_hadoopdfs_getdirsummarysize(void* lpObj);

int64 cloudstorage_hadoopdfs_getdirsummaryspacequota(void* lpObj);

int64 cloudstorage_hadoopdfs_getdirsummaryspaceused(void* lpObj);

int64 cloudstorage_hadoopdfs_getdirsummarystoragequota(void* lpObj);

int cloudstorage_hadoopdfs_getdirsummarystoragequotacount(void* lpObj);

int cloudstorage_hadoopdfs_getdirsummarystoragequotaindex(void* lpObj);

char* cloudstorage_hadoopdfs_getdirsummarystoragequotatype(void* lpObj);

int64 cloudstorage_hadoopdfs_getdirsummarystoragequotaused(void* lpObj);

qint64 GetDirSummaryDirCount();

qint64 GetDirSummaryFileCount();

qint64 GetDirSummaryNameQuota();

qint64 GetDirSummarySize();

qint64 GetDirSummarySpaceQuota();

qint64 GetDirSummarySpaceUsed();

qint64 GetDirSummaryStorageQuota();

int GetDirSummaryStorageQuotaCount();

int GetDirSummaryStorageQuotaIndex();

QString GetDirSummaryStorageQuotaType();

qint64 GetDirSummaryStorageQuotaUsed();

Remarks

This property holds an HDFSDirSummary object with information about a directory's content summary; it's populated after calling GetDirSummary.

This property is read-only and not available at design time.

Data Type

CloudStorageHDFSDirSummary

EncryptionAlgorithm Property (HadoopDFS Class)

The encryption algorithm.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetEncryptionAlgorithm();
int SetEncryptionAlgorithm(int iEncryptionAlgorithm);

Unicode (Windows)
INT GetEncryptionAlgorithm();
INT SetEncryptionAlgorithm(INT iEncryptionAlgorithm);

EA_AES(0), 
EA_BLOWFISH(1), 
EA_CAST(2), 
EA_DES(3), 
EA_IDEA(4), 
EA_RC2(5), 
EA_RC4(6), 
EA_TEA(7), 
EA_TRIPLE_DES(8), 
EA_TWOFISH(9), 
EA_RIJNDAEL(10), 
EA_CHA_CHA(11), 
EA_XSALSA_20(12)

int cloudstorage_hadoopdfs_getencryptionalgorithm(void* lpObj);
int cloudstorage_hadoopdfs_setencryptionalgorithm(void* lpObj, int iEncryptionAlgorithm);

int GetEncryptionAlgorithm();
int SetEncryptionAlgorithm(int iEncryptionAlgorithm);

Default Value

0

Remarks

This property specifies the encryption algorithm to be used. The maximum allowable key size is automatically used for the selected algorithm. Possible values are:

Data Type

Integer

EncryptionPassword Property (HadoopDFS Class)

The encryption password.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetEncryptionPassword();
int SetEncryptionPassword(const char* lpszEncryptionPassword);

Unicode (Windows)
LPWSTR GetEncryptionPassword();
INT SetEncryptionPassword(LPCWSTR lpszEncryptionPassword);

char* cloudstorage_hadoopdfs_getencryptionpassword(void* lpObj);
int cloudstorage_hadoopdfs_setencryptionpassword(void* lpObj, const char* lpszEncryptionPassword);

QString GetEncryptionPassword();
int SetEncryptionPassword(QString qsEncryptionPassword);

Default Value

""

Remarks

If this property is populated when UploadFile or DownloadFile is called, the class will attempt to encrypt or decrypt the data before uploading or after downloading it.

The class uses the value specified here to generate the necessary encryption Key and IV values using the PKCS5 password digest algorithm. This provides a simpler alternative to creating and managing Key and IV values directly.

However, it is also possible to explicitly specify the Key and IV values to use by setting the EncryptionKey and EncryptionIV configuration settings. This may be necessary if, e.g., the data needs to be encrypted/decrypted by another utility which generates Key and IV values differently.

This property is not available at design time.

Data Type

String

Firewall Property (HadoopDFS Class)

A set of properties related to firewall access.

Syntax

• C++
• C
• Qt

CloudStorageFirewall* GetFirewall();
int SetFirewall(CloudStorageFirewall* val);

int cloudstorage_hadoopdfs_getfirewallautodetect(void* lpObj);
int cloudstorage_hadoopdfs_setfirewallautodetect(void* lpObj, int bFirewallAutoDetect);

int cloudstorage_hadoopdfs_getfirewalltype(void* lpObj);
int cloudstorage_hadoopdfs_setfirewalltype(void* lpObj, int iFirewallType);

char* cloudstorage_hadoopdfs_getfirewallhost(void* lpObj);
int cloudstorage_hadoopdfs_setfirewallhost(void* lpObj, const char* lpszFirewallHost);

char* cloudstorage_hadoopdfs_getfirewallpassword(void* lpObj);
int cloudstorage_hadoopdfs_setfirewallpassword(void* lpObj, const char* lpszFirewallPassword);

int cloudstorage_hadoopdfs_getfirewallport(void* lpObj);
int cloudstorage_hadoopdfs_setfirewallport(void* lpObj, int iFirewallPort);

char* cloudstorage_hadoopdfs_getfirewalluser(void* lpObj);
int cloudstorage_hadoopdfs_setfirewalluser(void* lpObj, const char* lpszFirewallUser);

bool GetFirewallAutoDetect();
int SetFirewallAutoDetect(bool bFirewallAutoDetect);

int GetFirewallType();
int SetFirewallType(int iFirewallType);

QString GetFirewallHost();
int SetFirewallHost(QString qsFirewallHost);

QString GetFirewallPassword();
int SetFirewallPassword(QString qsFirewallPassword);

int GetFirewallPort();
int SetFirewallPort(int iFirewallPort);

QString GetFirewallUser();
int SetFirewallUser(QString qsFirewallUser);

Remarks

This is a Firewall-type property, which contains fields describing the firewall through which the class will attempt to connect.

Data Type

CloudStorageFirewall

Idle Property (HadoopDFS Class)

The current status of the class.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetIdle();

Unicode (Windows)
BOOL GetIdle();

int cloudstorage_hadoopdfs_getidle(void* lpObj);

bool GetIdle();

Default Value

TRUE

Remarks

This property will be False if the component is currently busy (communicating or waiting for an answer), and True at all other times.

This property is read-only.

Data Type

Boolean

LocalFile Property (HadoopDFS Class)

The location of the local file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetLocalFile();
wchar_t* GetLocalFile_W(); // Windows only
int SetLocalFile(const char* lpszLocalFile);
int SetLocalFile(const wchar_t* lpszLocalFile); // Windows only

Unicode (Windows)
LPWSTR GetLocalFile();
INT SetLocalFile(LPCWSTR lpszLocalFile);

char* cloudstorage_hadoopdfs_getlocalfile(void* lpObj);
wchar_t* cloudstorage_hadoopdfs_getlocalfile_W(void* lpObj); // Windows only
int cloudstorage_hadoopdfs_setlocalfile(void* lpObj, const char* lpszLocalFile);
int cloudstorage_hadoopdfs_setlocalfile(void* lpObj, const wchar_t* lpszLocalFile); // Windows only

QString GetLocalFile();
int SetLocalFile(QString qsLocalFile);

Default Value

""

Remarks

This property specifies the location of a file on disk. This is used as the source file when calling UploadFile or AppendFile, and as the destination file when calling DownloadFile or DoCustomOp.

Note: Setting this property to a non-empty value will discard any streams set using SetDownloadStream and SetUploadStream. Similarly, passing a non-null value to either of the aforementioned methods will clear this property.

Data Type

String

LocalHost Property (HadoopDFS Class)

The name of the local host or user-assigned IP interface through which connections are initiated or accepted.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetLocalHost();
int SetLocalHost(const char* lpszLocalHost);

Unicode (Windows)
LPWSTR GetLocalHost();
INT SetLocalHost(LPCWSTR lpszLocalHost);

char* cloudstorage_hadoopdfs_getlocalhost(void* lpObj);
int cloudstorage_hadoopdfs_setlocalhost(void* lpObj, const char* lpszLocalHost);

QString GetLocalHost();
int SetLocalHost(QString qsLocalHost);

Default Value

""

Remarks

This property contains the name of the local host as obtained by the gethostname() system call, or if the user has assigned an IP address, the value of that address.

In multihomed hosts (machines with more than one IP interface) setting LocalHost to the IP address of an interface will make the class initiate connections (or accept in the case of server classs) only through that interface. It is recommended to provide an IP address rather than a hostname when setting this property to ensure the desired interface is used.

If the class is connected, the LocalHost property shows the IP address of the interface through which the connection is made in internet dotted format (aaa.bbb.ccc.ddd). In most cases, this is the address of the local host, except for multihomed hosts (machines with more than one IP interface).

Note: LocalHost is not persistent. You must always set it in code, and never in the property window.

Data Type

String

OtherHeaders Property (HadoopDFS Class)

Other headers as determined by the user (optional).

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetOtherHeaders();
int SetOtherHeaders(const char* lpszOtherHeaders);

Unicode (Windows)
LPWSTR GetOtherHeaders();
INT SetOtherHeaders(LPCWSTR lpszOtherHeaders);

char* cloudstorage_hadoopdfs_getotherheaders(void* lpObj);
int cloudstorage_hadoopdfs_setotherheaders(void* lpObj, const char* lpszOtherHeaders);

QString GetOtherHeaders();
int SetOtherHeaders(QString qsOtherHeaders);

Default Value

""

Remarks

This property can be set to a string of headers to be appended to the HTTP request headers created from other properties like ContentType and From.

The headers must follow the format Header: Value as described in the HTTP specifications. Header lines should be separated by CRLF ("\r\n") .

Use this property with caution. If this property contains invalid headers, HTTP requests may fail.

This property is useful for extending the functionality of the class beyond what is provided.

This property is not available at design time.

Data Type

String

Overwrite Property (HadoopDFS Class)

Whether to overwrite the local or remote file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetOverwrite();
int SetOverwrite(int bOverwrite);

Unicode (Windows)
BOOL GetOverwrite();
INT SetOverwrite(BOOL bOverwrite);

int cloudstorage_hadoopdfs_getoverwrite(void* lpObj);
int cloudstorage_hadoopdfs_setoverwrite(void* lpObj, int bOverwrite);

bool GetOverwrite();
int SetOverwrite(bool bOverwrite);

Default Value

FALSE

Remarks

When calling DownloadFile, this property determines if LocalFile should be overwritten if it already exists.

When calling UploadFile, this property determines if the remote file should be overwritten if it already exists.

Data Type

Boolean

ParsedHeaders Property (HadoopDFS Class)

This property includes a collection of headers returned from the last request.

Syntax

• C++
• C
• Qt

CloudStorageList<CloudStorageHeader>* GetParsedHeaders();

int cloudstorage_hadoopdfs_getparsedheadercount(void* lpObj);

char* cloudstorage_hadoopdfs_getparsedheaderfield(void* lpObj, int parsedheaderindex);

char* cloudstorage_hadoopdfs_getparsedheadervalue(void* lpObj, int parsedheaderindex);

int GetParsedHeaderCount();

QString GetParsedHeaderField(int iParsedHeaderIndex);

QString GetParsedHeaderValue(int iParsedHeaderIndex);

Remarks

This property contains a collection of headers returned from the last request. Whenever headers are returned from the server, the headers are parsed into a collection of headers. Each Header in this collection contains information describing that header.

MaxHeaders can be used to control the maximum number of headers saved.

This property is read-only and not available at design time.

Data Type

CloudStorageHeader

Password Property (HadoopDFS Class)

The password to use for authentication.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetPassword();
int SetPassword(const char* lpszPassword);

Unicode (Windows)
LPWSTR GetPassword();
INT SetPassword(LPCWSTR lpszPassword);

char* cloudstorage_hadoopdfs_getpassword(void* lpObj);
int cloudstorage_hadoopdfs_setpassword(void* lpObj, const char* lpszPassword);

QString GetPassword();
int SetPassword(QString qsPassword);

Default Value

""

Remarks

This property specifies the password to user for authentication.

Refer to AuthMechanism for more information.

This property is not available at design time.

Data Type

String

Proxy Property (HadoopDFS Class)

A set of properties related to proxy access.

Syntax

• C++
• C
• Qt

CloudStorageProxy* GetProxy();
int SetProxy(CloudStorageProxy* val);

int cloudstorage_hadoopdfs_getproxyauthscheme(void* lpObj);
int cloudstorage_hadoopdfs_setproxyauthscheme(void* lpObj, int iProxyAuthScheme);

int cloudstorage_hadoopdfs_getproxyautodetect(void* lpObj);
int cloudstorage_hadoopdfs_setproxyautodetect(void* lpObj, int bProxyAutoDetect);

char* cloudstorage_hadoopdfs_getproxypassword(void* lpObj);
int cloudstorage_hadoopdfs_setproxypassword(void* lpObj, const char* lpszProxyPassword);

int cloudstorage_hadoopdfs_getproxyport(void* lpObj);
int cloudstorage_hadoopdfs_setproxyport(void* lpObj, int iProxyPort);

char* cloudstorage_hadoopdfs_getproxyserver(void* lpObj);
int cloudstorage_hadoopdfs_setproxyserver(void* lpObj, const char* lpszProxyServer);

int cloudstorage_hadoopdfs_getproxyssl(void* lpObj);
int cloudstorage_hadoopdfs_setproxyssl(void* lpObj, int iProxySSL);

char* cloudstorage_hadoopdfs_getproxyuser(void* lpObj);
int cloudstorage_hadoopdfs_setproxyuser(void* lpObj, const char* lpszProxyUser);

int GetProxyAuthScheme();
int SetProxyAuthScheme(int iProxyAuthScheme);

bool GetProxyAutoDetect();
int SetProxyAutoDetect(bool bProxyAutoDetect);

QString GetProxyPassword();
int SetProxyPassword(QString qsProxyPassword);

int GetProxyPort();
int SetProxyPort(int iProxyPort);

QString GetProxyServer();
int SetProxyServer(QString qsProxyServer);

int GetProxySSL();
int SetProxySSL(int iProxySSL);

QString GetProxyUser();
int SetProxyUser(QString qsProxyUser);

Remarks

This property contains fields describing the proxy through which the class will attempt to connect.

Data Type

CloudStorageProxy

QueryParams Property (HadoopDFS Class)

Additional query parameters to be included in the request.

Syntax

• C++
• C
• Qt

CloudStorageList<CloudStorageQueryParam>* GetQueryParams();
int SetQueryParams(CloudStorageList<CloudStorageQueryParam>* val);

int cloudstorage_hadoopdfs_getqueryparamcount(void* lpObj);
int cloudstorage_hadoopdfs_setqueryparamcount(void* lpObj, int iQueryParamCount);

char* cloudstorage_hadoopdfs_getqueryparamname(void* lpObj, int queryparamindex);
int cloudstorage_hadoopdfs_setqueryparamname(void* lpObj, int queryparamindex, const char* lpszQueryParamName);

char* cloudstorage_hadoopdfs_getqueryparamvalue(void* lpObj, int queryparamindex);
int cloudstorage_hadoopdfs_setqueryparamvalue(void* lpObj, int queryparamindex, const char* lpszQueryParamValue);

int GetQueryParamCount();
int SetQueryParamCount(int iQueryParamCount);

QString GetQueryParamName(int iQueryParamIndex);
int SetQueryParamName(int iQueryParamIndex, QString qsQueryParamName);

QString GetQueryParamValue(int iQueryParamIndex);
int SetQueryParamValue(int iQueryParamIndex, QString qsQueryParamValue);

Remarks

This is a collection of query parameters that will be added to the request. Parameters can be added via the AddQueryParam method.

Data Type

CloudStorageQueryParam

ReadBytes Property (HadoopDFS Class)

The number of bytes to read when downloading a file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int64 GetReadBytes();
int SetReadBytes(int64 lReadBytes);

Unicode (Windows)
LONG64 GetReadBytes();
INT SetReadBytes(LONG64 lReadBytes);

int64 cloudstorage_hadoopdfs_getreadbytes(void* lpObj);
int cloudstorage_hadoopdfs_setreadbytes(void* lpObj, int64 lReadBytes);

qint64 GetReadBytes();
int SetReadBytes(qint64 lReadBytes);

Default Value

-1

Remarks

This property specifies how many bytes should be read when DownloadFile is called. It can be used in tandem with StartByte to specify a specific range of the file to download.

If set to -1 (default), there is no limit on how many bytes will be read.

This property is not available at design time.

Data Type

Long64

ResourceData Property (HadoopDFS Class)

The data that was downloaded, or that should be uploaded.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetResourceData(char* &lpResourceData, int &lenResourceData);
int SetResourceData(const char* lpResourceData, int lenResourceData);

Unicode (Windows)
INT GetResourceData(LPSTR &lpResourceData, INT &lenResourceData);
INT SetResourceData(LPCSTR lpResourceData, INT lenResourceData);

int cloudstorage_hadoopdfs_getresourcedata(void* lpObj, char** lpResourceData, int* lenResourceData);
int cloudstorage_hadoopdfs_setresourcedata(void* lpObj, const char* lpResourceData, int lenResourceData);

QByteArray GetResourceData();
int SetResourceData(QByteArray qbaResourceData);

Default Value

""

Remarks

This property is populated with file data after calling DownloadFile if SetDownloadStream and LocalFile are not set.

This property can also be set before calling UploadFile; its data will be uploaded if SetUploadStream and LocalFile are not set.

This property is not available at design time.

Data Type

Binary String

Resources Property (HadoopDFS Class)

A collection of resources.

Syntax

• C++
• C
• Qt

CloudStorageList<CloudStorageHDFSResource>* GetResources();

int cloudstorage_hadoopdfs_getresourcecount(void* lpObj);

int64 cloudstorage_hadoopdfs_getresourceaccesstime(void* lpObj, int resourceindex);

int64 cloudstorage_hadoopdfs_getresourceblocksize(void* lpObj, int resourceindex);

int64 cloudstorage_hadoopdfs_getresourcechildcount(void* lpObj, int resourceindex);

char* cloudstorage_hadoopdfs_getresourcegroup(void* lpObj, int resourceindex);

int64 cloudstorage_hadoopdfs_getresourcemodifiedtime(void* lpObj, int resourceindex);

char* cloudstorage_hadoopdfs_getresourcename(void* lpObj, int resourceindex);

char* cloudstorage_hadoopdfs_getresourceowner(void* lpObj, int resourceindex);

char* cloudstorage_hadoopdfs_getresourcepath(void* lpObj, int resourceindex);

char* cloudstorage_hadoopdfs_getresourcepermission(void* lpObj, int resourceindex);

int cloudstorage_hadoopdfs_getresourcereplication(void* lpObj, int resourceindex);

int64 cloudstorage_hadoopdfs_getresourcesize(void* lpObj, int resourceindex);

char* cloudstorage_hadoopdfs_getresourcesymlinktarget(void* lpObj, int resourceindex);

int cloudstorage_hadoopdfs_getresourcetype(void* lpObj, int resourceindex);

int GetResourceCount();

qint64 GetResourceAccessTime(int iResourceIndex);

qint64 GetResourceBlockSize(int iResourceIndex);

qint64 GetResourceChildCount(int iResourceIndex);

QString GetResourceGroup(int iResourceIndex);

qint64 GetResourceModifiedTime(int iResourceIndex);

QString GetResourceName(int iResourceIndex);

QString GetResourceOwner(int iResourceIndex);

QString GetResourcePath(int iResourceIndex);

QString GetResourcePermission(int iResourceIndex);

int GetResourceReplication(int iResourceIndex);

qint64 GetResourceSize(int iResourceIndex);

QString GetResourceSymlinkTarget(int iResourceIndex);

int GetResourceType(int iResourceIndex);

Remarks

This collection holds a list of HDFS resources: files, directories, and symlinks.

Calling GetResourceInfo or ListResources will populate this collection.

This property is read-only and not available at design time.

Data Type

CloudStorageHDFSResource

SSLAcceptServerCert Property (HadoopDFS Class)

Instructs the class to unconditionally accept the server certificate that matches the supplied certificate.

Syntax

• C++
• C
• Qt

CloudStorageCertificate* GetSSLAcceptServerCert();
int SetSSLAcceptServerCert(CloudStorageCertificate* val);

char* cloudstorage_hadoopdfs_getsslacceptservercerteffectivedate(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertexpirationdate(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertextendedkeyusage(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertfingerprint(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertfingerprintsha1(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertfingerprintsha256(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertissuer(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertprivatekey(void* lpObj);

int cloudstorage_hadoopdfs_getsslacceptservercertprivatekeyavailable(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertprivatekeycontainer(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertpublickey(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertpublickeyalgorithm(void* lpObj);

int cloudstorage_hadoopdfs_getsslacceptservercertpublickeylength(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertserialnumber(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertsignaturealgorithm(void* lpObj);

int cloudstorage_hadoopdfs_getsslacceptservercertstore(void* lpObj, char** lpSSLAcceptServerCertStore, int* lenSSLAcceptServerCertStore);
int cloudstorage_hadoopdfs_setsslacceptservercertstore(void* lpObj, const char* lpSSLAcceptServerCertStore, int lenSSLAcceptServerCertStore);

char* cloudstorage_hadoopdfs_getsslacceptservercertstorepassword(void* lpObj);
int cloudstorage_hadoopdfs_setsslacceptservercertstorepassword(void* lpObj, const char* lpszSSLAcceptServerCertStorePassword);

int cloudstorage_hadoopdfs_getsslacceptservercertstoretype(void* lpObj);
int cloudstorage_hadoopdfs_setsslacceptservercertstoretype(void* lpObj, int iSSLAcceptServerCertStoreType);

char* cloudstorage_hadoopdfs_getsslacceptservercertsubjectaltnames(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertthumbprintmd5(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertthumbprintsha1(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertthumbprintsha256(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertusage(void* lpObj);

int cloudstorage_hadoopdfs_getsslacceptservercertusageflags(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertversion(void* lpObj);

char* cloudstorage_hadoopdfs_getsslacceptservercertsubject(void* lpObj);
int cloudstorage_hadoopdfs_setsslacceptservercertsubject(void* lpObj, const char* lpszSSLAcceptServerCertSubject);

int cloudstorage_hadoopdfs_getsslacceptservercertencoded(void* lpObj, char** lpSSLAcceptServerCertEncoded, int* lenSSLAcceptServerCertEncoded);
int cloudstorage_hadoopdfs_setsslacceptservercertencoded(void* lpObj, const char* lpSSLAcceptServerCertEncoded, int lenSSLAcceptServerCertEncoded);

QString GetSSLAcceptServerCertEffectiveDate();

QString GetSSLAcceptServerCertExpirationDate();

QString GetSSLAcceptServerCertExtendedKeyUsage();

QString GetSSLAcceptServerCertFingerprint();

QString GetSSLAcceptServerCertFingerprintSHA1();

QString GetSSLAcceptServerCertFingerprintSHA256();

QString GetSSLAcceptServerCertIssuer();

QString GetSSLAcceptServerCertPrivateKey();

bool GetSSLAcceptServerCertPrivateKeyAvailable();

QString GetSSLAcceptServerCertPrivateKeyContainer();

QString GetSSLAcceptServerCertPublicKey();

QString GetSSLAcceptServerCertPublicKeyAlgorithm();

int GetSSLAcceptServerCertPublicKeyLength();

QString GetSSLAcceptServerCertSerialNumber();

QString GetSSLAcceptServerCertSignatureAlgorithm();

QByteArray GetSSLAcceptServerCertStore();
int SetSSLAcceptServerCertStore(QByteArray qbaSSLAcceptServerCertStore);

QString GetSSLAcceptServerCertStorePassword();
int SetSSLAcceptServerCertStorePassword(QString qsSSLAcceptServerCertStorePassword);

int GetSSLAcceptServerCertStoreType();
int SetSSLAcceptServerCertStoreType(int iSSLAcceptServerCertStoreType);

QString GetSSLAcceptServerCertSubjectAltNames();

QString GetSSLAcceptServerCertThumbprintMD5();

QString GetSSLAcceptServerCertThumbprintSHA1();

QString GetSSLAcceptServerCertThumbprintSHA256();

QString GetSSLAcceptServerCertUsage();

int GetSSLAcceptServerCertUsageFlags();

QString GetSSLAcceptServerCertVersion();

QString GetSSLAcceptServerCertSubject();
int SetSSLAcceptServerCertSubject(QString qsSSLAcceptServerCertSubject);

QByteArray GetSSLAcceptServerCertEncoded();
int SetSSLAcceptServerCertEncoded(QByteArray qbaSSLAcceptServerCertEncoded);

Remarks

If it finds any issues with the certificate presented by the server, the class will normally terminate the connection with an error.

You may override this behavior by supplying a value for SSLAcceptServerCert. If the certificate supplied in SSLAcceptServerCert is the same as the certificate presented by the server, then the server certificate is accepted unconditionally, and the connection will continue normally.

Note: This functionality is provided only for cases in which you otherwise know that you are communicating with the right server. If used improperly, this property may create a security breach. Use it at your own risk.

Data Type

CloudStorageCertificate

SSLCert Property (HadoopDFS Class)

The certificate to be used during Secure Sockets Layer (SSL) negotiation.

Syntax

• C++
• C
• Qt

CloudStorageCertificate* GetSSLCert();
int SetSSLCert(CloudStorageCertificate* val);

char* cloudstorage_hadoopdfs_getsslcerteffectivedate(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertexpirationdate(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertextendedkeyusage(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertfingerprint(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertfingerprintsha1(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertfingerprintsha256(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertissuer(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertprivatekey(void* lpObj);

int cloudstorage_hadoopdfs_getsslcertprivatekeyavailable(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertprivatekeycontainer(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertpublickey(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertpublickeyalgorithm(void* lpObj);

int cloudstorage_hadoopdfs_getsslcertpublickeylength(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertserialnumber(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertsignaturealgorithm(void* lpObj);

int cloudstorage_hadoopdfs_getsslcertstore(void* lpObj, char** lpSSLCertStore, int* lenSSLCertStore);
int cloudstorage_hadoopdfs_setsslcertstore(void* lpObj, const char* lpSSLCertStore, int lenSSLCertStore);

char* cloudstorage_hadoopdfs_getsslcertstorepassword(void* lpObj);
int cloudstorage_hadoopdfs_setsslcertstorepassword(void* lpObj, const char* lpszSSLCertStorePassword);

int cloudstorage_hadoopdfs_getsslcertstoretype(void* lpObj);
int cloudstorage_hadoopdfs_setsslcertstoretype(void* lpObj, int iSSLCertStoreType);

char* cloudstorage_hadoopdfs_getsslcertsubjectaltnames(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertthumbprintmd5(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertthumbprintsha1(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertthumbprintsha256(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertusage(void* lpObj);

int cloudstorage_hadoopdfs_getsslcertusageflags(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertversion(void* lpObj);

char* cloudstorage_hadoopdfs_getsslcertsubject(void* lpObj);
int cloudstorage_hadoopdfs_setsslcertsubject(void* lpObj, const char* lpszSSLCertSubject);

int cloudstorage_hadoopdfs_getsslcertencoded(void* lpObj, char** lpSSLCertEncoded, int* lenSSLCertEncoded);
int cloudstorage_hadoopdfs_setsslcertencoded(void* lpObj, const char* lpSSLCertEncoded, int lenSSLCertEncoded);

QString GetSSLCertEffectiveDate();

QString GetSSLCertExpirationDate();

QString GetSSLCertExtendedKeyUsage();

QString GetSSLCertFingerprint();

QString GetSSLCertFingerprintSHA1();

QString GetSSLCertFingerprintSHA256();

QString GetSSLCertIssuer();

QString GetSSLCertPrivateKey();

bool GetSSLCertPrivateKeyAvailable();

QString GetSSLCertPrivateKeyContainer();

QString GetSSLCertPublicKey();

QString GetSSLCertPublicKeyAlgorithm();

int GetSSLCertPublicKeyLength();

QString GetSSLCertSerialNumber();

QString GetSSLCertSignatureAlgorithm();

QByteArray GetSSLCertStore();
int SetSSLCertStore(QByteArray qbaSSLCertStore);

QString GetSSLCertStorePassword();
int SetSSLCertStorePassword(QString qsSSLCertStorePassword);

int GetSSLCertStoreType();
int SetSSLCertStoreType(int iSSLCertStoreType);

QString GetSSLCertSubjectAltNames();

QString GetSSLCertThumbprintMD5();

QString GetSSLCertThumbprintSHA1();

QString GetSSLCertThumbprintSHA256();

QString GetSSLCertUsage();

int GetSSLCertUsageFlags();

QString GetSSLCertVersion();

QString GetSSLCertSubject();
int SetSSLCertSubject(QString qsSSLCertSubject);

QByteArray GetSSLCertEncoded();
int SetSSLCertEncoded(QByteArray qbaSSLCertEncoded);

Remarks

This property includes the digital certificate that the class will use during SSL negotiation. Set this property to a valid certificate before starting SSL negotiation. To set a certificate, you may set the Encoded field to the encoded certificate. To select a certificate, use the store and subject fields.

Data Type

CloudStorageCertificate

SSLProvider Property (HadoopDFS Class)

The Secure Sockets Layer/Transport Layer Security (SSL/TLS) implementation to use.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetSSLProvider();
int SetSSLProvider(int iSSLProvider);

Unicode (Windows)
INT GetSSLProvider();
INT SetSSLProvider(INT iSSLProvider);

SSLP_AUTOMATIC(0), 
SSLP_PLATFORM(1), 
SSLP_INTERNAL(2)

int cloudstorage_hadoopdfs_getsslprovider(void* lpObj);
int cloudstorage_hadoopdfs_setsslprovider(void* lpObj, int iSSLProvider);

int GetSSLProvider();
int SetSSLProvider(int iSSLProvider);

Default Value

0

Remarks

This property specifies the SSL/TLS implementation to use. In most cases the default value of 0 (Automatic) is recommended and should not be changed. When set to 0 (Automatic), the class will select whether to use the platform implementation or the internal implementation depending on the operating system as well as the TLS version being used.

Possible values are as follows:

In most cases using the default value (Automatic) is recommended. The class will select a provider depending on the current platform.

When Automatic is selected, on Windows, the class will use the platform implementation. On Linux/macOS, the class will use the internal implementation. When TLS 1.3 is enabled via SSLEnabledProtocols, the internal implementation is used on all platforms.

Data Type

Integer

SSLServerCert Property (HadoopDFS Class)

The server certificate for the last established connection.

Syntax

• C++
• C
• Qt

CloudStorageCertificate* GetSSLServerCert();

char* cloudstorage_hadoopdfs_getsslservercerteffectivedate(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertexpirationdate(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertextendedkeyusage(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertfingerprint(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertfingerprintsha1(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertfingerprintsha256(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertissuer(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertprivatekey(void* lpObj);

int cloudstorage_hadoopdfs_getsslservercertprivatekeyavailable(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertprivatekeycontainer(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertpublickey(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertpublickeyalgorithm(void* lpObj);

int cloudstorage_hadoopdfs_getsslservercertpublickeylength(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertserialnumber(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertsignaturealgorithm(void* lpObj);

int cloudstorage_hadoopdfs_getsslservercertstore(void* lpObj, char** lpSSLServerCertStore, int* lenSSLServerCertStore);

char* cloudstorage_hadoopdfs_getsslservercertstorepassword(void* lpObj);

int cloudstorage_hadoopdfs_getsslservercertstoretype(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertsubjectaltnames(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertthumbprintmd5(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertthumbprintsha1(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertthumbprintsha256(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertusage(void* lpObj);

int cloudstorage_hadoopdfs_getsslservercertusageflags(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertversion(void* lpObj);

char* cloudstorage_hadoopdfs_getsslservercertsubject(void* lpObj);

int cloudstorage_hadoopdfs_getsslservercertencoded(void* lpObj, char** lpSSLServerCertEncoded, int* lenSSLServerCertEncoded);

QString GetSSLServerCertEffectiveDate();

QString GetSSLServerCertExpirationDate();

QString GetSSLServerCertExtendedKeyUsage();

QString GetSSLServerCertFingerprint();

QString GetSSLServerCertFingerprintSHA1();

QString GetSSLServerCertFingerprintSHA256();

QString GetSSLServerCertIssuer();

QString GetSSLServerCertPrivateKey();

bool GetSSLServerCertPrivateKeyAvailable();

QString GetSSLServerCertPrivateKeyContainer();

QString GetSSLServerCertPublicKey();

QString GetSSLServerCertPublicKeyAlgorithm();

int GetSSLServerCertPublicKeyLength();

QString GetSSLServerCertSerialNumber();

QString GetSSLServerCertSignatureAlgorithm();

QByteArray GetSSLServerCertStore();

QString GetSSLServerCertStorePassword();

int GetSSLServerCertStoreType();

QString GetSSLServerCertSubjectAltNames();

QString GetSSLServerCertThumbprintMD5();

QString GetSSLServerCertThumbprintSHA1();

QString GetSSLServerCertThumbprintSHA256();

QString GetSSLServerCertUsage();

int GetSSLServerCertUsageFlags();

QString GetSSLServerCertVersion();

QString GetSSLServerCertSubject();

QByteArray GetSSLServerCertEncoded();

Remarks

This property contains the server certificate for the last established connection.

SSLServerCert is reset every time a new connection is attempted.

This property is read-only.

Data Type

CloudStorageCertificate

StartByte Property (HadoopDFS Class)

The byte offset from which to start downloading a file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int64 GetStartByte();
int SetStartByte(int64 lStartByte);

Unicode (Windows)
LONG64 GetStartByte();
INT SetStartByte(LONG64 lStartByte);

int64 cloudstorage_hadoopdfs_getstartbyte(void* lpObj);
int cloudstorage_hadoopdfs_setstartbyte(void* lpObj, int64 lStartByte);

qint64 GetStartByte();
int SetStartByte(qint64 lStartByte);

Default Value

0

Remarks

This property specifies an offset (in bytes) from which to start reading a file when DownloadFile is called. It can be used in tandem with ReadBytes to specify a specific range of the file to download.

This property is not available at design time.

Data Type

Long64

Timeout Property (HadoopDFS Class)

The timeout for the class.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetTimeout();
int SetTimeout(int iTimeout);

Unicode (Windows)
INT GetTimeout();
INT SetTimeout(INT iTimeout);

int cloudstorage_hadoopdfs_gettimeout(void* lpObj);
int cloudstorage_hadoopdfs_settimeout(void* lpObj, int iTimeout);

int GetTimeout();
int SetTimeout(int iTimeout);

Default Value

60

Remarks

If the Timeout property is set to 0, all operations will run uninterrupted until successful completion or an error condition is encountered.

If Timeout is set to a positive value, the class will wait for the operation to complete before returning control.

The class will use DoEvents to enter an efficient wait loop during any potential waiting period, making sure that all system events are processed immediately as they arrive. This ensures that the host application does not freeze and remains responsive.

If Timeout expires, and the operation is not yet complete, the class fails with an error.

Note: By default, all timeouts are inactivity timeouts, that is, the timeout period is extended by Timeout seconds when any amount of data is successfully sent or received.

The default value for the Timeout property is 60 seconds.

Data Type

Integer

URL Property (HadoopDFS Class)

The URL of the Hadoop WebHDFS server.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetURL();
int SetURL(const char* lpszURL);

Unicode (Windows)
LPWSTR GetURL();
INT SetURL(LPCWSTR lpszURL);

char* cloudstorage_hadoopdfs_geturl(void* lpObj);
int cloudstorage_hadoopdfs_seturl(void* lpObj, const char* lpszURL);

QString GetURL();
int SetURL(QString qsURL);

Default Value

""

Remarks

This property specifies the URL of the Hadoop WebHDFS server to make requests against.

A full WebHDFS URL looks like http[s]://<HOST>:<PORT>/webhdfs/v1. Anytime this property is set to a non-empty string, the class will automatically append /webhdfs/v1 to its value (if necessary).

The class automatically detects whether to secure the connection using SSL/TLS based on whether the URL begins with http (not secured) or https (secured).

This property is not available at design time.

Data Type

String

User Property (HadoopDFS Class)

The user name to use for authentication.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* GetUser();
int SetUser(const char* lpszUser);

Unicode (Windows)
LPWSTR GetUser();
INT SetUser(LPCWSTR lpszUser);

char* cloudstorage_hadoopdfs_getuser(void* lpObj);
int cloudstorage_hadoopdfs_setuser(void* lpObj, const char* lpszUser);

QString GetUser();
int SetUser(QString qsUser);

Default Value

""

Remarks

This property specifies the user name to use for authentication.

Refer to AuthMechanism for more information.

This property is not available at design time.

Data Type

String

AddQueryParam Method (HadoopDFS Class)

Adds a query parameter to the QueryParams properties.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int AddQueryParam(const char* lpszName, const char* lpszValue);

Unicode (Windows)
INT AddQueryParam(LPCWSTR lpszName, LPCWSTR lpszValue);

int cloudstorage_hadoopdfs_addqueryparam(void* lpObj, const char* lpszName, const char* lpszValue);

int AddQueryParam(const QString& qsName, const QString& qsValue);

Remarks

This method is used to add a query parameter to the QueryParams properties. Name specifies the name of the parameter, and Value specifies the value of the parameter.

All specified Values will be URL encoded by the class automatically. Consult the service documentation for details on the available parameters.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

AppendFile Method (HadoopDFS Class)

Appends data to an existing file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int AppendFile(const char* lpszFilePath);

Unicode (Windows)
INT AppendFile(LPCWSTR lpszFilePath);

int cloudstorage_hadoopdfs_appendfile(void* lpObj, const char* lpszFilePath);

int AppendFile(const QString& qsFilePath);

Remarks

This method appends data to the existing file at FilePath.

If SetUploadStream has been used to set an upload stream, it will take priority as the data source. If LocalFile is set data will be read from the file at the specified path. If LocalFile is not set the data in ResourceData will be used.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

CalcAuthorization Method (HadoopDFS Class)

Calculates the Authorization header based on provided credentials.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int CalcAuthorization();

Unicode (Windows)
INT CalcAuthorization();

int cloudstorage_hadoopdfs_calcauthorization(void* lpObj);

int CalcAuthorization();

Remarks

This method calculates the Authorization value using the values provided in AuthScheme, User and Password.

In most cases this method does not need to be called. The class will automatically calculate any required authorization values when a method is called such as Get or Post.

This method may be useful in cases where the Authorization value needs to be calculated prior to sending a request.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

Config Method (HadoopDFS Class)

Sets or retrieves a configuration setting.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* Config(const char* lpszConfigurationString);

Unicode (Windows)
LPWSTR Config(LPCWSTR lpszConfigurationString);

char* cloudstorage_hadoopdfs_config(void* lpObj, const char* lpszConfigurationString);

QString Config(const QString& qsConfigurationString);

Remarks

Config is a generic method available in every class. It is used to set and retrieve configuration settings for the class.

These settings are similar in functionality to properties, but they are rarely used. In order to avoid "polluting" the property namespace of the class, access to these internal properties is provided through the Config method.

To set a configuration setting named PROPERTY, you must call Config("PROPERTY=VALUE"), where VALUE is the value of the setting expressed as a string. For boolean values, use the strings "True", "False", "0", "1", "Yes", or "No" (case does not matter).

To read (query) the value of a configuration setting, you must call Config("PROPERTY"). The value will be returned as a string.

Error Handling (C++)

This method returns a String value; after it returns, call the GetLastErrorCode() method to obtain its result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message.

DeleteResource Method (HadoopDFS Class)

Deletes a resource.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int DeleteResource(const char* lpszPath);

Unicode (Windows)
INT DeleteResource(LPCWSTR lpszPath);

int cloudstorage_hadoopdfs_deleteresource(void* lpObj, const char* lpszPath);

int DeleteResource(const QString& qsPath);

Remarks

This method deletes the resource at Path.

The RecursiveDelete configuration setting can be used to control whether this method will recursively delete non-empty directories (it is enabled by default).

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

DoCustomOp Method (HadoopDFS Class)

Executes an arbitrary WebHDFS operation.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int DoCustomOp(const char* lpszHttpMethod, const char* lpszRequestPath, const char* lpszOp);

Unicode (Windows)
INT DoCustomOp(LPCWSTR lpszHttpMethod, LPCWSTR lpszRequestPath, LPCWSTR lpszOp);

int cloudstorage_hadoopdfs_docustomop(void* lpObj, const char* lpszHttpMethod, const char* lpszRequestPath, const char* lpszOp);

int DoCustomOp(const QString& qsHttpMethod, const QString& qsRequestPath, const QString& qsOp);

Remarks

This method can be used to execute any WebHDFS operation the class does not explicitly implement.

Valid values for HttpMethod are:

• GET (default if empty)
• POST
• PUT
• DELETE

RequestPath must either be a valid resource path, or empty string. Op must be a valid WebHDFS operation (refer to the Hadoop WebHDFS documentation for a full list of supported operations).

Usage

When this method is called, the class will do the following:

1. Build a request URL using URL, RequestPath, Op, and the query parameters held by the QueryParams properties.
2. Send the request using the given HttpMethod, the request URL built in step 1, the headers held by OtherHeaders, and the currently-configured authentication (if any; refer to AuthMechanism for more information). The request is always sent with an empty body.
3. Store the response headers in the ParsedHeaders properties, and the response body in the stream specified using SetDownloadStream, the specified LocalFile, or ResourceData (using the same logic as DownloadFile).

If the response body is JSON data, the XPath, XText, and other X* configuration settings can then be used to navigate and extract information from it.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

DownloadFile Method (HadoopDFS Class)

Downloads a file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int DownloadFile(const char* lpszFilePath);

Unicode (Windows)
INT DownloadFile(LPCWSTR lpszFilePath);

int cloudstorage_hadoopdfs_downloadfile(void* lpObj, const char* lpszFilePath);

int DownloadFile(const QString& qsFilePath);

Remarks

This method downloads the file at FilePath.

If a stream has been specified using SetDownloadStream, the file data will be sent through it. If a stream is not specified, and LocalFile is set, the file will be saved to the specified location; otherwise, the file data will be held by ResourceData.

To download and decrypt an encrypted file, set EncryptionAlgorithm and EncryptionPassword before calling this method.

Download Notes

In the simplest use-case, downloading a file looks like this: hdfs.LocalFile = "../MyFile.zip"; hdfs.DownloadFile(hdfs.Resources[0].Path);

Resuming Downloads

The class also supports resuming failed downloads by using the StartByte property. If a download is interrupted, set StartByte to the appropriate offset before calling this method to resume the download. string downloadFile = "../MyFile.zip"; hdfs.LocalFile = downloadFile; hdfs.DownloadFile(hdfs.Resources[0].Path); //The transfer is interrupted and DownloadFile() above fails. Later, resume the download: //Get the size of the partially downloaded file hdfs.StartByte = new FileInfo(downloadFile).Length; hdfs.DownloadFile(hdfs.Resources[0].Path);

Resuming Encrypted File Downloads

Resuming encrypted file downloads is only supported when LocalFile was set in the initial download attempt.

If LocalFile is set when beginning an encrypted download, the class creates a temporary file in TempPath to hold the encrypted data until the download is complete. If the download is interrupted, DownloadTempFile will be populated with the path of the temporary file that holds the partial data.

To resume, DownloadTempFile must be populated, along with StartByte, to allow the remainder of the encrypted data to be downloaded. Once the encrypted data is downloaded it will be decrypted and written to LocalFile. hdfs.LocalFile = "../MyFile.zip"; hdfs.EncryptionPassword = "password"; hdfs.DownloadFile(hdfs.Resources[0].Path); //The transfer is interrupted and DownloadFile() above fails. Later, resume the download: //Get the size of the partially download temp file hdfs.StartByte = new FileInfo(hdfs.Config("DownloadTempFile")).Length; hdfs.DownloadFile(hdfs.Resources[0].Path);

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

GetDirSummary Method (HadoopDFS Class)

Gets a content summary for a directory.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetDirSummary(const char* lpszDirPath);

Unicode (Windows)
INT GetDirSummary(LPCWSTR lpszDirPath);

int cloudstorage_hadoopdfs_getdirsummary(void* lpObj, const char* lpszDirPath);

int GetDirSummary(const QString& qsDirPath);

Remarks

This method gets a content summary for the directory at DirPath, populating the DirSummary object property's fields.

DirPath may be empty to get a content summary for the root directory (/).

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

GetResourceInfo Method (HadoopDFS Class)

Gets information about a specific resource.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int GetResourceInfo(const char* lpszPath);

Unicode (Windows)
INT GetResourceInfo(LPCWSTR lpszPath);

int cloudstorage_hadoopdfs_getresourceinfo(void* lpObj, const char* lpszPath);

int GetResourceInfo(const QString& qsPath);

Remarks

This method gets information about the resource at Path. Path may be empty to get information about the root directory (/).

Calling this method will fire the ResourceList event, and will populate the Resources properties with a single item (clearing any previously-held items in the process).

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

Interrupt Method (HadoopDFS Class)

This method interrupts the current method.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int Interrupt();

Unicode (Windows)
INT Interrupt();

int cloudstorage_hadoopdfs_interrupt(void* lpObj);

int Interrupt();

Remarks

If there is no method in progress, Interrupt simply returns, doing nothing.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

JoinFileBlocks Method (HadoopDFS Class)

Joins multiple files' blocks together into one file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int JoinFileBlocks(const char* lpszTargetFilePath, const char* lpszSourceFilePaths);

Unicode (Windows)
INT JoinFileBlocks(LPCWSTR lpszTargetFilePath, LPCWSTR lpszSourceFilePaths);

int cloudstorage_hadoopdfs_joinfileblocks(void* lpObj, const char* lpszTargetFilePath, const char* lpszSourceFilePaths);

int JoinFileBlocks(const QString& qsTargetFilePath, const QString& qsSourceFilePaths);

Remarks

This method joins the blocks from the files at one or more SourceFilePaths onto the end of the file at TargetFilePath. (If this operations is successful, the original source files will no longer be accessible.)

The order of the source files' paths in SourceFilePaths determines the order in which their blocks are joined onto the file at TargetFilePath.

Usage Constraints

The server imposes a number of constraints that must be satisfied in order for the join operation to complete successfully.

TargetFilePath and SourceFilePaths must both be non-empty.

SourceFilePaths must be specified as a comma-separated list of source file paths, with no duplicates or paths that match TargetFilePath.

Additionally, all files referred to by both TargetFilePath and SourceFilePaths must satisfy the following constraints:

• All files must already exist.
• All files must be located in the exact same directory (i.e., all files must be siblings).
• The block size of all source files must be less than or equal to the block size of the target file.

If any of the above constraints are not satisfied, the server will return an error.

The class will take care of verifying that all parameters are non-empty and that all paths begin with a forward slash (/).

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

ListResources Method (HadoopDFS Class)

Lists resources in a given directory.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int ListResources(const char* lpszDirPath);

Unicode (Windows)
INT ListResources(LPCWSTR lpszDirPath);

int cloudstorage_hadoopdfs_listresources(void* lpObj, const char* lpszDirPath);

int ListResources(const QString& qsDirPath);

Remarks

This method lists resources within the directory at DirPath.

Calling this method will fire the ResourceList event once for each resource, and will also populate the Resources properties.

DirPath may be empty to list resources in the root directory (/). // ResourceList event handler. hdfs.OnResourceList += (s, e) => { Console.WriteLine(e.Name); }; hdfs.ListResources("/work_files/serious_business/cats"); for (int i = 0; i < hdfs.Resources.Count; i++) { // Process resources here. }

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

MakeDirectory Method (HadoopDFS Class)

Makes a directory.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* MakeDirectory(const char* lpszNewDirPath);

Unicode (Windows)
LPWSTR MakeDirectory(LPCWSTR lpszNewDirPath);

char* cloudstorage_hadoopdfs_makedirectory(void* lpObj, const char* lpszNewDirPath);

QString MakeDirectory(const QString& qsNewDirPath);

Remarks

This method makes a new directory at NewDirPath and returns the full path of the new directory. Any non-existent parent directories will also be created.

If the CreatePermission configuration setting is non-empty, the directory will be created with the permission it specifies. Otherwise, the server's default (755) will be used.

Error Handling (C++)

This method returns a String value; after it returns, call the GetLastErrorCode() method to obtain its result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message.

MoveResource Method (HadoopDFS Class)

Moves a resource.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int MoveResource(const char* lpszFromPath, const char* lpszToPath);

Unicode (Windows)
INT MoveResource(LPCWSTR lpszFromPath, LPCWSTR lpszToPath);

int cloudstorage_hadoopdfs_moveresource(void* lpObj, const char* lpszFromPath, const char* lpszToPath);

int MoveResource(const QString& qsFromPath, const QString& qsToPath);

Remarks

This method moves the resource at FromPath to ToPath.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

Reset Method (HadoopDFS Class)

Resets the class to its initial state.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int Reset();

Unicode (Windows)
INT Reset();

int cloudstorage_hadoopdfs_reset(void* lpObj);

int Reset();

Remarks

This method resets the class to its initial state.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

SetDownloadStream Method (HadoopDFS Class)

Sets the stream to which downloaded data will be written.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int SetDownloadStream(CloudStorageStream* sDownloadStream);

Unicode (Windows)
INT SetDownloadStream(CloudStorageStream* sDownloadStream);

int cloudstorage_hadoopdfs_setdownloadstream(void* lpObj, CloudStorageStream* sDownloadStream);

int SetDownloadStream(CloudStorageStream* sDownloadStream);

Remarks

If a download stream is set before data is downloaded, the downloaded data will be written to the stream. The stream should be open and normally set to position 0.

Note: Passing a non-null value for DownloadStream will cause LocalFile to be cleared. Similarly, setting LocalFile to a non-empty value will discard any stream set using this method.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

SetFileReplication Method (HadoopDFS Class)

Sets the replication factor for a file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int SetFileReplication(const char* lpszFilePath, int iReplication);

Unicode (Windows)
INT SetFileReplication(LPCWSTR lpszFilePath, INT iReplication);

int cloudstorage_hadoopdfs_setfilereplication(void* lpObj, const char* lpszFilePath, int iReplication);

int SetFileReplication(const QString& qsFilePath, int iReplication);

Remarks

This method sets the replication factor for the file at FilePath to Replication.

Valid values for Replication are 1 through 32767 (inclusive); or 0 to reset the file's replication factor back to the server default.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

SetOwner Method (HadoopDFS Class)

Sets a resource's owner and/or group.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int SetOwner(const char* lpszPath, const char* lpszOwner, const char* lpszGroup);

Unicode (Windows)
INT SetOwner(LPCWSTR lpszPath, LPCWSTR lpszOwner, LPCWSTR lpszGroup);

int cloudstorage_hadoopdfs_setowner(void* lpObj, const char* lpszPath, const char* lpszOwner, const char* lpszGroup);

int SetOwner(const QString& qsPath, const QString& qsOwner, const QString& qsGroup);

Remarks

This method set the owner and/or group of the resource at Path to the given values.

If non-empty, Owner must be a valid user name, and Group must be a valid group name. If either parameter is empty, the file's current value will remain unchanged. (If both parameters are empty, no request is sent.)

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

SetPermission Method (HadoopDFS Class)

Assigns the given permission to a resource.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int SetPermission(const char* lpszPath, const char* lpszPermission);

Unicode (Windows)
INT SetPermission(LPCWSTR lpszPath, LPCWSTR lpszPermission);

int cloudstorage_hadoopdfs_setpermission(void* lpObj, const char* lpszPath, const char* lpszPermission);

int SetPermission(const QString& qsPath, const QString& qsPermission);

Remarks

This method assigns the given Permission to the resource at Path.

Permission must be formatted as an octal permission string. If Permission is empty string, the resource's permission will be reset to the server default (755).

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

SetTimes Method (HadoopDFS Class)

Sets a resource's modification and/or access times.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int SetTimes(const char* lpszPath, int64 lModifiedTime, int64 lAccessTime);

Unicode (Windows)
INT SetTimes(LPCWSTR lpszPath, LONG64 lModifiedTime, LONG64 lAccessTime);

int cloudstorage_hadoopdfs_settimes(void* lpObj, const char* lpszPath, int64 lModifiedTime, int64 lAccessTime);

int SetTimes(const QString& qsPath, qint64 lModifiedTime, qint64 lAccessTime);

Remarks

This method sets the modification and/or access times of the resource at Path to the given values.

Both ModifiedTime and AccessTime should be specified as a number of milliseconds relative to the Unix epoch. If either parameter is negative, the file's current value will remain unchanged. (If both parameters are negative, no request is sent.)

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

SetUploadStream Method (HadoopDFS Class)

Sets the stream from which data is read when uploading.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
int SetUploadStream(CloudStorageStream* sUploadStream);

Unicode (Windows)
INT SetUploadStream(CloudStorageStream* sUploadStream);

int cloudstorage_hadoopdfs_setuploadstream(void* lpObj, CloudStorageStream* sUploadStream);

int SetUploadStream(CloudStorageStream* sUploadStream);

Remarks

If an upload stream is set before data is uploaded, the content of the stream will be read by the class and uploaded to the server. The stream should be open and normally set to position 0.

Note: Passing a non-null value for UploadStream will cause LocalFile to be cleared. Similarly, setting LocalFile to a non-empty value will discard any stream set using this method.

Error Handling (C++)

This method returns a result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message. (Note: This method's result code can also be obtained by calling the GetLastErrorCode() method after it returns.)

TruncateFile Method (HadoopDFS Class)

Truncates a file to a given size.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
bool TruncateFile(const char* lpszFilePath, int64 lNewSize);

Unicode (Windows)
INT TruncateFile(LPCWSTR lpszFilePath, LONG64 lNewSize);

bool cloudstorage_hadoopdfs_truncatefile(void* lpObj, const char* lpszFilePath, int64 lNewSize);

bool TruncateFile(const QString& qsFilePath, qint64 lNewSize);

Remarks

This method truncates the file at FilePath to the given NewSize (specified in bytes). NewSize must not be less than 0 or greater than the file's current size.

This method returns true if the file has been truncated successfully and is immediately ready for further modifications.

This method returns false if the server is still in the process of truncating the file (this could happen, e.g., if the server needs to truncate multiple replicas of the file). In this case, the server will reject any further attempts to modify the file until it has finished truncating it. Monitor the file's size using GetResourceInfo to determine when the truncation process has finished.

Error Handling (C++)

This method returns a Boolean value; after it returns, call the GetLastErrorCode() method to obtain its result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message.

UploadFile Method (HadoopDFS Class)

Uploads a file.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
char* UploadFile(const char* lpszNewFilePath);

Unicode (Windows)
LPWSTR UploadFile(LPCWSTR lpszNewFilePath);

char* cloudstorage_hadoopdfs_uploadfile(void* lpObj, const char* lpszNewFilePath);

QString UploadFile(const QString& qsNewFilePath);

Remarks

This method uploads a new file to NewFilePath and returns the full path of the uploaded file. If a file already exists on the server at NewFilePath, the Overwrite property controls whether the server will overwrite the file or return an error.

If the CreatePermission configuration setting is non-empty, the file will be created with the permission it specifies. Otherwise, the server's default (755) will be used.

If SetUploadStream has been used to set an upload stream, it will take priority as the file data source. If LocalFile is set the file will be uploaded from the specified path. If LocalFile is not set the data in ResourceData will be used.

To encrypt the file before uploading it, set EncryptionAlgorithm and EncryptionPassword. hdfs.LocalFile = "../MyFile.zip"; hdfs.UploadFile("/MyFile.zip");

Error Handling (C++)

This method returns a String value; after it returns, call the GetLastErrorCode() method to obtain its result code; 0 indicates success, while a non-zero error code indicates that this method encountered an error during its execution. If an error occurs, the GetLastError() method can be called to retrieve the associated error message.

EndTransfer Event (HadoopDFS Class)

This event fires when a document finishes transferring.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireEndTransfer(HadoopDFSEndTransferEventParams *e);

typedef struct {
  int Direction;
  int reserved;
} HadoopDFSEndTransferEventParams;


Unicode (Windows)
virtual INT FireEndTransfer(HadoopDFSEndTransferEventParams *e);

typedef struct {
  INT Direction;
  INT reserved;
} HadoopDFSEndTransferEventParams;

#define EID_HADOOPDFS_ENDTRANSFER 1

virtual INT CLOUDSTORAGE_CALL FireEndTransfer(INT &iDirection);

class HadoopDFSEndTransferEventParams {
public:
  int Direction();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void EndTransfer(HadoopDFSEndTransferEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireEndTransfer(HadoopDFSEndTransferEventParams *e) {...}

Remarks

The EndTransfer event is fired when the document text finishes transferring from the server to the local host.

The Direction parameter shows whether the client (0) or the server (1) is sending the data.

Error Event (HadoopDFS Class)

Fired when information is available about errors during data delivery.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireError(HadoopDFSErrorEventParams *e);

typedef struct {
  int ErrorCode;
  const char *Description;
  int reserved;
} HadoopDFSErrorEventParams;


Unicode (Windows)
virtual INT FireError(HadoopDFSErrorEventParams *e);

typedef struct {
  INT ErrorCode;
  LPCWSTR Description;
  INT reserved;
} HadoopDFSErrorEventParams;

#define EID_HADOOPDFS_ERROR 2

virtual INT CLOUDSTORAGE_CALL FireError(INT &iErrorCode, LPSTR &lpszDescription);

class HadoopDFSErrorEventParams {
public:
  int ErrorCode();

  const QString &Description();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void Error(HadoopDFSErrorEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireError(HadoopDFSErrorEventParams *e) {...}

Remarks

The Error event is fired in case of exceptional conditions during message processing. Normally the class fails with an error.

The ErrorCode parameter contains an error code, and the Description parameter contains a textual description of the error. For a list of valid error codes and their descriptions, please refer to the Error Codes section.

Header Event (HadoopDFS Class)

Fired every time a header line comes in.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireHeader(HadoopDFSHeaderEventParams *e);

typedef struct {
  const char *Field;
  const char *Value;
  int reserved;
} HadoopDFSHeaderEventParams;


Unicode (Windows)
virtual INT FireHeader(HadoopDFSHeaderEventParams *e);

typedef struct {
  LPCWSTR Field;
  LPCWSTR Value;
  INT reserved;
} HadoopDFSHeaderEventParams;

#define EID_HADOOPDFS_HEADER 3

virtual INT CLOUDSTORAGE_CALL FireHeader(LPSTR &lpszField, LPSTR &lpszValue);

class HadoopDFSHeaderEventParams {
public:
  const QString &Field();

  const QString &Value();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void Header(HadoopDFSHeaderEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireHeader(HadoopDFSHeaderEventParams *e) {...}

Remarks

The Field parameter contains the name of the HTTP header (which is the same as it is delivered). The Value parameter contains the header contents.

If the header line being retrieved is a continuation header line, then the Field parameter contains "" (empty string).

Log Event (HadoopDFS Class)

Fired once for each log message.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireLog(HadoopDFSLogEventParams *e);

typedef struct {
  int LogLevel;
  const char *Message;
  const char *LogType;
  int reserved;
} HadoopDFSLogEventParams;


Unicode (Windows)
virtual INT FireLog(HadoopDFSLogEventParams *e);

typedef struct {
  INT LogLevel;
  LPCWSTR Message;
  LPCWSTR LogType;
  INT reserved;
} HadoopDFSLogEventParams;

#define EID_HADOOPDFS_LOG 4

virtual INT CLOUDSTORAGE_CALL FireLog(INT &iLogLevel, LPSTR &lpszMessage, LPSTR &lpszLogType);

class HadoopDFSLogEventParams {
public:
  int LogLevel();

  const QString &Message();

  const QString &LogType();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void Log(HadoopDFSLogEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireLog(HadoopDFSLogEventParams *e) {...}

Remarks

This event is fired once for each log message generated by the class. The verbosity is controlled by the LogLevel setting.

LogLevel indicates the level of message. Possible values are as follows:

The value 1 (Info) logs basic information, including the URL, HTTP version, and status details.

The value 2 (Verbose) logs additional information about the request and response.

The value 3 (Debug) logs the headers and body for both the request and response, as well as additional debug information (if any).

Message is the log entry.

LogType identifies the type of log entry. Possible values are as follows:

• "Info"
• "RequestHeaders"
• "ResponseHeaders"
• "RequestBody"
• "ResponseBody"
• "ProxyRequest"
• "ProxyResponse"
• "FirewallRequest"
• "FirewallResponse"

Progress Event (HadoopDFS Class)

Fires during an upload or download to indicate transfer progress.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireProgress(HadoopDFSProgressEventParams *e);

typedef struct {
  int Direction;
  int64 BytesTransferred;
  int64 TotalBytes;
  int PercentDone;
  int reserved;
} HadoopDFSProgressEventParams;


Unicode (Windows)
virtual INT FireProgress(HadoopDFSProgressEventParams *e);

typedef struct {
  INT Direction;
  LONG64 BytesTransferred;
  LONG64 TotalBytes;
  INT PercentDone;
  INT reserved;
} HadoopDFSProgressEventParams;

#define EID_HADOOPDFS_PROGRESS 5

virtual INT CLOUDSTORAGE_CALL FireProgress(INT &iDirection, LONG64 &lBytesTransferred, LONG64 &lTotalBytes, INT &iPercentDone);

class HadoopDFSProgressEventParams {
public:
  int Direction();

  qint64 BytesTransferred();

  qint64 TotalBytes();

  int PercentDone();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void Progress(HadoopDFSProgressEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireProgress(HadoopDFSProgressEventParams *e) {...}

Remarks

This event fires during an upload or download to indicate the progress of the transfer of the entire request. By default, this event will fire each time PercentDone increases by one percent; the ProgressStep configuration setting can be used to alter this behavior.

Direction indicates whether the transfer is an upload (0) or a download (1).

BytesTransferred reflects the number of bytes that have been transferred so far, or 0 if the transfer is starting (however, see note below).

TotalBytes reflects the total number of bytes that are to be transferred, or -1 if the total is unknown. This amount includes the size of everything in the request like HTTP headers.

PercentDone reflects the overall progress of the transfer, or -1 if the progress cannot be calculated.

Note: By default, the class tracks transfer progress absolutely. If a transfer is interrupted and later resumed, the values reported by this event upon and after resumption will account for the data that was transferred before the interruption.

For example, if 10MB of data was successfully transferred before the interruption, then this event will fire with a BytesTransferred value of 10485760 (10MB) when the transfer is first resumed, and then continue to fire with successively greater values as usual.

This behavior can be changed by disabling the ProgressAbsolute configuration setting, in which case the class will treat resumed transfers as "new" transfers. In this case, the BytesTransferred parameter will always be 0 the first time this event fires, regardless of whether the transfer is new or being resumed.

ResourceList Event (HadoopDFS Class)

Fires once for each resource returned when listing resources.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireResourceList(HadoopDFSResourceListEventParams *e);

typedef struct {
  const char *Name;
  const char *Path;
  int ResourceType;
  int64 ModifiedTime;
  int64 AccessTime;
  int64 Size;
  const char *Permission;
  const char *Owner;
  const char *Group;
  int Replication;
  int reserved;
} HadoopDFSResourceListEventParams;


Unicode (Windows)
virtual INT FireResourceList(HadoopDFSResourceListEventParams *e);

typedef struct {
  LPCWSTR Name;
  LPCWSTR Path;
  INT ResourceType;
  LONG64 ModifiedTime;
  LONG64 AccessTime;
  LONG64 Size;
  LPCWSTR Permission;
  LPCWSTR Owner;
  LPCWSTR Group;
  INT Replication;
  INT reserved;
} HadoopDFSResourceListEventParams;

#define EID_HADOOPDFS_RESOURCELIST 6

virtual INT CLOUDSTORAGE_CALL FireResourceList(LPSTR &lpszName, LPSTR &lpszPath, INT &iResourceType, LONG64 &lModifiedTime, LONG64 &lAccessTime, LONG64 &lSize, LPSTR &lpszPermission, LPSTR &lpszOwner, LPSTR &lpszGroup, INT &iReplication);

class HadoopDFSResourceListEventParams {
public:
  const QString &Name();

  const QString &Path();

  int ResourceType();

  qint64 ModifiedTime();

  qint64 AccessTime();

  qint64 Size();

  const QString &Permission();

  const QString &Owner();

  const QString &Group();

  int Replication();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void ResourceList(HadoopDFSResourceListEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireResourceList(HadoopDFSResourceListEventParams *e) {...}

Remarks

This event fires once for each resource returned when ListResources or GetResourceInfo is called.

Name is the name of the resource.

Path is the full path of the resource.

ResourceType reflects the resource's type. Possible values are:

ModifiedTime and AccessTime reflect the resource's last modified and last access times, in milliseconds relative to the Unix epoch.

Size reflects the size of the file, in bytes. Always 0 for directories.

Permission reflects the resource's permission bits, represented as an octal string (e.g., 755).

Owner is the name of the resource's owner.

Group is the name of the resource's group.

Replication reflects the file's replication factor. Always 0 for directories.

SSLServerAuthentication Event (HadoopDFS Class)

Fired after the server presents its certificate to the client.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireSSLServerAuthentication(HadoopDFSSSLServerAuthenticationEventParams *e);

typedef struct {
  const char *CertEncoded;
  int lenCertEncoded;
  const char *CertSubject;
  const char *CertIssuer;
  const char *Status;
  int Accept;
  int reserved;
} HadoopDFSSSLServerAuthenticationEventParams;


Unicode (Windows)
virtual INT FireSSLServerAuthentication(HadoopDFSSSLServerAuthenticationEventParams *e);

typedef struct {
  LPCSTR CertEncoded;
  INT lenCertEncoded;
  LPCWSTR CertSubject;
  LPCWSTR CertIssuer;
  LPCWSTR Status;
  BOOL Accept;
  INT reserved;
} HadoopDFSSSLServerAuthenticationEventParams;

#define EID_HADOOPDFS_SSLSERVERAUTHENTICATION 7

virtual INT CLOUDSTORAGE_CALL FireSSLServerAuthentication(LPSTR &lpCertEncoded, INT &lenCertEncoded, LPSTR &lpszCertSubject, LPSTR &lpszCertIssuer, LPSTR &lpszStatus, BOOL &bAccept);

class HadoopDFSSSLServerAuthenticationEventParams {
public:
  const QByteArray &CertEncoded();

  const QString &CertSubject();

  const QString &CertIssuer();

  const QString &Status();

  bool Accept();
  void SetAccept(bool bAccept);

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void SSLServerAuthentication(HadoopDFSSSLServerAuthenticationEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireSSLServerAuthentication(HadoopDFSSSLServerAuthenticationEventParams *e) {...}

Remarks

During this event, the client can decide whether or not to continue with the connection process. The Accept parameter is a recommendation on whether to continue or close the connection. This is just a suggestion: application software must use its own logic to determine whether or not to continue.

When Accept is False, Status shows why the verification failed (otherwise, Status contains the string OK). If it is decided to continue, you can override and accept the certificate by setting the Accept parameter to True.

SSLStatus Event (HadoopDFS Class)

Fired when secure connection progress messages are available.

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireSSLStatus(HadoopDFSSSLStatusEventParams *e);

typedef struct {
  const char *Message;
  int reserved;
} HadoopDFSSSLStatusEventParams;


Unicode (Windows)
virtual INT FireSSLStatus(HadoopDFSSSLStatusEventParams *e);

typedef struct {
  LPCWSTR Message;
  INT reserved;
} HadoopDFSSSLStatusEventParams;

#define EID_HADOOPDFS_SSLSTATUS 8

virtual INT CLOUDSTORAGE_CALL FireSSLStatus(LPSTR &lpszMessage);

class HadoopDFSSSLStatusEventParams {
public:
  const QString &Message();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void SSLStatus(HadoopDFSSSLStatusEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireSSLStatus(HadoopDFSSSLStatusEventParams *e) {...}

Remarks

The event is fired for informational and logging purposes only. This event tracks the progress of the connection.

StartTransfer Event (HadoopDFS Class)

This event fires when a document starts transferring (after the headers).

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireStartTransfer(HadoopDFSStartTransferEventParams *e);

typedef struct {
  int Direction;
  int reserved;
} HadoopDFSStartTransferEventParams;


Unicode (Windows)
virtual INT FireStartTransfer(HadoopDFSStartTransferEventParams *e);

typedef struct {
  INT Direction;
  INT reserved;
} HadoopDFSStartTransferEventParams;

#define EID_HADOOPDFS_STARTTRANSFER 9

virtual INT CLOUDSTORAGE_CALL FireStartTransfer(INT &iDirection);

class HadoopDFSStartTransferEventParams {
public:
  int Direction();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void StartTransfer(HadoopDFSStartTransferEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireStartTransfer(HadoopDFSStartTransferEventParams *e) {...}

Remarks

The StartTransfer event is fired when the document text starts transferring from the server to the local host.

The Direction parameter shows whether the client (0) or the server (1) is sending the data.

Transfer Event (HadoopDFS Class)

Fired while a document transfers (delivers document).

Syntax

• C++
• C
• Qt

ANSI (Cross Platform)
virtual int FireTransfer(HadoopDFSTransferEventParams *e);

typedef struct {
  int Direction;
  int64 BytesTransferred;
  int PercentDone;
  const char *Text;
  int lenText;
  int reserved;
} HadoopDFSTransferEventParams;


Unicode (Windows)
virtual INT FireTransfer(HadoopDFSTransferEventParams *e);

typedef struct {
  INT Direction;
  LONG64 BytesTransferred;
  INT PercentDone;
  LPCSTR Text;
  INT lenText;
  INT reserved;
} HadoopDFSTransferEventParams;

#define EID_HADOOPDFS_TRANSFER 10

virtual INT CLOUDSTORAGE_CALL FireTransfer(INT &iDirection, LONG64 &lBytesTransferred, INT &iPercentDone, LPSTR &lpText, INT &lenText);

class HadoopDFSTransferEventParams {
public:
  int Direction();

  qint64 BytesTransferred();

  int PercentDone();

  const QByteArray &Text();

  int EventRetVal();
  void SetEventRetVal(int iRetVal);
};

// To handle, connect one or more slots to this signal.
void Transfer(HadoopDFSTransferEventParams *e);

// Or, subclass HadoopDFS and override this emitter function.
virtual int FireTransfer(HadoopDFSTransferEventParams *e) {...}

Remarks

The Text parameter contains the portion of the document text being received. It is empty if data are being posted to the server.

The BytesTransferred parameter contains the number of bytes transferred in this Direction since the beginning of the document text (excluding HTTP response headers).

The Direction parameter shows whether the client (0) or the server (1) is sending the data.

The PercentDone parameter shows the progress of the transfer in the corresponding direction. If PercentDone can not be calculated the value will be -1.

Note: Events are not re-entrant. Performing time-consuming operations within this event will prevent it from firing again in a timely manner and may affect overall performance.

Certificate Type

This is the digital certificate being used.

Syntax

CloudStorageCertificate (declared in cloudstorage.h)

Remarks

This type describes the current digital certificate. The certificate may be a public or private key. The fields are used to identify or select certificates.

Fields

Constructors

Certificate()

Certificate(const char* lpEncoded, int lenEncoded)

Certificate(int iStoreType, const char* lpStore, int lenStore, const char* lpszStorePassword, const char* lpszSubject)

Firewall Type

The firewall the component will connect through.

Syntax

CloudStorageFirewall (declared in cloudstorage.h)

Remarks

When connecting through a firewall, this type is used to specify different properties of the firewall, such as the firewall Host and the FirewallType.

Fields

Constructors

Firewall()

HDFSDirSummary Type

HDFS directory content summary information.

Syntax

CloudStorageHDFSDirSummary (declared in cloudstorage.h)

Remarks

This type holds and HDFS directory's content summary information.

Fields

Constructors

HDFSDirSummary()

HDFSResource Type

An HDFS file, directory, or symlink.

Syntax

CloudStorageHDFSResource (declared in cloudstorage.h)

Remarks

This type represents and HDFS file, directory, or symlink.

Fields

Constructors

HDFSResource()

Header Type

This is an HTTP header as it is received from the server.

Syntax

CloudStorageHeader (declared in cloudstorage.h)

Remarks

When a header is received through a Header event, it is parsed into a Header type. This type contains a Field, and its corresponding Value.

Fields

Constructors

Header()

Header(const char* lpszField, const char* lpszValue)

Proxy Type

The proxy the component will connect to.

Syntax

CloudStorageProxy (declared in cloudstorage.h)

Remarks

When connecting through a proxy, this type is used to specify different properties of the proxy, such as the Server and the AuthScheme.

Fields

Constructors

Proxy()

Proxy(const char* lpszServer, int iPort)

Proxy(const char* lpszServer, int iPort, const char* lpszUser, const char* lpszPassword)

QueryParam Type

A query parameter to send in the request.

Syntax

CloudStorageQueryParam (declared in cloudstorage.h)

Remarks

This type represents a query parameter to send in the request.

Fields

Constructors

QueryParam()

QueryParam(const char* lpszName, const char* lpszValue)

CloudStorageList Type

Syntax

CloudStorageList<T> (declared in cloudstorage.h)

Remarks

CloudStorageList is a generic class that is used to hold a collection of objects of type T, where T is one of the custom types supported by the HadoopDFS class.

int GetCount() {}

int SetCount(int count) {}

T* Get(int index) {}

T* Set(int index, T* value) {}

CloudStorageStream Type

Syntax

CloudStorageStream (declared in cloudstorage.h)

Remarks

The HadoopDFS class includes one or more API members that take a stream object as a parameter. To use such API members, create a concrete class that implements the CloudStorageStream interface and pass the HadoopDFS class an instance of that concrete class.

When implementing the CloudStorageStream interface's properties and methods, they must behave as described below. If the concrete class's implementation does not behave as expected, undefined behavior may occur.

bool CanRead() { return true; }

bool CanSeek() { return true; }

bool CanWrite() { return true; }

int64 GetLength() = 0;

void Close() {}

int Flush() { return 0; }

int Read(void* buffer, int count) = 0;

int64 Seek(int64 offset, int seekOrigin) = 0;

int Write(const void* buffer, int count) = 0;

Config Settings (HadoopDFS Class)

HadoopDFS Config Settings

<firstlevel>
  <one>value</one>
  <two>
    <item>first</item>
    <item>second</item>
  </two>
  <three>value three</three>
</firstlevel>

{
  "firstlevel": {
    "one": "value",
    "two": ["first", "second"],
    "three": "value three"
  }
}

HTTP Config Settings

TCPClient Config Settings

SSL Config Settings

-----BEGIN CERTIFICATE-----
MIIEKzCCAxOgAwIBAgIRANTET4LIkxdH6P+CFIiHvTowDQYJKoZIhvcNAQELBQAw
... Intermediate Cert ...
eWHV5OW1K53o/atv59sOiW5K3crjFhsBOd5Q+cJJnU+SWinPKtANXMht+EDvYY2w
F0I1XhM+pKj7FjDr+XNj
-----END CERTIFICATE-----
\r \n
-----BEGIN CERTIFICATE-----
MIIEFjCCAv6gAwIBAgIQetu1SMxpnENAnnOz1P+PtTANBgkqhkiG9w0BAQUFADBp
... Root Cert ...
d8q23djXZbVYiIfE9ebr4g3152BlVCHZ2GyPdjhIuLeH21VbT/dyEHHA
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
MIIEKzCCAxOgAwIBAgIRANTET4LIkxdH6P+CFIiHvTowDQYJKoZIhvcNAQELBQAw
... Intermediate Cert ...
eWHV5OW1K53o/atv59sOiW5K3crjFhsBOd5Q+cJJnU+SWinPKtANXMht+EDvYY2w
F0I1XhM+pKj7FjDr+XNj
-----END CERTIFICATE-----
\r \n
-----BEGIN CERTIFICATE-----
MIIEFjCCAv6gAwIBAgIQetu1SMxpnENAnnOz1P+PtTANBgkqhkiG9w0BAQUFADBp
... Root Cert ...
d8q23djXZbVYiIfE9ebr4g3152BlVCHZ2GyPdjhIuLeH21VbT/dyEHHA
-----END CERTIFICATE-----