Welcome to IPWorks EDI, a suite of enterprise-class software components that facilitate Electronic Data Interchange (EDI) mapping and translation (X12 & EDIFACT), as well as the transmission of secure transactions over the Internet. The components are based on leading EDI-INT protocols for secure EDI communications such as AS2, SFTP, OFTP, RosettaNet, etc.
|AS2ProfileMgr||The AS2ProfileMgr component is used to manage self and trading partner profiles used for AS2 communication.|
|AS2Receiver||The AS2Receiver component is used to process EDI messages and generate receipts.|
|AS2Sender||The AS2Sender component implements an AS2 / EDI-INT client.|
|CertMgr||The CertMgr component is used to manage the digital certificates installed on a system.|
|EDIFACTReader||The EDIFACTReader component is optimized for EDIFACT documents, providing a simple way to parse EDIFACT documents.|
|EDIFACTTranslator||The EDIFACTTranslator component is optimized for EDIFACT translation, providing a simple way to convert EDIFACT documents to and from XML or JSON.|
|EDIFACTValidator||EDIFACTValidator is a lightweight EDI validation component designed for simple document validation.|
|EDIFACTWriter||The EDIFACTWriter component is optimized for EDIFACT documents, providing a simple way to create EDIFACT documents.|
|FTP||The FTP Component can be used to transfer files to and from FTP servers using the FTP protocol.|
|HL7Reader||The HL7Reader component is optimized for HL7 documents, providing a simple way to parse HL7 documents.|
|HL7Translator||The HL7Translator component is optimized for HL7 translation, providing a simple way to convert HL7 documents to and from XML or JSON.|
|HL7Validator||HL7Validator is a lightweight EDI validation component designed for simple document validation.|
|HL7Writer||The HL7Writer component is optimized for HL7 documents, providing a simple way to create HL7 documents.|
|HTMLMailer||The HTMLMailer Component is used to send HTML emails, including embedded images.|
|IMAP||The IMAP Component is used to communicate with IMAP servers using the Internet Message Access Protocol (IMAP).|
|MLLPClient||The MLLPClient component implements the client side of the Minimal Lower Layer Protocol.|
|MLLPServer||The MLLPServer component implements the server side of the Minimal Lower Layer Protocol.|
|OFTPClient||The OFTPClient component implements the Odette File Transfer Protocol.|
|OFTPServer||The OFTPServer component implements the server side of the Odette File Transfer Protocol.|
|POP||The POP Component is used to easily retrieve electronic mail from internet post office servers (POP).|
|SFTP||The SFTP component can be used to transfer files to and from SFTP servers using the SFTP Protocol.|
|SMTP||The SMTP Component is used to send Internet mail using the SMTP protocol (the Internet mail standard).|
|TRADACOMSReader||The TRADACOMSReader component is optimized for TRADACOMS documents, providing a simple way to parse TRADACOMS documents.|
|TRADACOMSTranslator||The TRADACOMSTranslator component is optimized for TRADACOMS translation, providing a simple way to convert TRADACOMS documents to and from XML or JSON.|
|TRADACOMSValidator||TRADACOMSValidator is a lightweight EDI validation component designed for simple document validation.|
|TRADACOMSWriter||The TRADACOMSWriter component is optimized for TRADACOMS documents, providing a simple way to create TRADACOMS documents.|
|VDAReader||The VDAReader component is optimized for VDA documents, providing a simple way to parse VDA documents.|
|VDATranslator||The VDATranslator component is optimized for VDA translation, providing a simple way to convert VDA documents to and from XML or JSON.|
|VDAValidator||VDAValidator is a lightweight EDI validation component designed for simple document validation.|
|VDAWriter||The VDAWriter component is optimized for VDA documents, providing a simple way to create VDA documents.|
|X12Reader||The X12Reader component is optimized for X12 documents, providing a simple way to parse X12 documents.|
|X12Translator||The X12Translator component is optimized for X12 translation, providing a simple way to convert X12 documents to and from XML or JSON.|
|X12Validator||X12Validator is a lightweight EDI validation component designed for simple document validation.|
|X12Writer||The X12Writer component is optimized for X12 documents, providing a simple way to create X12 documents.|
You will always find the latest information about IPWorks EDI at our web site: www.nsoftware.com. We offer free, fully-functional 30-day trials for all of our products, and our technical support staff are happy to answer any questions you may have during your evaluation.
Please direct all technical questions to firstname.lastname@example.org. To help support technicians assist you as quickly as possible, please provide an detailed and accurate description of your problem, the results you expected, and the results that you received while using our product. For questions about licensing and pricing, and all other general inquiries, please contact email@example.com.
Thank you for choosing IPWorks EDI for your development needs. We realize that you have a choice among development tools, and that by choosing us you are counting on us to be a key component in your business. We work around the clock to provide you with ongoing enhancements, support, and innovative products; and we will always do our best to exceed your expectations!
AS2 Security CapabilitiesThe AS2 protocol permits a variety of security features including digital signatures, encryption, SSL, and MDN receipts. These features, when used in combination, offer the same reliability you would expect from a VAN:
|Confidentiality:||S/MIME or SSL encryption ensures that only your intended recipient can read your business documents.|
|Digital Signatures:||Signing your documents allows you to prove to your trading partner that you are indeed the originator, and verifies the integrity of the message.|
|Non-Repudiation of Receipt:||AS2 allows you to request a signed Message Disposition Notification (MDN). Once received, this allows you to prove that your trading partner did indeed receive and process your data.|
All of these security features are implemented using X.509 certificates. You may, at your option, use certificates verified by a trusted CA (Certificate Authority), or generate your own certificates for use in Internet EDI.
Configuring CertificatesTo trade business documents securely, you will need to use digital certificates. The CertMgr component and demo application, included with the package, allow you to manage, import, export, and create certificates.
Certificates come in two basic types: private key certificates and public key certificates. You should use private key certificates for yourself, and obtain public key certificates from your trading partner.
Your Personal Certificates
Before trading business documents you will need to obtain a certificate with a private key. Typically, these are purchased from a Certificate Authority such as Verisign or Thawte. In Internet EDI, it is also common to use self-signed certificates. In either case, such certificates allow you to prove your identity by signing documents, or decrypt documents prepared for you.
Certificates with private keys may be stored in several ways. One common way is to use a file in PKCS12 format (typically with a .pfx or .p12 extension. If you have your certificate stored in a PKCS12 file you can set it as follows:
Setting a Signing Certificate (PFX File, Sender)
as2.SigningCert = new nsoftware.IPWorksEDI.Certificate(CertStoreTypes.cstPFXFile, // File type
"\\mypfx.pfx", // File name
"password", // Password
"CN=My Certificate"); // Subject
(Note that the code on the receiver side is analogous.)
You can also store the contents of a PKCS12 file in a byte array. In this case you would set the store type to cstPFXBlob, and set the store to the entire PFX contents.
In Windows, another common way to use certificates is to install them into the operating system. If you double-click on a PKCS12 file, this is exactly what Windows will do.
Certificates imported to the Windows certificate store can be used by the class, but be sure to mark the private key as exportable. If you don't, the private key cannot be used to perform the digital signing.
Setting a Signing Certificate (System Store, Sender)
as2.SigningCert = new nsoftware.IPWorksEDI.Certificate(CertStoreTypes.cstUser, // Store type
"MY", // Store name
"", // Password
"CN=My Certificate"); // Subject
Note that Windows has user and machine stores, and you can use certificates from either.
Your Trading Partner's Certificates
You will also need to obtain certificates from your trading partners. These will be in PKCS#7 or Base-64 encoded format, and will contain public keys only. Typically, these certificates will have extensions such as .cer, .crt, or .der.
To load a partner's public key from file, use the following:
Setting an Encryption Certificate (Sender)
as2.RecipientCert = new Certificate("\\myPartner.cer");
The RecipientCert, ReceiptSignerCert, and SSLAcceptServerCert properties on the sender side, and the SignerCert property on the receiver side follow this format. Please see Configuring Message Security for information on how to use these properties.
Configuring Message SecurityIn AS2 message security is always applied at the option of the sender. When you send outgoing messages, you need to configure the AS2Sender control to apply the appropriate message security. On the server side, the appropriate message security will automatically be applied based on the client's request. If you want to enforce message security on the server side, you can determine which security options were used by the client, and reject the request if needed.
On the sender side, you instruct the component to sign outgoing documents by setting the SigningCertStore and related properties (see "Using Certificates" for details).
Encryption is applied by setting the RecipientCert property. That certificate will then be used for encryption. You may also configure the algorithm used with EncryptingAlgorithm. The default of "3DES" is highly recommended; however you may also select "AES", "RC2", "DES", or "" (no encryption).
You may use SSL (transport-layer encryption) by posting to an HTTPS URL. If your trading partner is using a commercial certificate, no additional configuration is necessary; if your trading partner is using a self-signed certificate, you will need to set SSLAcceptServerCert.
Outgoing messages may also be compressed by setting CompressionFormat.
Requesting a Receipt
You may request an MDN-based receipt by setting MDNTo. The receipt options are given in MDNOptions, and by default the component will request a signed receipt. If you prefer an unsigned receipt, you may set MDNOptions to an empty string.
You should set ReceiptSignerCert to your partner's signing certificate if it is different than RecipientCert, or if you are not encrypting.
On the server side, you always should set your signing and decryption certificate with Certificate, and your trading partner's (signing) certificate with SignerCert. If you use a different certificate for signing and decryption, you will need to set Certificate to your signing certificate after calling ParseRequest.
When you call ProcessRequest (or ParseRequest), the component will determine the message security used, if any. The encryption type will be stored in EncryptionType, and the signature type will be stored in SignatureType. If an MDN was requested, the appropriate parameters will be set in MDNTo, RequestedSigningProtocol, and ReceiptDeliveryOption.
If an MDNReceipt was generated successfully, you can call SendResponse to deliver it to the sender. If not, you could, for example, return an HTTP error code instead of calling SendResponse.
Client-Side ProcessingClient-side processing of AS2 messages is easily done with the AS2Sender object. To send messages with AS2, you will need to follow the following steps:
1. Specify the needed AS2 identifiers, i.e., AS2From and AS2To.
2. Specify the appropriate certificates and message security, if needed (see "Configuring Message Security" for more details on how to do this).
3. Set MDNTo, if you would like to receive a receipt. You can request a signed or unsigned receipt; you can also request synchronous or asynchronous delivery. See "Configuring Message Security" and "Synchronous and Asynchronous Receipts" for more details.
4. Set your EDI data, using EDIData and EDIType.
5. Set LogDirectory, if you would like to produce detailed logs.
6. Set the URL and post. If you requested a synchronous receipt, it will automatically be checked for and validated (if you requested an asynchronous receipt, more work will be required on your end).
See the AS2Client Demo for an example.
Server-Side ProcessingServer-side processing of AS2 messages is accomplished with the AS2Receiver object. The .NET Edition is designed to interoperate with IIS and ASP.NET. The Java Edition is designed to work with HTTP servlets and a web server such as Apache Tomcat. You may also use the AS2Receiver with your own web server if you prefer, or use it to process signed and/or encrypted AS2 data offline.
To use the receiver in ASP.NET or with servlets, follow the following steps:
1. Set LogDirectory, if you would like to produce detailed logs.
2. Call ReadRequest. This will examine the request headers, validate that the incoming message is a valid message, and determine AS2From and AS2To.
3. Look up the originator in your database, make sure he/she is one of your valid trading partners, and set up the appropriate certificates (see "Using Certificates In AS2").
4. Call ProcessRequest to process the incoming data and generate an MDN receipt.
5. Optionally, catch any exceptions thrown by ProcessRequest. In particular, if there were any problems with the incoming transmission, the component will prepare an MDN with an appropriate error response before throwing an exception. If you want to send this MDN, you will need to handle this exception (otherwise, your application will generate an HTTP error.)
6. Optionally, check the message security used, to make sure it was what you were expecting.
7. Save the EDI data.
8. Invoke SendResponse. This will send either a synchronous or asynchronous MDN (or nothing), depending on what was requested.
See the AS2Server Web Form Demo for an example.
Synchronous and Asynchronous ReceiptsWhen requesting receipts you may request either a synchronous or an asynchronous receipt. A synchronous receipt is always returned in the same HTTP connection as the original request. An asynchronous receipt, however, will be returned via HTTP or e-mail over a separate connection. Processing asynchronous receipts requires more effort, but prevents your HTTP connections from waiting on the server's processing of your data. Requesting asynchronous receipts may be useful if you want to send large documents.
Both synchronous and asynchronous receipts are supported. On the receiver side it requires no effort on your part: the receipt will automatically be returned as requested by the client. On the sender side, requesting asynchronous receipts is a bit more difficult, because you will need to provide a server to process them, and coordinate the sent data to the received receipts.
Please see the Asynch MDN Web Form Demo for more information on receiving asynchronous receipts.
EDI Translation Components
The toolkit includes components for common EDI standards, including EDIFACT, X12, HL7 and more. The components can be used to read, translate, or create documents. The following components are available:
|EDIFACTReader||Parses an incoming EDIFACT document so you may read the data in the document.|
|EDIFACTWriter||Generates new EDIFACT documents from scratch.|
|EDIFACTTranslator||Provides a simple way to convert EDIFACT to XML and vice versa.|
|HL7Reader||Parses an incoming HL7 document so you may read the data in the document.|
|HL7Writer||Generates new HL7 documents from scratch.|
|HL7Translator||Provides a simple way to convert HL7 to XML and vice versa.|
|TRADACOMSReader||Parses an incoming TRADACOMS document so you may read the data in the document.|
|TRADACOMSWriter||Generates new TRADACOMS documents from scratch.|
|TRADACOMSTranslator||Provides a simple way to convert TRADACOMS to XML and vice versa.|
|VDAReader||Parses an incoming VDA document so you may read the data in the document.|
|VDAWriter||Generates new VDA documents from scratch.|
|VDATranslator||Provides a simple way to convert VDA to XML and vice versa.|
|X12Reader||Parses an incoming X12 document so you may read the data in the document.|
|X12Writer||Generates new X12 documents from scratch.|
|X12Translator||Provides a simple way to convert X12 to XML and vice versa.|
EDI Documents: Segments
The primary structure of EDI documents are segments. Each segment contains a specific set of known (expected) data. In many cases, a segment is represented as a single line. Segments are separated by a segment delimiter. A line feed is a common segment delimiter in EDI documents (but it could be something else).
Each segment starts with a tag that identifies the kind of data the segment contains. However, tags are not unique, and the context will determine the kind of segment the tag represents. Examples of tags are ISA, GS, N4, etc. Here's an example X12 segment in which the tag of the segment is BIG and the "~" character is the segment delimiter:
BIG*19971211*00001**A99999-01~A segment can be further decomposed into 0 or more data elements. The sample segment above contains 4 data elements (delimited by '*'), with one of them being empty. Elements are located within a segment by their position, which is why empty elements must be represented with successive asterisks as shown in the example above. However, if an element is optional (doesn't require a value) and there are no other elements with a value after that, it may not appear. In other words, the fact that the segment in the instance document contains 4 data elements does not mean that it always will if the remaining data elements are empty.
In some cases (more common in EDIFACT than X12), a data element itself can be a complex structure, split into components, separated with a different delimiter, for example:
DTM+137:20000101:102'This sample EDIFACT segment uses a single quote (') as the segment delimiter, '+' as the element delimiter, and ':' as the component delimiter, so it has a single data element with 3 components.
EDI Documents: Structure
As described previously, the structure of entire EDI documents is made out of segments, with header/footer segment pairs forming nested envelopes around the data. These envelopes are well known and organized as follows:
- All EDI Interchanges (documents) contain interchange header and footer segments. This is the outer envelope. The interchange header segment contains information such as an interchange ID, where it comes from, and where it should go. The interchange footer segment contains information that allows you to validate that the interchange is complete.
- An EDI Transaction (commonly called a message in EDIFACT) contains the actual EDI data. Again, each transaction contains header and footer segments that identify the type of transaction (i.e. an invoice or a purchase order), the number of segments, etc.
- An Interchange can contain EDI transactions directly, but more commonly transactions are grouped inside functional groups. Each functional group contains its own header and footer segments. The information contained in the group header and footer depends on the specification, but it generally identifies the type of transactions contained in the group and the specification version used.
In EDIFACT and X12, the header and footer segments for interchanges, functional groups, and transactions have known tags. Below is a table that identifies the Segment Type and its corresponding tag in X12 and EDIFACT:
|Segment Type||X12 Tag||EDIFACT Tag|
|Service String Advice||-||UNA|
|Functional Group Header||GS||UNG|
|Functional Group Footer||GE||UNE|
UNA:+.? 'This specifies that the component delimiter will be ':', the element delimiter will be '+', the escape character will be '?' and the segment delimiter will be the single quote ('). The '.' is the decimal point character, and the space ' ' is reserved for future use.
Parsing Transactions: Schemas
Within a transaction, segments can only be interpreted with a prior knowledge of the structure of the transaction. In other words, if you see an arbitrary segment within a transaction (for example, an N1 segment within an X12 purchase order document (850)), there's no way to tell what it means or how it should look without knowing where in the transaction it is located. Because of this, to parse EDI documents a schema is required for the transaction that is being parsed to determine the order in which segments should appear. Note that within EDI documents, segments are not unique. In fact, a schema can specify that a given segment be optional, be required but appear only once, or be repeating so that it appears one or more times in a row.
A loop is basically a series of elements with known tags that repeat as a group. For example: an X12 purchase order document (850) needs to contain the addresses of the places the items will travel through (the producer, the seller, the buyer, where the invoice should be sent, etc). Each address is represented as a set of segments, tagged N1..N4, of which only N1 is required. Consider then this part of a document:
N1*ST*BUYSNACKS PORT*9*1223334445~ N3*1000 N. SAMPLE HIGHWAY~ N4*ATHENS*GA*30603~ N1*BT*BUYSNACKS*9*1223334444~ N3*P.O. BOX 0000~ N1*RE*FOODSELLER*9*12345QQQQ~ N3*P.O. BOX 222222~ N4*DALLAS*TX*723224444~This sample actually contains 3 different N1-loops, for 3 different addresses. The first loop has 3 segments (N1, N3, and N4) and contains the Ship-To address (ST). The second loop has 2 segments (N1 and N3) and contains the Bill-To (BT) address. The last loop has 3 segments again (N1, N3, and N4) and specifies some other address.
It is important to recognize that, schema wise, these are just defined as a single, repeating loop of a set of segments (and not described one by one). Also, in an 850 document for example, there can be other loops in other parts of the document with different meanings and different structure.
Parsing Transactions: Loading and Compiling Schemas
There is no single official standard for EDI Schemas, and the components were designed with this in mind. Rather than require the use of a custom schema format, or require a single schema type, the components support multiple, common, schema formats. This allows for easier integration with existing EDI processing applications.
The following schema formats are supported:
|0 (schemaAutomatic - default)||The schema type is automatically determined based on file extension.|
|1 (schemaBinary)||A binary schema that was previously compiled by calling compile_schema.|
|2 (schemaBizTalk)||BizTalk (XSD): http://msdn.microsoft.com/en-us/library/aa559426(v=BTS.70).aspx|
|3 (schemaSEF)||TIBCO Standard Exchange Format (SEF): https://docs.tibco.com/products/tibco-foresight-edisim-6-18-0|
|5 (schemaAltova)||Altova: http://www.altova.com/|
|6 (schemaJSON - recommended)||ArcESB JSON: https://arc.cdata.com/|
Before loading or writing a document with the reader or writer components, a schema will need to be loaded using the LoadSchema() method. The LoadSchema() method can parse schemas and use them for basic validation and interpretation of EDI documents at runtime.
The components also support parsing an EDI or X12 document without loading a schema. In this scenario you can traverse the document using the xpath property. Note that when using this approach the organization and loop structures of the segments within the transaction are not known.
The schemas can be very big (most are around 1-2MB in size) and parsing them can be memory intensive. For this reason, once a schema is loaded into a component instance, it is rendered into an internal representation and cached there until the next time it is needed. Designing your application to create and keep one instance of the component is a good way to avoid unnecessary schema loading.
Compiled Schemas offer another form of optimization. The components have a CompileSchema method that takes the path to an EDI Schema file and generates a .BIN file that contains a binary representation of the file. You can then load these into the component using the LoadSchema method.
The EDIReader Components
The class allows you to parse an incoming EDI document. This class works in two states, loading an entire document at once, or streaming portions of the document. These states are controlled by the BuildDOM property. By default BuildDOM is set to bdEntireDocument which parses the entire document at once, allowing you to use the xpath property to navigate the document.
To save memory for larger documents, you can choose to parse only sections of the document, instead of the entire document. When build_dom is set to per interchange (bdInterchange ) or per transaction (bdTransaction), the respective section of the document will be available for use with xpath from within the corresponding Start and End events. Finally, you may choose to set build_dom to bdNone, which means no DOM will be built and all data will be available only through events, but also will use very little memory. Below are example steps to parse an entire document:
- First, use load_schema to load a schema file into the class. (Only necessary when preserving document structure).
- Open an EDI document or stream by setting input via input_file or input_data and calling parse.
- If build_dom is set to bdEntireDocument, the events of the class will fire as the document is parsed, and xpath may be set to access any part of the document.
If bdInterchange or bdTransaction are specified, parse is called the entire document will be parsed, with only the specified section being saved in memory at any given time. This means if you wish to set xpath to navigate within the section of the document, you will need to do so within the events of the class to prevent further processing of the document while you access the section. When parsing is completely, only the most recently parsed section will be available for use with xpath
If bdNone is specified, then all document information must be obtained through the events fired during parsing.
During parsing, the class performs basic validation of the incoming document. If validation fails, a warning is generated (fired as an event).
The XPath navigation is done through the xpath property. For example:
EDIReader.XPath = "/IX/FG/TX/N1Loop1/N1";
This example path means the following: Select the first N1 segment within the first iteration of the N1Loop1, within second transaction in the first functional group and interchange.
You can also make use of XPath conditional statements to locate the first element which matches a name=value. For example, you could use the following XPath to locate the path of the first element within any N1Loop1 that has a name=N101 and value=BT:
EDIReader.XPath = "IX/FG/TX/N1Loop1[N101='BT']";
Note that the conditional statements will search the children, but not the grand children of the element on which the conditional statement is applied. For instance in the above example the children of N1Loop1 will be searched, but the grandchildren will not.
Additionally if the schema loaded is a ArcESB JSON schema the element Id (from the schema) can be used in the conditional statement. For instance instead of N101 the following is also acceptable:
EDIReader.XPath = "IX/FG/TX/N1Loop1[98='BT']";
To display the structure of the parsed document use display_schema_info. This is helpful when deciding how to navigate the document.
The EDIWriter Components
The class allows you to create a document from scratch. The class allows you to create an EDI document one segment at a time. Here's how a document would normally be created:
- Call load_schema to load the necessary schemas for the transactions that will be used.
- Specify where to write the output document by setting the output_file property or set_output_stream method, or set neither and check the output_data property.
- Create a new interchange start segment using the start_interchange_header method and set its properties using write_element_string and write_component_string.
- To write a basic element value to the current location, call the write_element_string method. For complex element values, there are two possibilities, elements which are split into components, and elements which repeat. To write these complex element values, use the start_element and end_element methods, with write_component_string and repeat_element methods for writing the values. (Examples available below).
- Create a new functional group using start_functional_group_header and set its properties using write_element_string and write_component_string.
- Create a new transaction using start_transaction_header and set the properties for the header segment.
- Write all the data for the transaction by creating new data segments using start_segment and providing the path of the segment to create using the schema names of the loops and segments, like /N1Loop1/N1.
- Once you are done with the segment, call end_segment.
- Once you are done with the transaction, call create_transaction_footer.
- Once you are done with the functional group, call create_functional_group_footer.
- Once the interchange is complete, call create_interchange_footer.
The EDITranslator Components
The class will convert a document from the format specified by input_format to the format specified by output_format. In practice this allows for converting to XML or JSON from EDI and vice versa.
Before translating from EDI to XML or JSON it is recommended to load a schema using the load_schema method. This ensures additional information can be included in the XML or JSON document. If a schema is specified the XML or JSON will include types and descriptions as element attributes which are useful for interpreting the data.
EDI elements may optionally be renamed when creating XML. To define how an element is renamed add a renaming rule by calling add_renaming_rule.
After calling translate the resulting output will contain the EDI, XML or JSON data as defined by output_format. If the output data is XML the export_xml_schema method may be called to export a schema (.xsd) defining the structure of a valid XML document. XML documents which adhere to this document may be translated from XML to EDI.
Input and Output Properties
The class will determine the source and destination of the input and output based on which properties are set.
The order in which the input properties are checked is as follows:
- output_data: The output data is written to this property if no other destination is specified.