Introduction

Welcome to IPWorks EDI, a suite of enterprise-class software components that facilitate Electronic Data Interchange (EDI) mapping and translation (X12 & EDIFACT), as well as the transmission of secure transactions over the Internet. The components are based on leading EDI-INT protocols for secure EDI communications such as AS2, SFTP, OFTP, RosettaNet, etc.

Included Classes

AS2ProfileMgr	The AS2ProfileMgr component is used to manage self and trading partner profiles used for AS2 communication.
AS2Receiver	The AS2Receiver component is used to process EDI messages and generate receipts.
AS2Sender	The AS2Sender component implements an AS2 / EDI-INT client.
AS4Client	The AS4Client component connects to a server to send or receive files.
AS4Server	The AS4Server component implements an AS4 / ebHandler.
CertMgr	The CertMgr component is used to create, read, and manage certificates.
EDIFACTReader	The EDIFACTReader component is optimized for EDIFACT documents, providing a simple way to parse EDIFACT documents.
EDIFACTTranslator	The EDIFACTTranslator component is optimized for EDIFACT translation, providing a simple way to convert EDIFACT documents to and from XML or JSON.
EDIFACTValidator	EDIFACTValidator is a lightweight EDI validation component designed for simple document validation.
EDIFACTWriter	The EDIFACTWriter component is optimized for EDIFACT documents, providing a simple way to create EDIFACT documents.
FTP	The FTP Component can be used to transfer files to and from FTP servers using the FTP protocol.
HL7Reader	The HL7Reader component is optimized for HL7 documents, providing a simple way to parse HL7 documents.
HL7Translator	The HL7Translator component is optimized for HL7 translation, providing a simple way to convert HL7 documents to and from XML or JSON.
HL7Validator	HL7Validator is a lightweight EDI validation component designed for simple document validation.
HL7Writer	The HL7Writer component is optimized for HL7 documents, providing a simple way to create HL7 documents.
HTMLMailer	The HTMLMailer Component is used to send HTML emails, including embedded images.
IMAP	The IMAP Component is used to communicate with IMAP servers using the Internet Message Access Protocol (IMAP).
MLLPClient	The MLLPClient component implements the client side of the Minimal Lower Layer Protocol.
MLLPServer	The MLLPServer component implements the server side of the Minimal Lower Layer Protocol.
OFTPClient	The OFTPClient component implements the Odette File Transfer Protocol.
OFTPServer	The OFTPServer component implements the server side of the Odette File Transfer Protocol.
POP	The POP Component is used to easily retrieve electronic mail from internet post office servers (POP).
SFTP	The SFTP component can be used to transfer files to and from SFTP servers using the SFTP Protocol.
SMTP	The SMTP Component is used to send Internet mail using the SMTP protocol (the Internet mail standard).
TRADACOMSReader	The TRADACOMSReader component is optimized for TRADACOMS documents, providing a simple way to parse TRADACOMS documents.
TRADACOMSTranslator	The TRADACOMSTranslator component is optimized for TRADACOMS translation, providing a simple way to convert TRADACOMS documents to and from XML or JSON.
TRADACOMSValidator	TRADACOMSValidator is a lightweight EDI validation component designed for simple document validation.
TRADACOMSWriter	The TRADACOMSWriter component is optimized for TRADACOMS documents, providing a simple way to create TRADACOMS documents.
VDAReader	The VDAReader component is optimized for VDA documents, providing a simple way to parse VDA documents.
VDATranslator	The VDATranslator component is optimized for VDA translation, providing a simple way to convert VDA documents to and from XML or JSON.
VDAValidator	VDAValidator is a lightweight EDI validation component designed for simple document validation.
VDAWriter	The VDAWriter component is optimized for VDA documents, providing a simple way to create VDA documents.
X12Reader	The X12Reader component is optimized for X12 documents, providing a simple way to parse X12 documents.
X12Translator	The X12Translator component is optimized for X12 translation, providing a simple way to convert X12 documents to and from XML or JSON.
X12Validator	X12Validator is a lightweight EDI validation component designed for simple document validation.
X12Writer	The X12Writer component is optimized for X12 documents, providing a simple way to create X12 documents.

Additional Information

You will always find the latest information about IPWorks EDI at our web site: www.nsoftware.com. We offer free, fully-functional 30-day trials for all of our products, and our technical support staff are happy to answer any questions you may have during your evaluation.

Please direct all technical questions to support@nsoftware.com. To help support technicians assist you as quickly as possible, please provide an detailed and accurate description of your problem, the results you expected, and the results that you received while using our product. For questions about licensing and pricing, and all other general inquiries, please contact sales@nsoftware.com.

Thank You!

Thank you for choosing IPWorks EDI for your development needs. We realize that you have a choice among development tools, and that by choosing us you are counting on us to be a key component in your business. We work around the clock to provide you with ongoing enhancements, support, and innovative products; and we will always do our best to exceed your expectations!

AS2 Introduction

AS2 Security Capabilities

The AS2 protocol permits a variety of security features including digital signatures, encryption, SSL, and MDN receipts. These features, when used in combination, offer the same reliability you would expect from a VAN:

All of these security features are implemented using X.509 certificates. You may, at your option, use certificates verified by a trusted CA (Certificate Authority), or generate your own certificates for use in Internet EDI.

Configuring Certificates

To trade business documents securely, you will need to use digital certificates. The CertMgr component and demo application, included with the package, allow you to manage, import, export, and create certificates.

Certificates come in two basic types: private key certificates and public key certificates. You should use private key certificates for yourself, and obtain public key certificates from your trading partner.

Your Personal Certificates

Before trading business documents you will need to obtain a certificate with a private key. Typically, these are purchased from a Certificate Authority such as Verisign or Thawte. In Internet EDI, it is also common to use self-signed certificates. In either case, such certificates allow you to prove your identity by signing documents, or decrypt documents prepared for you.

Certificates with private keys may be stored in several ways. One common way is to use a file in PKCS12 format (typically with a .pfx or .p12 extension. If you have your certificate stored in a PKCS12 file you can set it as follows:

Setting a Signing Certificate (PFX File, Sender) as2.SigningCert = new nsoftware.IPWorksEDI.Certificate(CertStoreTypes.cstPFXFile, // File type "\\mypfx.pfx", // File name "password", // Password "CN=My Certificate"); // Subject

(Note that the code on the receiver side is analogous.)

You can also store the contents of a PKCS12 file in a byte array. In this case you would set the store type to cstPFXBlob, and set the store to the entire PFX contents.

In Windows, another common way to use certificates is to install them into the operating system. If you double-click on a PKCS12 file, this is exactly what Windows will do.

Certificates imported to the Windows certificate store can be used by the class, but be sure to mark the private key as exportable. If you don't, the private key cannot be used to perform the digital signing.

Setting a Signing Certificate (System Store, Sender) as2.SigningCert = new nsoftware.IPWorksEDI.Certificate(CertStoreTypes.cstUser, // Store type "MY", // Store name "", // Password "CN=My Certificate"); // Subject

Note that Windows has user and machine stores, and you can use certificates from either.

Your Trading Partner's Certificates

You will also need to obtain certificates from your trading partners. These will be in PKCS#7 or Base-64 encoded format, and will contain public keys only. Typically, these certificates will have extensions such as .cer, .crt, or .der.

To load a partner's public key from file, use the following:

Setting an Encryption Certificate (Sender) as2.RecipientCert = new Certificate("\\myPartner.cer");

The RecipientCert, ReceiptSignerCert, and SSLAcceptServerCert properties on the sender side, and the SignerCert property on the receiver side follow this format. Please see Configuring Message Security for information on how to use these properties.

Configuring Message Security

In AS2 message security is always applied at the option of the sender. When you send outgoing messages, you need to configure the AS2Sender control to apply the appropriate message security. On the server side, the appropriate message security will automatically be applied based on the client's request. If you want to enforce message security on the server side, you can determine which security options were used by the client, and reject the request if needed.

Signing

On the sender side, you instruct the component to sign outgoing documents by setting the SigningCertStore and related properties (see "Using Certificates" for details).

Encryption

Encryption is applied by setting the RecipientCert property. That certificate will then be used for encryption. You may also configure the algorithm used with EncryptionAlgorithm. The default of "3DES" is highly recommended; however you may also select "AES", "RC2", "DES", or "" (no encryption).

SSL

You may use SSL (transport-layer encryption) by posting to an HTTPS URL. If your trading partner is using a commercial certificate, no additional configuration is necessary; if your trading partner is using a self-signed certificate, you will need to set SSLAcceptServerCert.

Compression

Outgoing messages may also be compressed by setting CompressionFormat.

Requesting a Receipt

You may request an MDN-based receipt by setting MDNTo. The receipt options are given in MDNOptions, and by default the component will request a signed receipt. If you prefer an unsigned receipt, you may set MDNOptions to an empty string.

You should set ReceiptSignerCert to your partner's signing certificate if it is different than RecipientCert, or if you are not encrypting.

Server-Side Security

On the server side, you always should set your signing and decryption certificate with Certificate, and your trading partner's (signing) certificate with SignerCert. If you use a different certificate for signing and decryption, you will need to set Certificate to your signing certificate after calling ParseRequest.

When you call ProcessRequest (or ParseRequest), the component will determine the message security used, if any. The encryption type will be stored in EncryptionType, and the signature type will be stored in SignatureType. If an MDN was requested, the appropriate parameters will be set in MDNTo, RequestedSigningProtocol, and ReceiptDeliveryOption.

If an MDNReceipt was generated successfully, you can call SendResponse to deliver it to the sender. If not, you could, for example, return an HTTP error code instead of calling SendResponse.

Client-Side Processing

Client-side processing of AS2 messages is easily done with the AS2Sender object. To send messages with AS2, you will need to follow the following steps:

1. Specify the needed AS2 identifiers, i.e., AS2From and AS2To.

2. Specify the appropriate certificates and message security, if needed (see "Configuring Message Security" for more details on how to do this).

3. Set MDNTo, if you would like to receive a receipt. You can request a signed or unsigned receipt; you can also request synchronous or asynchronous delivery. See "Configuring Message Security" and "Synchronous and Asynchronous Receipts" for more details.

4. Set your EDI data, using EDIData and EDIType.

5. Set LogDirectory, if you would like to produce detailed logs.

6. Set the URL and post. If you requested a synchronous receipt, it will automatically be checked for and validated (if you requested an asynchronous receipt, more work will be required on your end).

See the AS2Client Demo for an example.

Server-Side Processing

Server-side processing of AS2 messages is accomplished with the AS2Receiver object. The .NET Edition is designed to interoperate with IIS and ASP.NET. The Java Edition is designed to work with HTTP servlets and a web server such as Apache Tomcat. You may also use the AS2Receiver with your own web server if you prefer, or use it to process signed and/or encrypted AS2 data offline.

To use the receiver in ASP.NET or with servlets, follow the following steps:

1. Set LogDirectory, if you would like to produce detailed logs.

2. Call ReadRequest. This will examine the request headers, validate that the incoming message is a valid message, and determine AS2From and AS2To.

3. Look up the originator in your database, make sure he/she is one of your valid trading partners, and set up the appropriate certificates (see "Using Certificates In AS2").

4. Call ProcessRequest to process the incoming data and generate an MDN receipt.

5. Optionally, catch any exceptions thrown by ProcessRequest. In particular, if there were any problems with the incoming transmission, the component will prepare an MDN with an appropriate error response before throwing an exception. If you want to send this MDN, you will need to handle this exception (otherwise, your application will generate an HTTP error.)

6. Optionally, check the message security used, to make sure it was what you were expecting.

7. Save the EDI data.

8. Invoke SendResponse. This will send either a synchronous or asynchronous MDN (or nothing), depending on what was requested.

See the AS2Server Web Form Demo for an example.

Synchronous and Asynchronous Receipts

When requesting receipts you may request either a synchronous or an asynchronous receipt. A synchronous receipt is always returned in the same HTTP connection as the original request. An asynchronous receipt, however, will be returned via HTTP or e-mail over a separate connection. Processing asynchronous receipts requires more effort, but prevents your HTTP connections from waiting on the server's processing of your data. Requesting asynchronous receipts may be useful if you want to send large documents.

Both synchronous and asynchronous receipts are supported. On the receiver side it requires no effort on your part: the receipt will automatically be returned as requested by the client. On the sender side, requesting asynchronous receipts is a bit more difficult, because you will need to provide a server to process them, and coordinate the sent data to the received receipts.

Please see the Asynch MDN Web Form Demo for more information on receiving asynchronous receipts.

EDI Translation Components

The toolkit includes components for common EDI standards, including EDIFACT, X12, HL7 and more. The components can be used to read, translate, or create documents. The following components are available:

EDI Documents: Segments

The primary structure of EDI documents are segments. Each segment contains a specific set of known (expected) data. In many cases, a segment is represented as a single line. Segments are separated by a segment delimiter. A line feed is a common segment delimiter in EDI documents (but it could be something else).

Each segment starts with a tag that identifies the kind of data the segment contains. However, tags are not unique, and the context will determine the kind of segment the tag represents. Examples of tags are ISA, GS, N4, etc. Here's an example X12 segment in which the tag of the segment is BIG and the "~" character is the segment delimiter:

BIG*19971211*00001**A99999-01~

A segment can be further decomposed into 0 or more data elements. The sample segment above contains 4 data elements (delimited by '*'), with one of them being empty. Elements are located within a segment by their position, which is why empty elements must be represented with successive asterisks as shown in the example above. However, if an element is optional (doesn't require a value) and there are no other elements with a value after that, it may not appear. In other words, the fact that the segment in the instance document contains 4 data elements does not mean that it always will if the remaining data elements are empty.

In some cases (more common in EDIFACT than X12), a data element itself can be a complex structure, split into components, separated with a different delimiter, for example:

DTM+137:20000101:102'

This sample EDIFACT segment uses a single quote (') as the segment delimiter, '+' as the element delimiter, and ':' as the component delimiter, so it has a single data element with 3 components.

EDI Documents: Structure

As described previously, the structure of entire EDI documents is made out of segments, with header/footer segment pairs forming nested envelopes around the data. These envelopes are well known and organized as follows:

• All EDI Interchanges (documents) contain interchange header and footer segments. This is the outer envelope. The interchange header segment contains information such as an interchange ID, where it comes from, and where it should go. The interchange footer segment contains information that allows you to validate that the interchange is complete.
• An EDI Transaction (commonly called a message in EDIFACT) contains the actual EDI data. Again, each transaction contains header and footer segments that identify the type of transaction (i.e. an invoice or a purchase order), the number of segments, etc.
• An Interchange can contain EDI transactions directly, but more commonly transactions are grouped inside functional groups. Each functional group contains its own header and footer segments. The information contained in the group header and footer depends on the specification, but it generally identifies the type of transactions contained in the group and the specification version used.

In EDIFACT and X12, the header and footer segments for interchanges, functional groups, and transactions have known tags. Below is a table that identifies the Segment Type and its corresponding tag in X12 and EDIFACT:

Note: The Service String Advice segment only occurs in EDIFACT documents and may appear at the start of the document. It serves to let the parser determine the delimiters that will be used in the rest of the document. Here's an example of a Service String Advice Segment:

UNA:+.? '

This specifies that the component delimiter will be ':', the element delimiter will be '+', the escape character will be '?' and the segment delimiter will be the single quote ('). The '.' is the decimal point character, and the space ' ' is reserved for future use.

Parsing Transactions: Schemas

Within a transaction, segments can only be interpreted with a prior knowledge of the structure of the transaction. In other words, if you see an arbitrary segment within a transaction (for example, an N1 segment within an X12 purchase order document (850)), there's no way to tell what it means or how it should look without knowing where in the transaction it is located. Because of this, to parse EDI documents a schema is required for the transaction that is being parsed to determine the order in which segments should appear. Note that within EDI documents, segments are not unique. In fact, a schema can specify that a given segment be optional, be required but appear only once, or be repeating so that it appears one or more times in a row.

A loop is basically a series of elements with known tags that repeat as a group. For example: an X12 purchase order document (850) needs to contain the addresses of the places the items will travel through (the producer, the seller, the buyer, where the invoice should be sent, etc). Each address is represented as a set of segments, tagged N1..N4, of which only N1 is required. Consider then this part of a document:

N1*ST*BUYSNACKS PORT*9*1223334445~
N3*1000 N. SAMPLE HIGHWAY~
N4*ATHENS*GA*30603~
N1*BT*BUYSNACKS*9*1223334444~
N3*P.O. BOX 0000~
N1*RE*FOODSELLER*9*12345QQQQ~
N3*P.O. BOX 222222~
N4*DALLAS*TX*723224444~

This sample actually contains 3 different N1-loops, for 3 different addresses. The first loop has 3 segments (N1, N3, and N4) and contains the Ship-To address (ST). The second loop has 2 segments (N1 and N3) and contains the Bill-To (BT) address. The last loop has 3 segments again (N1, N3, and N4) and specifies some other address.

It is important to recognize that, schema wise, these are just defined as a single, repeating loop of a set of segments (and not described one by one). Also, in an 850 document for example, there can be other loops in other parts of the document with different meanings and different structure.

Parsing Transactions: Loading and Compiling Schemas

There is no single official standard for EDI Schemas, and the components were designed with this in mind. Rather than require the use of a custom schema format, or require a single schema type, the components support multiple, common, schema formats. This allows for easier integration with existing EDI processing applications.

The following schema formats are supported:

Before loading or writing a document with the reader or writer components, a schema will need to be loaded using the LoadSchema() method. The LoadSchema() method can parse schemas and use them for basic validation and interpretation of EDI documents at runtime.

The components also support parsing an EDI or X12 document without loading a schema. In this scenario you can traverse the document using the XPath property. Note that when using this approach the organization and loop structures of the segments within the transaction are not known.

The schemas can be very big (most are around 1-2MB in size) and parsing them can be memory intensive. For this reason, once a schema is loaded into a component instance, it is rendered into an internal representation and cached there until the next time it is needed. Designing your application to create and keep one instance of the component is a good way to avoid unnecessary schema loading.

Compiled Schemas offer another form of optimization. The components have a CompileSchema method that takes the path to an EDI Schema file and generates a .BIN file that contains a binary representation of the file. You can then load these into the component using the LoadSchema method.

The EDIReader Components

The class allows you to parse an incoming EDI document. This class works in two states, loading an entire document at once, or streaming portions of the document. These states are controlled by the BuildDOM property. By default BuildDOM is set to bdEntireDocument which parses the entire document at once, allowing you to use the XPath property to navigate the document.

To save memory for larger documents, you can choose to parse only sections of the document, instead of the entire document. When BuildDOM is set to per interchange (bdInterchange ) or per transaction (bdTransaction), the respective section of the document will be available for use with XPath from within the corresponding Start and End events. Finally, you may choose to set BuildDOM to bdNone, which means no DOM will be built and all data will be available only through events, but also will use very little memory. Below are example steps to parse an entire document:

1. First, use LoadSchema to load a schema file into the class. (Only necessary when preserving document structure).
2. Open an EDI document or stream by setting input via InputFile or InputData and calling Parse.
3. If BuildDOM is set to bdEntireDocument, the events of the class will fire as the document is parsed, and XPath may be set to access any part of the document.

   If bdInterchange or bdTransaction are specified, Parse is called the entire document will be parsed, with only the specified section being saved in memory at any given time. This means if you wish to set XPath to navigate within the section of the document, you will need to do so within the events of the class to prevent further processing of the document while you access the section. When parsing is completely, only the most recently parsed section will be available for use with XPath

   If bdNone is specified, then all document information must be obtained through the events fired during parsing.

During parsing, the class performs basic validation of the incoming document. If validation fails, a warning is generated (fired as an event).

The XPath navigation is done through the XPath property. For example: EDIReader.XPath = "/IX[1]/FG[1]/TX[2]/N1Loop1[1]/N1[1]"; This example path means the following: Select the first N1 segment within the first iteration of the N1Loop1, within second transaction in the first functional group and interchange.

You can also make use of XPath conditional statements to locate the first element which matches a name=value. For example, you could use the following XPath to locate the path of the first element within any N1Loop1 that has a name=N101 and value=BT: EDIReader.XPath = "IX[1]/FG[1]/TX[1]/N1Loop1[N101='BT']";

Note that the conditional statements will search the children, but not the grand children of the element on which the conditional statement is applied. For instance in the above example the children of N1Loop1 will be searched, but the grandchildren will not.

Additionally if the schema loaded is a ArcESB JSON schema the element Id (from the schema) can be used in the conditional statement. For instance instead of N101 the following is also acceptable: EDIReader.XPath = "IX[1]/FG[1]/TX[1]/N1Loop1[98='BT']";

To display the structure of the parsed document use DisplaySchemaInfo. This is helpful when deciding how to navigate the document.

The EDIWriter Components

The class allows you to create a document from scratch. The class allows you to create an EDI document one segment at a time. Here's how a document would normally be created:

1. Call LoadSchema to load the necessary schemas for the transactions that will be used.
2. Specify where to write the output document by setting the OutputFile property or SetOutputStream method, or set neither and check the OutputData property.
3. Create a new interchange start segment using the StartInterchangeHeader method and set its properties using WriteElementString and WriteComponentString.
4. To write a basic element value to the current location, call the WriteElementString method. For complex element values, there are two possibilities, elements which are split into components, and elements which repeat. To write these complex element values, use the StartElement and EndElement methods, with WriteComponentString and RepeatElement methods for writing the values. (Examples available below).
5. Create a new functional group using StartFunctionalGroupHeader and set its properties using WriteElementString and WriteComponentString.
6. Create a new transaction using StartTransactionHeader and set the properties for the header segment.
7. Write all the data for the transaction by creating new data segments using StartSegment and providing the path of the segment to create using the schema names of the loops and segments, like /N1Loop1/N1.
8. Once you are done with the segment, call EndSegment.
9. Once you are done with the transaction, call CreateTransactionFooter.
10. Once you are done with the functional group, call CreateFunctionalGroupFooter.
11. Once the interchange is complete, call CreateInterchangeFooter.

The EDITranslator Components

The class will convert a document from the format specified by InputFormat to the format specified by OutputFormat. In practice this allows for converting to XML or JSON from EDI and vice versa.

Before translating from EDI to XML or JSON it is recommended to load a schema using the LoadSchema method. This ensures additional information can be included in the XML or JSON document. If a schema is specified the XML or JSON will include types and descriptions as element attributes which are useful for interpreting the data.

EDI elements may optionally be renamed when creating XML. To define how an element is renamed add a renaming rule by calling AddRenamingRule.

After calling Translate the resulting output will contain the EDI, XML or JSON data as defined by OutputFormat. If the output data is XML the ExportXMLSchema method may be called to export a schema (.xsd) defining the structure of a valid XML document. XML documents which adhere to this document may be translated from XML to EDI.

Input and Output Properties

The class will determine the source and destination of the input and output based on which properties are set.

The order in which the input properties are checked is as follows:

• InputFile
• InputData

When a valid source is found the search stops. The order in which the output properties are checked is as follows:

• OutputFile
• OutputData: The output data is written to this property if no other destination is specified.