S/MIME: Analyze and decrypt/verify message

Discuss this help topic in SecureBlackbox Forum

When a message is received, you need to parse it, check if the message is secured, and if it is, decrypt and/or verify it. To do this

Load and parse the message as described in the corresponding how-to article;
Check if the message is S/MIME-protected by inspecting Message.MainPart.MessagePartHandler property. If the value of this property is not null/nil, and it is of class TElMessagePartHandlerSMime, then the message is considered to be S/MIME protected.

Possible message formats

There are several combinations of methods, how the message can be encrypted and/or signed. Suppose you have created a message with a text part and an attachment, then encrypted and signed it in different ways. Below all possible combinations of signatures and encryptions are listed and explained.

Signed message with a "clear format" signature. This signature format consists of 2 parts: one part for signed data (in the example below, this part is "multipart/mixed") and one part for the signature (application/pkcs7-signature).
After parsing a message call the MessagePartHandler.DecoderIsSignedOnly() method. If this method returns true, then the message is signed with this type of signature. In this case, it is needed to check the signature verification result (available as MessagePartHandler.DecoderSignIsCorrectly property), and then process the signed part (available with a call to MainPart.GetPart(0)) in the same way as unsigned messages of such structure are processed.
The message in this format has the following structure:
```
[0] multipart/signed
    [1] multipart/mixed
        [2] text/plain
        [2] application/octet-stream (attachment)
    [1] application/pkcs7-signature (attachment)
```
Signed message with an "enveloped format" signature. This signature format consists of a single part. The signed data is enveloped inside that part. To get an idea if a message is signed with this type of signature, call the MessagePartHandler.DecoderIsSigned() method (it has to return true) and the MessagePartHandler.DecoderIsSignedOnly() method (it has to return false). After calling these methods it is needed to call the MessagePartHandler.Decode() method. This method decodes the enveloped data and makes it available as a value of MessagePartHandler.DecodedPart property. Once the data is decoded, it is necessary to check the signature verification result (available as MessagePartHandler.DecoderSignIsCorrectly property) and then process the enveloped part in the same way as unsigned messages of such structure are processed.
The message in this format has the following structure:
```
[0] application/pkcs7-mime; smime-type=signed-data  (attachment)
Decoded part:
[0] multipart/mixed
    [1] text/plain
    [1] application/octet-stream  (attachment)
```
Encrypted message. Encrypted-only messages consist of a single part (just as a signed message with "enveloped format" signature). Call the MessagePartHandler.DecoderIsCrypted() method to determine if the message is encrypted. After this you need to provide a decryption certificate. This is done by assigning a certificate storage with a certificate suitable for decryption (this should be one or more certificates with associated private key(s), issued to one or more recipients of the message) to MessagePartHandler.CertificatesStorage property. Next call the MessagePartHandler.Decode() method to decrypt the message data. Once the message is decrypted, its data becomes available as a value of MessagePartHandler.DecodedPart property.
The message in this format has the following structure:
```
[0] application/pkcs7-mime; smime-type=enveloped-data  (attachment)
Decoded part:
[0] multipart/mixed
    [1] text/plain
    [1] application/octet-stream  (attachment)
```
Signed and encrypted message with "clear format" signature. Such message is a combination of message structures described above in sections 3 and 1. To decode such message, it is needed to decrypt it first as described in section 3, then verify the signature of the signed part as described in section 1.
The message in this format has the following structure:
```
[0] application/pkcs7-mime; smime-type=enveloped-data  (attachment)
Decoded part:
[0] multipart/signed     (encrypted)
    [1] multipart/mixed  (encrypted)(signed)
        [2] text/plain   (encrypted)(signed)
        [2] application/octet-stream (attachment)(encrypted)(signed)
    [1] application/pkcs7-signature (attachment)(encrypted)
```
Signed and encrypted message with "enveloped format" signature. Such message is a combination of message structures described above in section 3 and 2. To decode such message, it is needed to decrypt it first (as described in section 3), then decode the data from the signed part (as described in section 2).
The message in this format has the following structure:
```
[0] application/pkcs7-mime; smime-type=enveloped-data  (attachment)
Decoded part:
[0] application/pkcs7-mime; smime-type=signed-data  (attachment)(encrypted)
Decoded part 2:
[0] multipart/mixed     (encrypted)(signed)
    [1] text/plain      (encrypted)(signed)
    [1] application/octet-stream  (attachment)(encrypted)(signed)
```
Also, messages could be first encrypted, then signed. Both signature types could be used to sign the encrypted part.
Encrypted and signed message with "clear format" signature. Such message consists of 2 parts - first one is the encrypted data and another one is the signature. First, it is needed to check the signature as described above in section 1.
The message in this format has the following structure:
```
[0] multipart/signed
    [1] application/pkcs7-mime; smime-type=enveloped-data  (attachment)(signed)
    [1] application/pkcs7-signature (attachment)
```
Next it is needed to check Message.MainPart.GetPart(0).MessagePartHandler property. If it is not null/nil and it is of class TElMessagePartHandlerSMime, call its MessagePartHandler.DecoderIsCrypted() method to determine if the part is encrypted. Then assign a certificate storage which contains the private key needed to decrypt the data, as described in section 3 above, and call the MessagePartHandler.Decode() method to decrypt the message data. After this, the DecodedPart property will contain the decrypted message part:
```
[0] multipart/mixed     (signed)(encrypted)
    [1] text/plain      (signed)(encrypted)
    [1] application/octet-stream  (attachment)(signed)(encrypted)
```
Encrypted and signed message with "enveloped format" signature. Such message is very similar to the one described in section 5. But in this case the encrypted data is inside of the signed part. To decode such message it is needed to decode it first as described in section 2, then decrypt the data from the encrypted part as described in section 3.
The message in this format has the following structure:
```
[0] application/pkcs7-mime; smime-type=signed-data  (attachment)
```
Decoded part:
```
[0] application/pkcs7-mime; smime-type=enveloped-data  (attachment)(signed)
```
Decoded part 2:
```
[0] multipart/mixed     (signed)(encrypted)
    [1] text/plain      (signed)(encrypted)
    [1] application/octet-stream  (attachment)(signed)(encrypted)
```

Below is a code snippet that analyzes a message, decrypts it if necessary, verifies its signature (if present) and returns the data part for further processing:

C#:


// The method processes the passed message, decrypts its data (if the message
// is encrypted), checks included signature (if any) and returns the decoded
// message part which should be handled as the main part of an unprotected message

TElMessagePart ProcessSMimeMessage(TElMessage message)
{
    if (message == null || message.MainPart == null)
        throw new ArgumentNullException();

    // check if the message is S/MIME protected
    if (!(message.MainPart.MessagePartHandler is TElMessagePartHandlerSMime))
    {
        // the message is not S/MIME protected, so just return its main part
        return message.MainPart;
    }

    // typecast the handler for further use
    TElMessagePartHandlerSMime handler =
        (TElMessagePartHandlerSMime)message.MainPart.MessagePartHandler;

    // check if the message is encrypted
    if (handler.DecoderIsCrypted())
    {
        TElMessagePart decryptedPart = ProcessEncryptedMessage(message, handler);

        // check if the decrypted message part is signed also
        if ((decryptedPart != null) &&
            (decryptedPart.MessagePartHandler is TElMessagePartHandlerSMime))
        {
            handler = (TElMessagePartHandlerSMime)decryptedPart.MessagePartHandler;

            if (handler.DecoderIsSigned())
                return ProcessSignedMessage(decryptedPart);
        }

        return decryptedPart;
    }
    else
    // check if the message is signed
    if (handler.DecoderIsSigned())
    {
        TElMessagePart signedPart = ProcessSignedMessage(message.MainPart);

        // check if the signed part is encrypted also
        if ((signedPart != null) &&
            (signedPart.MessagePartHandler is TElMessagePartHandlerSMime))
        {
            handler = (TElMessagePartHandlerSMime)signedPart.MessagePartHandler;

            if (handler.DecoderIsCrypted())
                return ProcessEncryptedMessage(message, handler);
        }

        return signedPart;
    }

    // something goes wrong
    return null;
}

// The method processes the encrypted message part, decrypts it and
// returns the decrypted message part

TElMessagePart ProcessEncryptedMessage(TElMessage message,
                                       TElMessagePartHandlerSMime handler)
{
    if (message == null || handler == null)
        throw new ArgumentNullException();

    // check message's To_ and Cc address lists and load the appropriate
    // certificate and its private key for message decryption
    TElMemoryCertStorage decryptingStorage = new TElMemoryCertStorage();
    ...

    handler.CertificatesStorage = decryptingStorage;

    int err = handler.Decode(false);
    if (err != 0)
        throw new Exception(String.Format(
            "Failed to decrypt the message: (code {0}) {1}",
            err, handler.ErrorText));

    return handler.DecodedPart;
}

TElMessagePart ProcessSignedMessage(TElMessagePart part)
{
    if (part == null)
        throw new ArgumentNullException();

    // check if the part is protected
    if (part.MessagePartHandler == null)
        return part;

    // check if the part is S/MIME protected
    if (!(part.MessagePartHandler is TElMessagePartHandlerSMime))
        throw new Exception("Invalid message part handler type");

    TElMessagePartHandlerSMime handler =
        (TElMessagePartHandlerSMime)part.MessagePartHandler;

    // check if the message part is signed
    if (!handler.DecoderIsSigned())
    {
        if (handler.DecodedPart != null)
            return handler.DecodedPart;

        return part;
    }

    // check if the message part is signed with "clear format" signature
    if (handler.DecoderIsSignedOnly())
    {
        // check if the part is really a "multipart/signed" part
        if (!part.IsMultipart() ||
            !String.Equals(part.ContentSubtype, "signed",
                StringComparison.OrdinalIgnoreCase) ||
            (part.PartsCount != 2))
            throw new Exception("Invalid message part type or invalid signature structure");

        // is the signature verified ok?
        if (!handler.DecoderSignIsCorrectly)
            throw new Exception(
                String.Format("Failed to verify the signature: {0}",
                handler.ErrorText));

        if (handler.DecoderSignCertStorage != null)
        {
            // verify the signing certificate(s) available in the storage
        }

        // check if the part #0 is the data part
        TElMessagePart dataPart = part.GetPart(0);

        if (!dataPart.IsApplication() ||
            !String.Equals(dataPart.ContentSubtype, "pkcs7-signature",
                           StringComparison.OrdinalIgnoreCase))
            return dataPart;

        // check if the part #1 is the data part
        dataPart = part.GetPart(1);

        if (!dataPart.IsApplication() ||
            !String.Equals(dataPart.ContentSubtype, "pkcs7-signature",
                            StringComparison.OrdinalIgnoreCase))
            throw new Exception("Invalid signature structure");

        return dataPart;
    }

    // the message part is signed with "enveloped format" signature;
    // it is needed to decode the enveloped data

    int err = handler.Decode(false);
    if (!handler.DecoderSignIsCorrectly)
        throw new Exception(
            String.Format("Failed to verify the signature: (code {0}) {1}",
            err, handler.ErrorText));

    if (handler.DecoderSignCertStorage != null)
    {
        // verify signing certificate(s) available in the storage
    }

    return handler.DecodedPart;
}

How To articles about S/MIME

Discuss this help topic in SecureBlackbox Forum