Creating an OpenSSL Engine to use indigenous ECDH, ECDSA and HASH Algorithms

From OpenSSLWiki
Revision as of 10:31, 9 October 2015 by Oezgan (talk | contribs) (ECDH)

Introduction

This tutorial is intended to provide an example implementation of an OpenSSL Engine such that indigenous cryptographic code for ECDSA and ECDH as well as some sha2 family algorithms can be used in OpenSSL for different purposes.

This guide will not provide the reader with implementation of actual cryptographic primitives but only with the necessary code to embed cryptographic software into OpenSSL as an engine.

To test our implementation we will also write a program “EngineTester” to see the that our engine is actually working.

Preparations

In this tutorial is done on an (X)Ubuntu 15.04, with custom compiled OpenSSL version 1.0.2d. The GCC version is: “gcc (Ubuntu 4.9.2-10ubuntu13) 4.9.2”

Note that the OpenSSL Engine has to be written in pure C, anything C++ related causes OpenSSL not to load the Engine. You can however embed C++ code, with some tricks more on that later. Also the OpenSSL Engine is a shared library object “libXXXX.so”.

If the compiler warns with something like “implicit declaration of …” then the engine will also not work, you have to use strictly C syntax. OpenSSL Engines are stored in “/usr/lib/engines/” on the mentioned Ubuntu 15.04 System others may vary. To make it easy we create a symbolic link in this directory pointing to our output shared library. When we call our engine oezganEngine and the shared library will be liboezganEngine.so, then we create our symbolic link with: sudo ln –s ~/workspace/oezganEngine/Debug/liboezganengine.so liboezgan.so

You can also copy the compiled shared library into the “/usr/lib/engines/” directory but this should be done when the engine is ready to be deployed.You can find the OpenSSL include files in “/usr/include/openssl”.

Beginning

We will start with implementing an engine which only has its name and nothing else. The oezganEngine.c file:

#include <openssl/engine.h>
static const char *engine_oezgan_id = "oezgan";
static const char *engine_oezgan_name = "oezgan engine by Fraunhofer FKIE";

IMPLEMENT_DYNAMIC_CHECK_FN();
IMPLEMENT_DYNAMIC_BIND_FN(bind_helper);

int oezgan_init(ENGINE *e) {
    printf("Oezgan Engine Initializatzion!\n");
    return 786;
}

int bind_helper(ENGINE * e, const char *id)
{
    if (!ENGINE_set_id(e, engine_oezgan_id) ||
            !ENGINE_set_name(e, engine_oezgan_name) ||
            !ENGINE_set_init_function(e,oezgan_init)
    )
        return 0;

    return 1;
}

and the tester oezganEngineTester.cc file:

#include <openssl/engine.h>
#include <stdio.h>
#include <string.h>

int main(int argc, const char* argv[] ) {
    OpenSSL_add_all_algorithms();

    ERR_load_crypto_strings();

    ENGINE_load_dynamic();
    ENGINE *oezgan_engine = ENGINE_by_id("oezgan");

    if( oezgan_engine == NULL )
    {
        printf("Could not Load Oezgan Engine!\n");
        exit(1);
    }
    printf("Oezgan Engine successfully loaded\n");

    int init_res = ENGINE_init(oezgan_engine);
    printf("Engine name: %s init result : %d \n",ENGINE_get_name(oezgan_engine), init_res);
    return 0;
}

Note that we use a c++ file to test the engine. Running the tester gives us the following result:

oezgan@kehf-dev1:~/workspace3/oezganEngineTester/Debug$ ./oezganEngineTester
Oezgan Engine successfully loaded
Oezgan Engine Initializatzion!
Engine name: oezgan engine by Fraunhofer FKIE init result : 786

Our engine is successfully loaded and when it`s asked it gives back its name. Now we proceed by implementing our own random function.

The Random Function

OpenSSL has its own method declarations which our engine has to comply with. In the header file ossl_typ.h we find the declaration s:

typedef struct rand_meth_st RAND_METHOD;
typedef struct ecdh_method ECDH_METHOD;
typedef struct ecdsa_method ECDSA_METHOD;

For now we are only interested in the RAND_METHOD declaration. The RAND_METHOD declaration is actually a macro for the rand_method_st declaration which can be found in rand.h of the OpenSSL includes. Here we see that the rand_method_st has the following format:

struct rand_meth_st {
    void (*seed) (const void *buf, int num);
    int (*bytes) (unsigned char *buf, int num);
    void (*cleanup) (void);
    void (*add) (const void *buf, int num, double entropy);
    int (*pseudorand) (unsigned char *buf, int num);
    int (*status) (void);
};

Thus we see that the OpenSSL random method structure defines six functions where the headers of these functions have specific formats. For simplicity purposes we will only implement the “bytes” and the “status” methods. The status method should return an integer and takes no arguments. Presumably this method is used for indication of the state of the random machine. Our engine`s random machine has no states, therefore we only return a positive integer to indicate a thumbs up state. The random status method:

int oezgan_random_status(void)
{
    return 1;
}

Now we implement our own method for generating a number of random bytes as follows:

//new includes
#include <string.h>
#include <openssl/ossl_typ.h>


int get_random_bytes(unsigned char *buffer, int num) {
    printf("oezgan engine random length %d\n", num);
    memset(buffer,1,num);
    return 99;
}

This method will fill the given buffer with num many ones. Now we indicate our random machine structure as:

RAND_METHOD oezgan_random_method = {
        NULL,                       /* seed */
        get_random_bytes,
        NULL,                       /* cleanup */
        NULL,                       /* add */
        NULL,
        oezgan_random_status,
};

We also modify the bind_helper function from above:

int bind_helper(ENGINE * e, const char *id)
{
    if (!ENGINE_set_id(e, engine_oezgan_id) ||
        !ENGINE_set_name(e, engine_oezgan_name) ||
        !ENGINE_set_init_function(e, oezgan_init) ||
        !ENGINE_set_RAND(e, &oezgan_random_method)
                )
        return 0;
    return 1;
}

In the tester we add the following lines:

ENGINE_set_default_RAND(oezgan_engine);

unsigned char * rand_buf= new unsigned char[5];
int err = RAND_bytes(rand_buf,5);
for(int i= 0; i < 57; i++) {
   printf("%x",rand_buf[i]);
}
printf("\n")
free(rand_buf);

The result is seen below:

oezgan@kehf-dev1:~/workspace3/oezganEngineTester/Debug$ ./oezganEngineTester 
Oezgan Engine successfully loaded
Oezgan Engine Initializatzion!
Engine name: oezgan engine by Fraunhofer FKIE init result : 786 
oezgan engine random length 5
11111

We now successfully implemented an OpenSSL engine that gives always returns a lot of 1s as random values.

Digests

Now we want OpenSSL not only to use our own random function but also to use our sha2 family hash functions. We first implement a digest selector function, which tells OpenSSL which digests are available in our engine. This kind of implementation is adapted from the OpenSSL`s build-in engine ccghost. We will implement only one hash function namely SHA256. Following the the ossl_typ.h header to the evp.h header one can see that the Message Digest Structure is defined as follows:

struct env_md_st {
    int type;
    int pkey_type;
    int md_size;
    unsigned long flags;
    int (*init) (EVP_MD_CTX *ctx);
    int (*update) (EVP_MD_CTX *ctx, const void *data, size_t count);
    int (*final) (EVP_MD_CTX *ctx, unsigned char *md);
    int (*copy) (EVP_MD_CTX *to, const EVP_MD_CTX *from);
    int (*cleanup) (EVP_MD_CTX *ctx);
    /* FIXME: prototype these some day */
    int (*sign) (int type, const unsigned char *m, unsigned int m_length,
                 unsigned char *sigret, unsigned int *siglen, void *key);
    int (*verify) (int type, const unsigned char *m, unsigned int m_length,
                   const unsigned char *sigbuf, unsigned int siglen,
                   void *key);
    int required_pkey_type[5];  /* EVP_PKEY_xxx */
    int block_size;
    int ctx_size;               /* how big does the ctx->md_data need to be */
    /* control function */
    int (*md_ctrl) (EVP_MD_CTX *ctx, int cmd, int p1, void *p2);
} /* EVP_MD */ ;

(Commentaries are original form OpenSSL code). From this we use our own message digest hash SHA256 declaration:

static EVP_MD oezgan_engine_sha256_method=  {
        NID_sha256,
        NID_undef,
        32,
        EVP_MD_FLAG_PKEY_METHOD_SIGNATURE,
        oezgan_engine_sha256_init,
        oezgan_engine_sha256_update,
        oezgan_engine_sha256_final,
        oezgan_engine_sha256_copy,
        oezgan_engine_sha256_cleanup,
        /* FIXME: prototype these some day */
        NULL,
        NULL,
        {NID_undef, NID_undef, 0, 0, 0},
        64, /*Block Size*/
        32, /* how big does the ctx->md_data need to be */
        /* control function */
        NULL,
} ;

SHA256 uses a block size of 512 Bit = 64 byte and resulting digest is 256 Bit = 32 byte long. You can also use your own block size and outcome size but for a realistic approach we stick to the original sha256 standard. Now we need to implement the init, update, final and copy functions for our own sha256 implementation. In our implementation the SHA256 Value is always 2222…

static int oezgan_engine_sha256_init(EVP_MD_CTX *ctx) {
    ctx->update = &oezgan_engine_sha256_update;
    printf("initialized! SHA256\n");
    return 1;
}

static int oezgan_engine_sha256_update(EVP_MD_CTX *ctx,const void *data,size_t count) 
{
    printf("SHA256 update \n");
    unsigned char * digest256 = (unsigned char*) malloc(sizeof(unsigned char)*32);
    memset(digest256,2,32);
    count = 32;
    ctx->md_data = digest256;
    return 1;
}

static int oezgan_engine_sha256_final(EVP_MD_CTX *ctx,unsigned char *md) {
    printf("SHA256 final size of EVP_MD: %d\n", sizeof(EVP_MD));
    memcpy(md,(unsigned char*)ctx->md_data,32);
    return 1;
}

int oezgan_engine_sha256_copy(EVP_MD_CTX *to, const EVP_MD_CTX *from)
{
    printf("Copy SHA256\n");
    if (to->md_data && from->md_data) {
        memcpy(to->md_data, from->md_data,sizeof(from->md_data));
    }
    return 1;
}

static int oezgan_engine_sha256_cleanup(EVP_MD_CTX *ctx) {
    printf("SHA256 cleanup\n");
    if (ctx->md_data)
        memset(ctx->md_data, 0, 32);
    return 1;
}

Now we have to tell OpenSSL that whenever a SHA256 digest is requested use the engine implementation of sha256 this will be the digest selector function.

static int oezgan_digest_ids[] = {NID_sha256};

static int oezgan_engine_digest_selector(ENGINE *e, const EVP_MD **digest,
        const int **nids, int nid) {
    int ok = 1;
    if (!digest) {
        *nids = oezgan_digest_ids;
        printf("\n Digest is empty! Nid:%d\n", nid);
        return 2;
    }
    printf("Digest no %d requested\n",nid);
    if (nid == NID_sha256) {
        *digest = &oezgan_engine_sha256_method;
    }
    else {
        ok = 0;
        *digest = NULL;
    }
    return ok;
}

We also need to modify the bind_helper function again:

int bind_helper(ENGINE * e, const char *id)
{
    if (!ENGINE_set_id(e, engine_oezgan_id) ||
        !ENGINE_set_name(e, engine_oezgan_name) ||
        !ENGINE_set_RAND(e, &oezgan_random_method) ||
        !ENGINE_set_init_function(e, oezgan_init) ||
        !ENGINE_set_digests(e, &oezgan_engine_digest_selector)
    )
        return 0;
    return 1;
}

And we modify our tester to generate us a SHA256 Hash function.

    char * str = "Fraunhofer FKIE Wachtberg!";
    int str_len =  26;
    int er = ENGINE_set_default_digests(oezgan_engine);
    printf("ENGINE SETTING DEFAULT DIGESTS %d\n",er);

    unsigned char * digest = new unsigned char[32];
    unsigned int digestSize = -1;

    EVP_MD_CTX *evp_ctx;
    evp_ctx = EVP_MD_CTX_create();
    er = EVP_DigestInit_ex(evp_ctx, EVP_sha256(),oezgan_engine);
    printf("Digest INIT %d\n",er);
    er = EVP_DigestUpdate(evp_ctx, (unsigned char*)str, str_len);
    printf("Digest Update %d\n",er);
    er = EVP_DigestFinal(evp_ctx, digest, &digestSize);
    printf("Digest Final %d Digest size:%d\n",er,digestSize);
    for(int i= 0; i< digestSize; i++) {
        printf("%x", digest[i]);
    }
    printf("\n");
    EVP_MD_CTX_destroy(evp_ctx);

The result is:

oezgan@kehf-dev1:~/workspace3/oezganEngineTester/Debug$ ./oezganEngineTester 
Oezgan Engine successfully loaded
Oezgan Engine Initializatzion!
Engine name: oezgan engine by Fraunhofer FKIE init result : 786 
oezgan engine random length 5
11111
ENGINE SETTING DEFAULT DIGESTS 1
initialized! SHA256
Digest INIT 1
SHA256 update 
Digest Update 1
SHA256 final size of EVP_MD: 76
SHA256 cleanup
Digest Final 1 Digest size:32
22222222222222222222222222222222

Similarly one can also implement other digest functions. Now we move on to more complicated things like the ECDSA or the ECDH algorithms.

ECDH

We now want to use our own Elliptic Curve Diffie-Hellman Key Agreement function. This is a little bit different from what we have seen above. We get us the definition of the ECDH method structe from the file crypto/ecdh/ecdh.h the definition which is provided in this file is not included in the OpenSSL headers, therefore we have to put it in a new header file which will be included in our engine.

The file oezganEngine.h:

#ifndef OEZGANENGINE_H_
#define OEZGANENGINE_H_
#ifdef  __cplusplus
extern "C" {
#endif

struct ecdh_method {
    const char *name;
    int (*compute_key) (void *key, size_t outlen, const EC_POINT *pub_key,
                        EC_KEY *ecdh, void *(*KDF) (const void *in,
                                                    size_t inlen, void *out,
                                                    size_t *outlen));

# if 0
    int (*init) (EC_KEY *eckey);
    int (*finish) (EC_KEY *eckey);
# endif
    int flags;
    char *app_data;
};

#ifdef  __cplusplus
}
#endif


#endif /* OEZGANENGINE_H_ */

KDF is the Key Derivation Function which will be NULL here. Now we add the following declaration in to our oezganEngine.c file:

ECDH_METHOD arcano_ecdh_method = {
        "Oezgan Engine ECDH Method",
        oezgan_engine_compute_ecdh_key,
# if 0
        NULL,
        NULL,
# endif
        0,
        NULL,
};

And the implementation of the ECDH method:

static int arcano_compute_ecdh_key(void * key, size_t outlen, const EC_POINT *pubkey,
        EC_KEY *eckey, void *(*arcano_ecdh_kdf) (const void *in, size_t inlen, void *out,
                size_t *outlen)) {
    printf("Oezgan Engine ECDH Method\n");

    EC_GROUP *group = EC_KEY_get0_group(eckey);
    //convert pubkey
    int pubkey_len = EC_POINT_point2oct(group, pubkey,
                            POINT_CONVERSION_UNCOMPRESSED,
                              NULL, 0, NULL);
    unsigned char * pubkey_buf = malloc(sizeof(unsigned char)*pubkey_len);
    pubkey_len = EC_POINT_point2oct(group, pubkey,
                            POINT_CONVERSION_UNCOMPRESSED,
                             pubkey_buf, pubkey_len, NULL);

    //convert private key
    const BIGNUM* prikey_bn = BN_new();
    prikey_bn = EC_KEY_get0_private_key(eckey);
    int privkey_len =  BN_num_bytes(prikey_bn);
    unsigned char * privkey_buf = malloc(sizeof(unsigned char)*privkey_len);
    BN_bn2bin(prikey_bn, privkey_buf);

    memset(key,3,pubkey_len);
    outlen = pubkey_len;	

    free(privkey_buf);
    free(pubkey_buf);

    printf("Oezgan engine ECDH method End returning: %d!\n", outlen);

    return outlen;
}

Our implementation of the ECDH Method copies the public key into an unsigned char array pubkey also the private key is converted into an unsigned char array for demonstration purposes since a real ECDH implementation has to calculate the from the values bP = B and aP=A the value abP where a and b are the secret keys of two users Aylin and Boris respectively and P is the base point on the elliptic curve. Again we modify our bind_helper method as follows:

int bind_helper(ENGINE * e, const char *id)
{
    if (!ENGINE_set_id(e, engine_oezgan_id) ||
        !ENGINE_set_name(e, engine_oezgan_name) ||
        !ENGINE_set_RAND(e, &oezgan_random_method) ||
        !ENGINE_set_init_function(e, oezgan_init) ||
        !ENGINE_set_digests(e, &oezgan_engine_digest_selector) ||
        !ENGINE_set_ECDH(e, &oezgan_engine_ecdh_method)
    )
        return 0;
    return 1;
}

To be able to test this we need a certificate file and the corresponding private key file. There are many guides in the internet how to generate such certificates and private key pairs. Later on we will need that certificate file to test our ECDSA methods. The test program will now read in a private key file “ownPrivkey.pem” and try to establish an ECDH key with its own public key. The result will be a lot of 3s, exactly as many as the length of the public key. In our certificate file we use a brainpoolP384r1 curve as well as for the ECDH. Beware that the brainpool curves are not supported in OpenSSL versions < 1.0.2. You can use default NIST Curves such as the secp384r1 instead. The following is added to the tester:

//includes
 #include <openssl/engine.h>
 #include <openssl/crypto.h>
 #include <openssl/evp.h>
 #include <openssl/pem.h>
 #include <stdio.h>
 #include <string.h>


 er = ENGINE_set_default_ECDH(oezgan_engine);
 printf(" %d\N ",er);

 FILE* fp = fopen("./ownPrivkey.pem", "r");
 if (fp == NULL) {
       printf( "Could not open private key file\N");
       exit(1);
 }
 EVP_PKEY *privateKey;
 privateKey= PEM_read_PrivateKey(fp, NULL, 0, NULL);
 if ((privateKey) == NULL) {
      printf("Could not extract private key from file\N");
      exit(1);
 }
 fclose(fp);

 EC_KEY *eckey;
 eckey = EC_KEY_new();
 ECDSA_SIG * ecdsa_sig;
 ecdsa_sig = ECDSA_SIG_new();
 eckey = EVP_PKEY_get1_EC_KEY(privateKey);

 EC_GROUP *ec_group;
 ec_group = EC_GROUP_new_by_curve_name(NID_brainpoolP384r1);
 const EC_POINT* pub_key;
 pub_key = EC_KEY_get0_public_key(eckey);

 unsigned char agreed_value[200];
 EC_KEY *ecdh;
 ecdh = EC_KEY_new();
 EC_KEY_set_group(ecdh, ec_group);
 er = EC_KEY_set_private_key(ecdh, EC_KEY_get0_private_key(eckey));



 int agreed_value_len = ECDH_compute_key(agreed_value, 200,pub_key, ecdh, NULL);

 printf("Oezgan engine Agreed Value: %d\n",agreed_value_len);
 for(int i= 0; i < agreed_value_len; i++) {
        printf("%x", agreed_value[i]);
 }
 printf("\n");

Running the tester results in:

oezgan@kehf-dev1:~/workspace3/oezganEngineTester/Debug$ ./oezganEngineTester 
Oezgan Engine successfully loaded
Oezgan Engine Initializatzion!
Engine name: oezgan engine by Fraunhofer FKIE init result : 786 
oezgan engine random length 5
11111
ENGINE SETTING DEFAULT DIGESTS 1
initialized! SHA256
Digest INIT 1
SHA256 update 
Digest Update 1
SHA256 final size of EVP_MD: 76
SHA256 cleanup
Digest Final 1 Digest size:32
22222222222222222222222222222222
ENGINE SETTING DEFAULT ECDH: 1 
Oezgan Engine ECDH Method
Oezgan engine ECDH method End returning: 97!
Oezgan engine Agreed Value: 97
3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333

ECDSA

Using C++ Implementations

In our engine implementation of random generation, sha256, ECDH and ECDSA we used strict C methods. In practice however it is easier to implement elliptic curve functionality in C++; at least this is the author’s perception. Let’s assume that we have some elliptic curve and sha256 implementation in C++. How can we use this implementation as an engine in OpenSSL? Consider the following diagram.

Wrapper.png

The engine is implemented in pure C; the wrapper which is kind of an interface to the C++ implementation is also implemented in C++. The header of the wrapper provides the engine with the necessary methods to be called; these are declared as extern c functions and have a C method signature. The actual implementations of these extern C functions can include C++ syntax and types. From these functions we use the C++ implementation of the elliptic curve libraries. The wrapper should be compiled a static library together with the elliptic curve C++ implementation.


FILES