Creating an OpenSSL Engine to use indigenous ECDH, ECDSA and HASH Algorithms

From OpenSSLWiki
Revision as of 09:01, 9 October 2015 by Oezgan (talk | contribs) (Beginning)


This tutorial is intended to provide an example implementation of an OpenSSL Engine such that indigenous cryptographic code for ECDSA and ECDH as well as some sha2 family algorithms can be used in OpenSSL for different purposes.

This guide will not provide the reader with implementation of actual cryptographic primitives but only with the necessary code to embed cryptographic software into OpenSSL as an engine.

To test our implementation we will also write a program “EngineTester” to see the that our engine is actually working.


In this tutorial is done on an (X)Ubuntu 15.04, with custom compiled OpenSSL version 1.0.2d. The GCC version is: “gcc (Ubuntu 4.9.2-10ubuntu13) 4.9.2”

Note that the OpenSSL Engine has to be written in pure C, anything C++ related causes OpenSSL not to load the Engine. You can however embed C++ code, with some tricks more on that later. Also the OpenSSL Engine is a shared library object “”.

If the compiler warns with something like “implicit declaration of …” then the engine will also not work, you have to use strictly C syntax. OpenSSL Engines are stored in “/usr/lib/engines/” on the mentioned Ubuntu 15.04 System others may vary. To make it easy we create a symbolic link in this directory pointing to our output shared library. When we call our engine oezganEngine and the shared library will be, then we create our symbolic link with: sudo ln –s ~/workspace/oezganEngine/Debug/

You can also copy the compiled shared library into the “/usr/lib/engines/” directory but this should be done when the engine is ready to be deployed.You can find the OpenSSL include files in “/usr/include/openssl”.


We will start with implementing an engine which only has its name and nothing else. The oezganEngine.c file:

#include <openssl/engine.h>
static const char *engine_oezgan_id = "oezgan";
static const char *engine_oezgan_name = "oezgan engine by Fraunhofer FKIE";


int oezgan_init(ENGINE *e) {
    printf("Oezgan Engine Initializatzion!\n");
    return 786;

int bind_helper(ENGINE * e, const char *id)
    if (!ENGINE_set_id(e, engine_oezgan_id) ||
            !ENGINE_set_name(e, engine_oezgan_name) ||
        return 0;

    return 1;

and the tester file:

#include <openssl/engine.h>
#include <stdio.h>
#include <string.h>

int main(int argc, const char* argv[] ) {


    ENGINE *oezgan_engine = ENGINE_by_id("oezgan");

    if( oezgan_engine == NULL )
        printf("Could not Load Oezgan Engine!\n");
    printf("Oezgan Engine successfully loaded\n");

    int init_res = ENGINE_init(oezgan_engine);
    printf("Engine name: %s init result : %d \n",ENGINE_get_name(oezgan_engine), init_res);
    return 0;

Note that we use a c++ file to test the engine. Running the tester gives us the following result:

oezgan@kehf-dev1:~/workspace3/oezganEngineTester/Debug$ ./oezganEngineTester
Oezgan Engine successfully loaded
Oezgan Engine Initializatzion!
Engine name: oezgan engine by Fraunhofer FKIE init result : 786

Our engine is successfully loaded and when it`s asked it gives back its name. Now we proceed by implementing our own random function.

The Random Function

OpenSSL has its own method declarations which our engine has to comply with. In the header file ossl_typ.h we find the declaration s:

typedef struct rand_meth_st RAND_METHOD;
typedef struct ecdh_method ECDH_METHOD;
typedef struct ecdsa_method ECDSA_METHOD;

For now we are only interested in the RAND_METHOD declaration. The RAND_METHOD declaration is actually a macro for the rand_method_st declaration which can be found in rand.h of the OpenSSL includes. Here we see that the rand_method_st has the following format:

struct rand_meth_st {
    void (*seed) (const void *buf, int num);
    int (*bytes) (unsigned char *buf, int num);
    void (*cleanup) (void);
    void (*add) (const void *buf, int num, double entropy);
    int (*pseudorand) (unsigned char *buf, int num);
    int (*status) (void);

Thus we see that the OpenSSL random method structure defines six functions where the headers of these functions have specific formats. For simplicity purposes we will only implement the “bytes” and the “status” methods. The status method should return an integer and takes no arguments. Presumably this method is used for indication of the state of the random machine. Our engine`s random machine has no states, therefore we only return a positive integer to indicate a thumbs up state. The random status method:

int oezgan_random_status(void)
    return 1;

Now we implement our own method for generating a number of random bytes as follows:

//new includes
#include <string.h>
#include <openssl/ossl_typ.h>

int get_random_bytes(unsigned char *buffer, int num) {
    printf("oezgan engine random length %d\n", num);
    return 99;

This method will fill the given buffer with num many ones. Now we indicate our random machine structure as:

RAND_METHOD oezgan_random_method = {
        NULL,                       /* seed */
        NULL,                       /* cleanup */
        NULL,                       /* add */

We also modify the bind_helper function from above:

int bind_helper(ENGINE * e, const char *id)
    if (!ENGINE_set_id(e, engine_oezgan_id) ||
        !ENGINE_set_name(e, engine_oezgan_name) ||
        !ENGINE_set_init_function(e, oezgan_init) ||
        !ENGINE_set_RAND(e, &oezgan_random_method)
        return 0;
    return 1;

In the tester we add the following lines:


unsigned char * rand_buf= new unsigned char[5];
int err = RAND_bytes(rand_buf,5);
for(int i= 0; i < 57; i++) {

The result is seen below:

oezgan@kehf-dev1:~/workspace3/oezganEngineTester/Debug$ ./oezganEngineTester 
Oezgan Engine successfully loaded
Oezgan Engine Initializatzion!
Engine name: oezgan engine by Fraunhofer FKIE init result : 786 
oezgan engine random length 5

We now successfully implemented an OpenSSL engine that gives always returns a lot of 1s as random values.


Now we want OpenSSL not only to use our own random function but also to use our sha2 family hash functions. We first implement a digest selector function, which tells OpenSSL which digests are available in our engine. This kind of implementation is adapted from the OpenSSL`s build-in engine ccghost. We will implement only one hash function namely SHA256. Following the the ossl_typ.h header to the evp.h header one can see that the Message Digest Structure is defined as follows:

struct env_md_st {
    int type;
    int pkey_type;
    int md_size;
    unsigned long flags;
    int (*init) (EVP_MD_CTX *ctx);
    int (*update) (EVP_MD_CTX *ctx, const void *data, size_t count);
    int (*final) (EVP_MD_CTX *ctx, unsigned char *md);
    int (*copy) (EVP_MD_CTX *to, const EVP_MD_CTX *from);
    int (*cleanup) (EVP_MD_CTX *ctx);
    /* FIXME: prototype these some day */
    int (*sign) (int type, const unsigned char *m, unsigned int m_length,
                 unsigned char *sigret, unsigned int *siglen, void *key);
    int (*verify) (int type, const unsigned char *m, unsigned int m_length,
                   const unsigned char *sigbuf, unsigned int siglen,
                   void *key);
    int required_pkey_type[5];  /* EVP_PKEY_xxx */
    int block_size;
    int ctx_size;               /* how big does the ctx->md_data need to be */
    /* control function */
    int (*md_ctrl) (EVP_MD_CTX *ctx, int cmd, int p1, void *p2);
} /* EVP_MD */ ;

(Commentaries are original form OpenSSL code). From this we use our own message digest hash SHA256 declaration:

static EVP_MD oezgan_engine_sha256_method=  {
        /* FIXME: prototype these some day */
        {NID_undef, NID_undef, 0, 0, 0},
        64, /*Block Size*/
        32, /* how big does the ctx->md_data need to be */
        /* control function */
} ;