Library Initialization

From OpenSSLWiki
Revision as of 11:09, 22 November 2017 by Aerowolf (talk | contribs) (→‎Autoconf: OPENSSL_init_ssl had incorrect multiple references in the prose. Replaced the incorrect references with SSL_library_init.)
Jump to navigationJump to search

This page discusses OpenSSL library initialization when using the libssl and libcrypto components.

There are two ways to initialize the OpenSSL library, and they depend on the version of the library you are using. If you are using OpenSSL 1.0.2 or below, then you would use SSL_library_init. If you are using OpenSSL 1.1.0 or above, then you would use OPENSSL_init_ssl. A compatibility macro exists in ssl.h that maps SSL_library_init to OPENSSL_init_ssl, so you can continue to use SSL_library_init if desired. Also see SSL_library_init on the OpenSSL-dev mailing list.

If you fail to initialize the library, then you will experience unexplained errors like SSL_CTX_new returning NULL, error messages like SSL_CTX_new:library has no ciphers and alert handshake failure with no shared ciphers.

Below is a list of some initialization calls you might encounter in code or documentation. Unfortunately, all the initialization function return a useless values (for example, always 1) or are void functions. There is no way to determine if a failure occurred.

  • SSL_library_init
  • OpenSSL_add_ssl_algorithms
  • OpenSSL_add_all_algorithms
  • SSL_load_error_strings
  • ERR_load_crypto_strings


libssl Initialization

libssl should be initialized with calls to SSL_library_init and SSL_load_error_strings. If your program is multi-threaded, you should install the static locks. If you need (or don't need) configuration from openssl.cnf, then you should call OPENSSL_config or OPENSSL_noconfig.

If you are supporting both pre-1.1.0 and post-1.1.0 version of the OpenSSL library and you want to take control of SSL_library_init and OPENSSL_init_ssl, then you can perform: 

#include <openssl/opensslv.h>
...

#if OPENSSL_VERSION_NUMBER < 0x10100000L
SSL_library_init();
#else
OPENSSL_init_ssl(0, NULL);
#endif

When you call libssl, the function will also initialize libcrypto components. There are two corner cases discussed in later sections. The first corner case is static locks, and second is OPENSSL_config.

OpenSSL_add_ssl_algorithms is a #define for SSL_library_init. You only need to call one or the other. If you want to print error strings using OpenSSL's built in functions, then call SSL_load_error_strings.

The SSL_library_init function loads the algorithms use by libssl. Below is an excerpt from ssl_algs.c (with some additional formatting for clarity). 

int SSL_library_init(void)
{

#ifndef OPENSSL_NO_DES
    EVP_add_cipher(EVP_des_cbc());
    EVP_add_cipher(EVP_des_ede3_cbc());
#endif
#ifndef OPENSSL_NO_IDEA
    EVP_add_cipher(EVP_idea_cbc());
#endif
    ...

#ifndef OPENSSL_NO_COMP
    (void)SSL_COMP_get_compression_methods();
#endif

    ...

    /* initialize cipher/digest methods table */
    ssl_load_ciphers();

    return(1);
}

The call to ssl_load_ciphers simply builds a table for use in the library. The following is from ssl_ciph.c (with some additional formatting for clarity). 

void ssl_load_ciphers(void)
{
    ssl_cipher_methods[SSL_ENC_DES_IDX] = EVP_get_cipherbyname(SN_des_cbc);
    ssl_cipher_methods[SSL_ENC_3DES_IDX] = EVP_get_cipherbyname(SN_des_ede3_cbc);
    ...
    ssl_digest_methods[SSL_MD_MD5_IDX] = EVP_get_digestbyname(SN_md5);
    ssl_mac_secret_size[SSL_MD_MD5_IDX] = EVP_MD_size(ssl_digest_methods[SSL_MD_MD5_IDX]);
    ...
    ssl_digest_methods[SSL_MD_SHA384_IDX] = EVP_get_digestbyname(SN_sha384);
    ssl_mac_secret_size[SSL_MD_SHA384_IDX] = EVP_MD_size(ssl_digest_methods[SSL_MD_SHA384_IDX]);
    ...
}

Library Apps

The following examines how the OpenSSL development team uses initialization in the OpenSSL utilities.

s_client initializes itself with the following calls:

  • OpenSSL_add_ssl_algorithms
  • SSL_load_error_strings

s_server initializes itself with the following calls:

  • SSL_load_error_strings();
  • OpenSSL_add_ssl_algorithms();

s_time initializes itself with the following calls:

  • OpenSSL_add_ssl_algorithms();

state_machine initializes itself with the following calls:

  • SSL_library_init();
  • OpenSSL_add_ssl_algorithms();
  • SSL_load_error_strings();
  • ERR_load_crypto_strings();

libcrypto Initialization

libcrypto should be initialized with calls to OpenSSL_add_all_algorithms and ERR_load_crypto_strings. If your program is multi-threaded, you should install the static locks. If you need (or don't need) configuration from openssl.cnf, then you should call OPENSSL_config or OPENSSL_noconfig.

The OPENSSL_add_all_algorithms function is #define'd to either OPENSSL_add_all_algorithms_conf or OPENSSL_add_all_algorithms_noconf depending upon the value of OPENSSL_LOAD_CONF. A typical installation does not define OPENSSL_LOAD_CONF, which means OPENSSL_add_all_algorithms_noconf is used. Below is an excerpt from c_all.c (with some additional formatting for clarity). 

void OPENSSL_add_all_algorithms_noconf(void)
{
    /*
     * For the moment OPENSSL_cpuid_setup does something
     * only on IA-32, but we reserve the option for all
     * platforms...
     */
    OPENSSL_cpuid_setup();
    OpenSSL_add_all_ciphers();
    OpenSSL_add_all_digests();
    ...
}

OpenSSL_add_all_ciphers looks a lot like SSL_library_init from the libssl initialization routines (sans the call to ssl_load_ciphers). Below is an excerpt from c_allc.c (with some additional formatting for clarity). 

void OpenSSL_add_all_ciphers(void)
{

#ifndef OPENSSL_NO_DES
    EVP_add_cipher(EVP_des_cfb());
    EVP_add_cipher(EVP_des_cfb1());
    EVP_add_cipher(EVP_des_cfb8());
    EVP_add_cipher(EVP_des_ede_cfb());
    EVP_add_cipher(EVP_des_ede3_cfb());
    EVP_add_cipher(EVP_des_ede3_cfb1());
    EVP_add_cipher(EVP_des_ede3_cfb8());
    ...
#endif

#ifndef OPENSSL_NO_RC4
    EVP_add_cipher(EVP_rc4());
    EVP_add_cipher(EVP_rc4_40());
# ifndef OPENSSL_NO_MD5
    EVP_add_cipher(EVP_rc4_hmac_md5());
# endif
#endif

    ...

    /* Note: there is no call to ssl_load_ciphers() here */
}

Finally, OpenSSL_add_all_algorithms(3) offers the following advice:

Calling OpenSSL_add_all_algorithms() links in all algorithms: as a result a statically linked executable can be quite large. If this is important it is possible to just add the required ciphers and digests.

If you want the small footprint, then call EVP_add_cipher with the ciphers and algorithms you need (and nothing more).

Library Apps

The following examines how the OpenSSL development team uses initialization in the OpenSSL utilities.

enc initializes itself with the following calls:

  • OpenSSL_add_all_algorithms();

dec initializes itself with the following calls:

  • OpenSSL_add_all_algorithms();

pkcs8 initializes itself with the following calls:

  • ERR_load_crypto_strings();
  • OpenSSL_add_all_algorithms();

cms_sign initializes itself with the following calls:

  • OpenSSL_add_all_algorithms();
  • ERR_load_crypto_strings();

cms_ver initializes itself with the following calls:

  • OpenSSL_add_all_algorithms();
  • ERR_load_crypto_strings();

ENGINEs and RDRAND

A call to ENGINE_load_builtin_engines loads all built-in engines, including those for AES_NI instructions and RDRAND. After the call, OpenSSL will use the engines for AES encryption and random number generation, if available. In this case, RDRAND will be the only source of random numbers.

If you are concerned over possible RDRAND tampering, then you should explicitly call RAND_set_rand_engine(NULL) after loading all engines. If another module in the program happens to call ENGINE_load_builtin_engines again, then you will go back to using RDRAND.

You can also call ENGINE_unregister_RAND followed by ENGINE_register_all_complete to unregister RDRAND as default random number generator implementation.

To avoid accidental use of RDRAND, you can build OpenSSL with OPENSSL_NO_RDRAND defined. This is the preferred method to avoid all use of RDRAND.

Future version of the library will change the default behavior. That is, in the future, you will have to explicitly call ENGINE_load_rdrand if you want to use RDRAND. The change has been checked in, but its only available through git at the moment.

For the full discussion, see coderman's RDRAND used directly when default engines loaded in openssl-1.0.1-beta1 through openssl-1.0.1e.

Static Locks

If your program is multi-threaded, then you will need to install the static locks. The static locks are used for extensively for libssl, and used in the random number generator for libcrypto. The locks should be installed after the calling:

  • SSL_library_init
  • OpenSSL_add_ssl_algorithms
  • OpenSSL_add_all_algorithms
  • SSL_load_error_strings
  • ERR_load_crypto_strings

See Multithreaded program using OpenSSL and locks randomly crashes on Stack Overflow and threads(3) for details until the wiki is updated with an example.

OPENSSL_config

OPENSSL_config and OPENSSL_noconfig loads and unloads openssl.cnf. More correctly, a call to OPENSSL_config(NULL) loads the default configuration in openssl.cnf, OPENSSL_config(filename) loads another configuration, and OPENSSL_noconfig unlods a configuration.

OPENSSL_config may (or may not) be called depending upon how the OpenSSL library was configured, and it depends on whether OPENSSL_LOAD_CONF was defined. Because OPENSSL_config may (or may not) be called, your program may or may not need to make the call to OPENSSL_config. If, for example, your program is dynamically loading an ENGINE from OPENSSL_config, then you will need to ensure a call to OPENSSL_config.

You can check the value of OPENSSL_LOAD_CONF by cat'ing <openssl/opensslconf.h>. You can then decide to call OPENSSL_config or OPENSSL_noconfig based upon the definition (or lack threof) for OPENSSL_LOAD_CONF. 

$ cat /usr/local/ssl/include/openssl/opensslconf.h | grep -i load
$

Here are the rules you should observe. In either case, your program should not depend upon the OpenSSL library and get into a known state.

  • If you need something from openssl.cnf, then call OPENSSL_config. Don't depend on the library to do it for you.
  • If you don't need something from openssl.cnf (or its mucking up you program), then call OPENSSL_noconfig. The library may have called OPENSSL_config for you.

Engines

If your application needs to use engines, then it should either call call ENGINE_load_builtin_engines or OPENSSL_config to load the built-in engines (including dynamically configured engines from openssl.cnf).

Engines are are automatically loaded (or not loaded) based on the definition of OPENSSL_LOAD_CONF (or lack of definition). You should not depend on library behavior, so you should call OPENSSL_config or ENGINE_load_builtin_engines if you need engines.

You can also load a particular engine if you know what you want to use. eng_all.c lists the built-in engines you can load. For example, the following loads the rdrand engine provided for some Intel CPUs. 

unsigned long err = 0;
int rc = 0;

OPENSSL_cpuid_setup();
ENGINE_load_rdrand();

ENGINE* eng = ENGINE_by_id("rdrand");
if(NULL == eng) handleFailure();

rc = ENGINE_init(eng);
if(1 != rc) handleFailure();

rc = ENGINE_set_default(eng, ENGINE_METHOD_RAND);
if(1 != rc) handleFailure();

/* OK to proceed */
...

ENGINE_finish(eng);
ENGINE_free(eng);
ENGINE_cleanup();

If you want an engine to provide all incumbent functionality for the OpenSSL library, then then call ENGINE_register_complete after loading the engine. Incumbent functionality is determined by the manufacturer and includes includes RSA, DSA, DH, ECDH, MD, and RAND operations. See eng_all.c, eng_fat.c, and engine(3) for details. 

ENGINE* eng = ENGINE_by_id("XXX");
if(!(eng->flags & ENGINE_FLAGS_NO_REGISTER_ALL))
    ENGINE_register_complete(eng);

Cleanup

How to cleanup the library arises on occasion. Its often in the context of running a program under a memory checker like Valgrind.

OpenSSL does not provide a SSL_library_uninit or SSL_library_cleanup function (also see Issue #3824, FEATURE: Please provide a function to unintialize the library). To cleanup the library the library call the following functions:

  • FIPS_mode_set(0);
  • ENGINE_cleanup();
  • CONF_modules_unload(1);
  • EVP_cleanup();
  • CRYPTO_cleanup_all_ex_data();
  • ERR_remove_state();
  • ERR_free_strings();

Note: ERR_remove_state() was deprecated in OpenSSL 1.0.0 when ERR_remove_thread_state() was introduced. ERR_remove_thread_state() was deprecated in OpenSSL 1.1.0 when the thread handling functionality was entirely rewritten.

CRYPTO_cleanup_all_ex_data and ERR_remove_state should be called on each thread, and not just the main thread.

The above list is a minimum to call. You will still need to cleanup Diffie-Hellman parameters, server contexts, static locks, etc.

After cleanup, you may have some memory leaks due to dynamic allocation of private static variables like ssl_comp_methods. This is a well known issue (see Issue #2561, Memory leak with SSL built-in compressions).

Autoconf

OpenSSL uses its own configuration system, and does not use Autoconf. However, a number of popular projects use both OpenSSL and Autoconf, and it would be usful to detect either OPENSSL_init_ssl and SSL_library_init from libssl. To craft a feature test for OpenSSL that recognizes both OPENSSL_init_ssl and SSL_library_init, you can use the following. 

if test "$with_openssl" = yes ; then
  dnl Order matters!
  if test "$PORTNAME" != "win32"; then
     AC_CHECK_LIB(crypto, CRYPTO_new_ex_data, [], [AC_MSG_ERROR([library 'crypto' is required for OpenSSL])])
     FOUND_SSL_LIB="no"
     AC_CHECK_LIB(ssl, OPENSSL_init_ssl, [FOUND_SSL_LIB="yes"])
     AC_CHECK_LIB(ssl, SSL_library_init, [FOUND_SSL_LIB="yes"])
     AS_IF([test "x$FOUND_SSL_LIB" = xno], [AC_MSG_ERROR([library 'ssl' is required for OpenSSL])])
  else
     AC_SEARCH_LIBS(CRYPTO_new_ex_data, eay32 crypto, [], [AC_MSG_ERROR([library 'eay32' or 'crypto' is required for OpenSSL])])
     FOUND_SSL_LIB="no"
     AC_SEARCH_LIBS(OPENSSL_init_ssl, ssleay32 ssl, [FOUND_SSL_LIB="yes"])
     AC_SEARCH_LIBS(SSL_library_init, ssleay32 ssl, [FOUND_SSL_LIB="yes"])
     AS_IF([test "x$FOUND_SSL_LIB" = xno], [AC_MSG_ERROR([library 'ssleay32' or 'ssl' is required for OpenSSL])])
  fi
fi

Many thanks to the Postgres folks for donating part of their configure.in. Also see How to tell Autoconf “require symbol A or B” from LIB? on Stack Overflow.

Errata

Retrieved from "https://wiki.openssl.org/index.php?title=Library_Initialization&oldid=2615"