Library Initialization

2016-05-13T14:18:13Z

Wdtj: /* Cleanup */

This page discusses OpenSSL library initialization when using the <tt>libssl</tt> and <tt>libcrypto</tt> components.

There are two ways to initialize the OpenSSL library, and they depend on the version of the library you are using. If you are using OpenSSL 1.0.2 or below, then you would use <tt>SSL_library_init</tt>. If you are using OpenSSL 1.1.0 or above, then you would use <tt>OPENSSL_init_ssl</tt>. A compatibility macro exists in <tt>ssl.h</tt> that maps <tt>SSL_library_init</tt> to <tt>OPENSSL_init_ssl</tt>, so you can continue to use <tt>SSL_library_init</tt> if desired. Also see ''[http://mta.openssl.org/pipermail/openssl-dev/2016-February/005491.html SSL_library_init]'' on the OpenSSL-dev mailing list.

If you fail to initialize the library, then you will experience unexplained errors like <tt>SSL_CTX_new</tt> returning <tt>NULL</tt>, error messages like <tt>SSL_CTX_new:library has no ciphers</tt> and <tt>alert handshake failure</tt> with no shared ciphers.

Below is a list of some initialization calls you might encounter in code or documentation. Unfortunately, all the initialization function return a useless values (for example, always 1) or are void functions. There is no way to determine if a failure occurred.

* SSL_library_init
* OpenSSL_add_ssl_algorithms
* OpenSSL_add_all_algorithms
* SSL_load_error_strings
* ERR_load_crypto_strings

== libssl Initialization ==

<tt>libssl</tt> should be initialized with calls to <tt>SSL_library_init</tt> and <tt>SSL_load_error_strings</tt>. If your program is multi-threaded, you should install the static locks. If you need (or don't need) configuration from <tt>openssl.cnf</tt>, then you should call <tt>OPENSSL_config</tt> or <tt>OPENSSL_noconfig</tt>.

If you are supporting both pre-1.1.0 and post-1.1.0 version of the OpenSSL library and you want to take control of <tt>SSL_library_init</tt> and <tt>OPENSSL_init_ssl</tt>, then you can perform:

<pre>#include <openssl/opensslv.h>
...

#if OPENSSL_VERSION_NUMBER < 0x10100000L
SSL_library_init();
#else
OPENSSL_init_ssl(0, NULL);
#endif</pre>

When you call <tt>libssl</tt>, the function will also initialize <tt>libcrypto</tt> components. There are two corner cases discussed in later sections. The first corner case is static locks, and second is <tt>OPENSSL_config</tt>.

<tt>OpenSSL_add_ssl_algorithms</tt> is a <tt>#define</tt> for <tt>SSL_library_init</tt>. You only need to call one or the other. If you want to print error strings using OpenSSL's built in functions, then call <tt>SSL_load_error_strings</tt>.

The <tt>SSL_library_init</tt> function loads the algorithms use by <tt>libssl</tt>. Below is an excerpt from <tt>ssl_algs.c</tt> (with some additional formatting for clarity).

<pre>int SSL_library_init(void)
{

#ifndef OPENSSL_NO_DES
EVP_add_cipher(EVP_des_cbc());
EVP_add_cipher(EVP_des_ede3_cbc());
#endif
#ifndef OPENSSL_NO_IDEA
EVP_add_cipher(EVP_idea_cbc());
#endif
...

#ifndef OPENSSL_NO_COMP
(void)SSL_COMP_get_compression_methods();
#endif

...

/* initialize cipher/digest methods table */
ssl_load_ciphers();

return(1);
}</pre>

The call to <tt>ssl_load_ciphers</tt> simply builds a table for use in the library. The following is from <tt>ssl_ciph.c</tt> (with some additional formatting for clarity).

<pre>void ssl_load_ciphers(void)
{
ssl_cipher_methods[SSL_ENC_DES_IDX] = EVP_get_cipherbyname(SN_des_cbc);
ssl_cipher_methods[SSL_ENC_3DES_IDX] = EVP_get_cipherbyname(SN_des_ede3_cbc);
...
ssl_digest_methods[SSL_MD_MD5_IDX] = EVP_get_digestbyname(SN_md5);
ssl_mac_secret_size[SSL_MD_MD5_IDX] = EVP_MD_size(ssl_digest_methods[SSL_MD_MD5_IDX]);
...
ssl_digest_methods[SSL_MD_SHA384_IDX] = EVP_get_digestbyname(SN_sha384);
ssl_mac_secret_size[SSL_MD_SHA384_IDX] = EVP_MD_size(ssl_digest_methods[SSL_MD_SHA384_IDX]);
...
}</pre>

=== Library Apps ===

The following examines how the OpenSSL development team uses initialization in the OpenSSL utilities.

<tt>s_client</tt> initializes itself with the following calls:
* OpenSSL_add_ssl_algorithms
* SSL_load_error_strings

<tt>s_server</tt> initializes itself with the following calls:
* SSL_load_error_strings();
* OpenSSL_add_ssl_algorithms();

<tt>s_time</tt> initializes itself with the following calls:
* OpenSSL_add_ssl_algorithms();

<tt>state_machine</tt> initializes itself with the following calls:
* SSL_library_init();
* OpenSSL_add_ssl_algorithms();
* SSL_load_error_strings();
* ERR_load_crypto_strings();

== libcrypto Initialization ==

<tt>libcrypto</tt> should be initialized with calls to <tt>OpenSSL_add_all_algorithms</tt> and <tt>ERR_load_crypto_strings</tt>. If your program is multi-threaded, you should install the static locks. If you need (or don't need) configuration from <tt>openssl.cnf</tt>, then you should call <tt>OPENSSL_config</tt> or <tt>OPENSSL_noconfig</tt>.

The <tt>OPENSSL_add_all_algorithms</tt> function is <tt>#define</tt>'d to either <tt>OPENSSL_add_all_algorithms_conf</tt> or <tt>OPENSSL_add_all_algorithms_noconf</tt> depending upon the value of <tt>OPENSSL_LOAD_CONF</tt>. A typical installation does ''not'' define <tt>OPENSSL_LOAD_CONF</tt>, which means <tt>OPENSSL_add_all_algorithms_noconf</tt> is used. Below is an excerpt from <tt>c_all.c</tt> (with some additional formatting for clarity).

<pre>void OPENSSL_add_all_algorithms_noconf(void)
{
/*
* For the moment OPENSSL_cpuid_setup does something
* only on IA-32, but we reserve the option for all
* platforms...
*/
OPENSSL_cpuid_setup();
OpenSSL_add_all_ciphers();
OpenSSL_add_all_digests();
...
}</pre>

<tt>OpenSSL_add_all_ciphers</tt> looks a lot like <tt>SSL_library_init</tt> from the <tt>libssl</tt> initialization routines (sans the call to <tt>ssl_load_ciphers</tt>). Below is an excerpt from <tt>c_allc.c</tt> (with some additional formatting for clarity).

<pre>void OpenSSL_add_all_ciphers(void)
{

#ifndef OPENSSL_NO_DES
EVP_add_cipher(EVP_des_cfb());
EVP_add_cipher(EVP_des_cfb1());
EVP_add_cipher(EVP_des_cfb8());
EVP_add_cipher(EVP_des_ede_cfb());
EVP_add_cipher(EVP_des_ede3_cfb());
EVP_add_cipher(EVP_des_ede3_cfb1());
EVP_add_cipher(EVP_des_ede3_cfb8());
...
#endif

#ifndef OPENSSL_NO_RC4
EVP_add_cipher(EVP_rc4());
EVP_add_cipher(EVP_rc4_40());
# ifndef OPENSSL_NO_MD5
EVP_add_cipher(EVP_rc4_hmac_md5());
# endif
#endif

...

/* Note: there is no call to ssl_load_ciphers() here */
}</pre>

Finally, [http://www.openssl.org/docs/crypto/OpenSSL_add_all_algorithms.html <tt>OpenSSL_add_all_algorithms(3)</tt>] offers the following advice:

<blockquote>Calling OpenSSL_add_all_algorithms() links in all algorithms: as a result a statically linked executable can be quite large. If this is important it is possible to just add the required ciphers and digests.</blockquote>

If you want the small footprint, then call <tt>EVP_add_cipher</tt> with the ciphers and algorithms you need (and nothing more).

=== Library Apps ===

The following examines how the OpenSSL development team uses initialization in the OpenSSL utilities.

<tt>enc</tt> initializes itself with the following calls:
* OpenSSL_add_all_algorithms();

<tt>dec</tt> initializes itself with the following calls:
* OpenSSL_add_all_algorithms();

<tt>pkcs8</tt> initializes itself with the following calls:
* ERR_load_crypto_strings();
* OpenSSL_add_all_algorithms();

<tt>cms_sign</tt> initializes itself with the following calls:
* OpenSSL_add_all_algorithms();
* ERR_load_crypto_strings();

<tt>cms_ver</tt> initializes itself with the following calls:
* OpenSSL_add_all_algorithms();
* ERR_load_crypto_strings();

== ENGINEs and RDRAND ==

A call to <tt>ENGINE_load_builtin_engines</tt> loads all built-in engines, including those for <tt>AES_NI</tt> instructions and <tt>RDRAND</tt>. After the call, OpenSSL will use the engines for AES encryption and random number generation, if available. In this case, <tt>RDRAND</tt> will be the only source of random numbers.

If you are concerned over possible <tt>RDRAND</tt> tampering, then you should explicitly call <tt>RAND_set_rand_engine(NULL)</tt> after loading all engines. If another module in the program happens to call <tt>ENGINE_load_builtin_engines</tt> again, then you will go back to using <tt>RDRAND</tt>.

You can also call <tt>ENGINE_unregister_RAND</tt> followed by <tt>ENGINE_register_all_complete</tt> to unregister <tt>RDRAND</tt> as default random number generator implementation.

To avoid accidental use of <tt>RDRAND</tt>, you can build OpenSSL with <tt>OPENSSL_NO_RDRAND</tt> defined. This is the preferred method to avoid all use of <tt>RDRAND</tt>.

Future version of the library will change the default behavior. That is, in the future, you will have to explicitly call <tt>ENGINE_load_rdrand</tt> if you want to use <tt>RDRAND</tt>. The change has been checked in, but its only available through <tt>git</tt> at the moment.

For the full discussion, see coderman's ''[http://seclists.org/fulldisclosure/2013/Dec/99 RDRAND used directly when default engines loaded in openssl-1.0.1-beta1 through openssl-1.0.1e]''.

== Static Locks ==

If your program is multi-threaded, then you will need to install the static locks. The static locks are used for extensively for <tt>libssl</tt>, and used in the random number generator for <tt>libcrypto</tt>.

See [http://www.openssl.org/docs/crypto/threads.html threads(3)] for details until the wiki is updated with an example.

== OPENSSL_config ==

<tt>OPENSSL_config</tt> and <tt>OPENSSL_noconfig</tt> loads and unloads <tt>openssl.cnf</tt>. More correctly, a call to <tt>OPENSSL_config(NULL)</tt> loads the default configuration in <tt>openssl.cnf</tt>, <tt>OPENSSL_config(filename)</tt> loads another configuration, and <tt>OPENSSL_noconfig</tt> unlods a configuration.

<tt>OPENSSL_config</tt> may (or may not) be called depending upon how the OpenSSL library was configured, and it depends on whether <tt>OPENSSL_LOAD_CONF</tt> was defined. Because <tt>OPENSSL_config</tt> may (or may not) be called, your program may or may not need to make the call to <tt>OPENSSL_config</tt>. If, for example, your program is dynamically loading an ENGINE from <tt>OPENSSL_config</tt>, then you will need to ensure a call to <tt>OPENSSL_config</tt>.

You can check the value of <tt>OPENSSL_LOAD_CONF</tt> by <tt>cat</tt>'ing <tt><nowiki><openssl/opensslconf.h></nowiki></tt>. You can then decide to call <tt>OPENSSL_config</tt> or <tt>OPENSSL_noconfig</tt> based upon the definition (or lack threof) for <tt>OPENSSL_LOAD_CONF</tt>.

<pre>$ cat /usr/local/ssl/include/openssl/opensslconf.h | grep -i load
$</pre>

Here are the rules you should observe. In either case, your program should not depend upon the OpenSSL library and get into a known state.

* If you need something from <tt>openssl.cnf</tt>, then call <tt>OPENSSL_config</tt>. Don't depend on the library to do it for you.
* If you don't need something from <tt>openssl.cnf</tt> (or its mucking up you program), then call <tt>OPENSSL_noconfig</tt>. The library may have called <tt>OPENSSL_config</tt> for you.

== Engines ==

If your application needs to use engines, then it should either call call <tt>ENGINE_load_builtin_engines</tt> or <tt>OPENSSL_config</tt> to load the built-in engines (including dynamically configured engines from <tt>openssl.cnf</tt>).

Engines are are automatically loaded (or not loaded) based on the definition of <tt>OPENSSL_LOAD_CONF</tt> (or lack of definition). You should not depend on library behavior, so you should call <tt>OPENSSL_config</tt> or <tt>ENGINE_load_builtin_engines</tt> if you need engines.

You can also load a particular engine if you know what you want to use. <tt>eng_all.c</tt> lists the built-in engines you can load. For example, the following loads the <tt>rdrand</tt> engine provided for some Intel CPUs.

<pre>unsigned long err = 0;
int rc = 0;

OPENSSL_cpuid_setup();
ENGINE_load_rdrand();

ENGINE* eng = ENGINE_by_id("rdrand");
if(NULL == eng) handleFailure();

rc = ENGINE_init(eng);
if(1 != rc) handleFailure();

rc = ENGINE_set_default(eng, ENGINE_METHOD_RAND);
if(1 != rc) handleFailure();

/* OK to proceed */
...

ENGINE_finish(eng);
ENGINE_free(eng);
ENGINE_cleanup();</pre>

If you want an engine to provide all incumbent functionality for the OpenSSL library, then then call <tt>ENGINE_register_complete</tt> after loading the engine. Incumbent functionality is determined by the manufacturer and includes includes RSA, DSA, DH, ECDH, MD, and RAND operations. See <tt>eng_all.c</tt>, <tt>eng_fat.c</tt>, and [http://www.openssl.org/docs/crypto/engine.html engine(3)] for details.

<pre>ENGINE* eng = ENGINE_by_id("XXX");
if(!(eng->flags & ENGINE_FLAGS_NO_REGISTER_ALL))
ENGINE_register_complete(eng);</pre>

== Cleanup ==

How to cleanup the library arises on occasion. Its often in the context of running a program under a memory checker like Valgrind.

OpenSSL does not provide a <tt>SSL_library_uninit</tt> or <tt>SSL_library_cleanup</tt> function (also see [http://rt.openssl.org/Ticket/Display.html?id=3824&user=guest&pass=guest Issue #3824, FEATURE: Please provide a function to unintialize the library]). To cleanup the library the library call the following functions:

* FIPS_mode_set(0);
* ENGINE_cleanup();
* CONF_modules_unload(1);
* EVP_cleanup();
* CRYPTO_cleanup_all_ex_data();
* ERR_remove_state();
* ERR_free_strings();

'''Note:''' ERR_remove_state() was deprecated in OpenSSL 1.0.0 when ERR_remove_thread_state() was introduced. ERR_remove_thread_state() was deprecated in OpenSSL 1.1.0 when the thread handling functionality was entirely rewritten.

<tt>CRYPTO_cleanup_all_ex_data</tt> and <tt>ERR_remove_state</tt> should be called on each thread, and not just the main thread.

The above list is a minimum to call. You will still need to cleanup Diffie-Hellman parameters, server contexts, static locks, etc.

After cleanup, you may have some memory leaks due to dynamic allocation of private static variables like <tt>ssl_comp_methods</tt>. This is a well known issue (see [http://rt.openssl.org/Ticket/Display.html?id=2561&user=guest&pass=guest Issue #2561, Memory leak with SSL built-in compressions]).

== Errata ==

OpenSSLWiki - User contributions [en]

Library Initialization