Talk:Libcrypto API

Current Discussions

Initialization, OPENSSL_conf and engines?

Should the recommended initialization code include a call to ENGINE_load_builtin_engines? (Or to OPENSSL_config, which calls ENGINE_load_builtin_engines.) Otherwise, the RdRand engine for getting better random numbers from newer Intel chips (as one example) won't be used.

(My own thoughts on OpenSSL initialization are here.)

--Ppelleti 18:05, 3 March 2013 (UTC)

Hmmm - I've not come across this as a recommendation before. What is the original source for your recommendation?

--Matt 22:15, 3 March 2013 (UTC)

It's not from any existing documentation source, other than gleaning some information from the engine manpage (see "Automatically using builtin ENGINE implementations") and the CHANGES file. But mostly it's my own conclusion, based on reading the source code and performing experiments.

The basic question I was trying to answer was, on modern Intel processors which support AES-NI and RdRand, is OpenSSL taking advantage of these hardware features. The answer appears to be different for the two different features. For AES-NI, it appears from the source code (and was recently confirmed on the mailing list) that AES-NI is automatically used if it is available, without needing to do anything special.

However, for RdRand, it appears that the answer is different. In the source code, there is a separate RdRand engine. If the RdRand engine is not used, then the default pool implementation in md_rand.c is used, and you don't get the benefits of RdRand.

From the section I already mentioned in the "engine" manpage, it sounded like no engines are used by default, and you must enable them by calling ENGINE_load_builtin_engines() followed by ENGINE_register_all_complete(). Although the CHANGES file partially contradicts this advice, saying:

  *) Add call to ENGINE_register_all_complete() to
     ENGINE_load_builtin_engines(), so some implementations get used
     automatically instead of needing explicit application support.
     [Steve Henson]

I did some experiments on a machine with RdRand. I wrote the following little bit of code:

  ENGINE *rnd = ENGINE_get_default_RAND ();
  if (rnd)
    printf ("default rand engine: %s\n", ENGINE_get_name (rnd));
  else
    printf ("no default rand engine\n");

If I initialize OpenSSL the typical way:

  SSL_load_error_strings();                /* readable error messages */
  SSL_library_init();                      /* initialize library */
  OpenSSL_add_all_algorithms();

without calling any ENGINE functions, then my little code fragment will print "no default rand engine", indicating the implementation from md_rand.c is being used. But if I call ENGINE_load_builtin_engines() after the other initialization functions, and before my little test, it then prints out that RdRand is the default rand engine.

So, this is how I drew the conclusion that it's necessary to call ENGINE_load_builtin_engines() as part of your initialization, if you want to get RdRand support.

However, this is all made a little bit trickier by the fact that OpenSSL_add_all_algorithms() can actually mean one of two vastly different things, depending on a #define at compile time. If OPENSSL_LOAD_CONF is defined, then OpenSSL_add_all_algorithms() is really OPENSSL_add_all_algorithms_conf(), but if OPENSSL_LOAD_CONF is not defined (which is the default), then OpenSSL_add_all_algorithms() is really OPENSSL_add_all_algorithms_noconf().

OPENSSL_add_all_algorithms_conf() is a two-line function:

void OPENSSL_add_all_algorithms_conf(void)
	{
	OPENSSL_add_all_algorithms_noconf();
	OPENSSL_config(NULL);
	}

So the difference is that if OPENSSL_LOAD_CONF is defined, then OPENSSL_config() is called, when it otherwise wouldn't be. What does this have to do with RdRand? The thing is that OPENSSL_config() calls ENGINE_load_builtin_engines(). (And then ENGINE_load_builtin_engines() in turn calls ENGINE_register_all_complete(), as mentioned in the CHANGES entry.)

So, to get RdRand support, you can either #define OPENSSL_LOAD_CONF when building your program, or you can call either ENGINE_load_builtin_engines() or OPENSSL_config() in your initialization sequence. However, it appears that calling ENGINE_load_builtin_engines() more than once will leak memory, so ideally you don't want to call ENGINE_load_builtin_engines() if you also plan on calling OPENSSL_config(), or if you've defined OPENSSL_LOAD_CONF. (Of course, since it's just a small fixed-size leak at initialization, this wouldn't really be a practical problem, but still makes me feel icky.)

--Ppelleti 04:33, 4 March 2013 (UTC)

> For AES-NI, it appears from the source code

> (and was recently confirmed on the mailing list)

> that AES-NI is automatically used if it is

> available, without needing to do anything special.

> However, for RdRand, it appears that the answer is

> different.

Perhaps it is because the hardware support for AES-NI and RDRAND was provided at different times. Its somewhat odd the Change Log shows them being cut-in at the same time in March, 2012 (http://www.openssl.org/news/changelog.html).

AES-NI and PCLMULQDQ was introduced with Sandy Bridge in January, 2011. Confer, http://en.wikipedia.org/wiki/Sandy_Bridge_(microarchitecture). RdRand was introduced with 3rd generation i5's and i7's via Ivy Bridge in April, 2012. Confer, http://en.wikipedia.org/wiki/Ivy_Bridge_(computer_processor).

Jwalton 04:00, 7 March 2013 (UTC)

Note that the use of OPENSSL_config() is recommended during initialisation: this is mentioned in the manual page. Currently the routines associated with OPENSSL_config() can be used for adding OIDs and configuring ENGINEs. In future it may well do much more and calling OPENSSL_config() (or the actual conf library if finer control is needed) will automatically take advantage of that.

Here's an example of what I mean. Suppose you have a user who wants to do something weird with an ENGINE: perhaps load an unusual one that needs various ctrls to get it to work. Maybe they want to do something peculiar like use RSA with one ENGINE and DSA with another. You'd have to delve quite deep into the way ENGINE works to support that kind of thing and would it be worth it for something hardly anyone would use?

If instead you called OPENSSL_config() that user can just set up the config file to do what they want and the application writer doesn't have to worry about all the messy ENGINE calls.

--Steve 13:37, 4 March 2013 (UTC)

So, I've added a call to OPENSSL_config() during the initialisation example. This I think covers both Steve's point above, and Patrick's concern about loading the builtin engines.

--Matt 21:01, 4 March 2013 (UTC)

Return values

Note that not all of the libcrypto functions return 0 for error and 1 for success. There are exceptions which can trip up the unwary. For example if you want to check a signature with some functions you get 1 if the signature is correct, 0 if it is not correct and -1 if something bad happened like a memory allocation failure. So if you do:

 if (some_verify_function())
    /* signature successful */

and someone can induce the "something bad happened" condition you end up behaving as though a bad signature is good. This one cropped up in the library internals at one point and was fixed in a security release. Currently you should check the manual pages or the source to be sure. It would be really useful if the exceptions were all documented, double checking with the source.

--Steve 13:57, 4 March 2013 (UTC)

I've added this to the error handling section

--Matt 21:05, 4 March 2013 (UTC)

Yeah, I've noticed this, and in my own code I've chosen to always compare OpenSSL return values against 1 explicitly. In the spirit of "be bold", I've gone ahead and added this as a recommendation on the page itself. But if anyone thinks this is not a good approach, feel free to change it.

--Ppelleti 00:37, 5 March 2013 (UTC)

I'm happy with that...but now I'm wondering whether for consistency we should use this throughout the examples that we post on the wiki.

Also, the way the page now reads it looks like we are only recommending your idiom for those functions which might return something other than 0 or 1. Would it not be better to recommend this for all functions even if they do only return 0 or 1. By getting into the habit of always checking in this way it probably means you are less likely to inadvertently go wrong.

Finally, is it not the case that most of the time the if statement is checking of an error condition. Therefore shouldn't we write the code more like this:

if(1 != some_function())
    /* handle the error */

--Matt 22:31, 5 March 2013 (UTC)

I agree on both points. I'd meant "this idiom can be used to avoid having to worry about whether a particular function can return more than just 0 or 1 or not", and I always use that idiom in my own code for precisely that reason. But I don't think I made my meaning clear enough on the page. I've rephrased it a bit now, but the phrasing still feels a bit awkward, so feel free to improve it to be clearer.

And yes, I agree about flipping the sense of the check. I'd just been trying to be symmetrical with the example right above it.

But I've now flipped the sense, and I've also modified the earlier example (demonstrating "goto") to use the "1 !=" idiom.

--Ppelleti 01:25, 6 March 2013 (UTC)

Old Discussions

Best practices for printing errors

I'm curious about the recommendation to do this:

 err:
   unsigned long errCode;
   while(errCode = ERR_get_error())
   {
     char *err = ERR_error_string(errCode, NULL);
     printf("%s\n", err);
   }

Wouldn't it be much simpler to just do:

 err:
  ERR_print_errors_fp(stderr);

Or, if one really does want to iterate through each line of the error queue individually, wouldn't it still be better for us to recommend using ERR_error_string_n with an explicit buffer? ERR_error_string with a NULL argument is not thread-safe.

--Ppelleti 18:12, 3 March 2013 (UTC)

Either way does the trick, but I agree yours is simpler. I'll change it.,

--Matt 22:16, 3 March 2013 (UTC)

ERR_print_errors_fp is the best "call it and forget it" method for errors if it is appropriate to use an fp. Calling the ERR routines directly can be done but it's trickier and the example given is incomplete: I'd have to check it further to see how best to call all the routines. [per Steve Henson]

--Stevem 14:27, 4 March 2013 (UTC)

OPENSSL_config deprecated

The sample code includes a call to OPENSSL_config which was deprecated in 1.1.0. I'm still familiarizing myself with the codebase and the documentation, and I therefore do not have a replacement suggestion yet, but if anyone is willing to impart their wisdom I'd be much obliged.

 /* Load config file, and other important initialisation */
 OPENSSL_config(NULL);

Jflopezfernandez (talk) 23:28, 26 July 2019 (UTC)

The preferred approach to loading the default config file is:

   OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL);

This must be the first OpenSSL call you make (typically at the beginning of your application).

--Matt (talk) 09:38, 28 July 2019 (UTC)

Hey, thanks, Matt, that helped a lot; I was able to use that to grep right to the file where this is all defined. I'll post back here again once I'm confident I have some sample code that's both simple enough to be digestible as an introduction, but not so contrived it isn't useful.

Jflopezfernandez (talk) 03:27, 29 July 2019 (UTC)