Difference between revisions of "Random fork-safety"

From OpenSSLWiki
Jump to navigationJump to search
(Added section on fork-safety remediations.)
m (refer to array name in example code)
 
(20 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
One of the most important issues in the proper cryptographic use of random numbers is that random numbers must not be reused.  Since the UNIX fork() system call duplicates the entire process state, a random number generator which does not take this issue into account will produce the same sequence of random numbers in both the parent and the child (or in multiple children), leading to cryptographic disaster (i. e. people being able to read your communications).
 
One of the most important issues in the proper cryptographic use of random numbers is that random numbers must not be reused.  Since the UNIX fork() system call duplicates the entire process state, a random number generator which does not take this issue into account will produce the same sequence of random numbers in both the parent and the child (or in multiple children), leading to cryptographic disaster (i. e. people being able to read your communications).
 +
 +
'''The situation has changed greatly, starting with OpenSSL 1.1.1 which completely rewrote RNG.  The concerns below do not really apply any more.'''
  
 
OpenSSL's default random number generator mixes in the PID, which provides a certain degree of fork safety.  However, once the PIDs wrap, new children will start to produce the same random sequence as previous children which had the same PID.  This is unlikely to happen in most common cases, but it is not impossible, which makes the issue even more insidious.
 
OpenSSL's default random number generator mixes in the PID, which provides a certain degree of fork safety.  However, once the PIDs wrap, new children will start to produce the same random sequence as previous children which had the same PID.  This is unlikely to happen in most common cases, but it is not impossible, which makes the issue even more insidious.
Line 13: Line 15:
 
* [http://www.postgresql.org/message-id/E1UKzBn-0006c2-Cy@gemulon.postgresql.org pgsql: Reset OpenSSL randomness state in each postmaster childprocess - Tom Lane - Mar 27, 2013]
 
* [http://www.postgresql.org/message-id/E1UKzBn-0006c2-Cy@gemulon.postgresql.org pgsql: Reset OpenSSL randomness state in each postmaster childprocess - Tom Lane - Mar 27, 2013]
 
* [https://plus.google.com/+AndroidDevelopers/posts/YxWzeNQMJS2 Google+ discussion of Android SecureRandom issue, caused by OpenSSL PID wraparound - Aug 14, 2013]
 
* [https://plus.google.com/+AndroidDevelopers/posts/YxWzeNQMJS2 Google+ discussion of Android SecureRandom issue, caused by OpenSSL PID wraparound - Aug 14, 2013]
 +
* [http://bugs.python.org/issue18747 Python: Re-seed OpenSSL's PRNG after fork - Aug 15, 2013]
 +
* [http://jbp.io/2013/08/15/android-securerandom-guess/ Android SecureRandom vulnerability guess - Joseph Birr-Pixton - Aug 15, 2013]
 
* [http://www.mail-archive.com/openssl-users@openssl.org/msg71749.html openssl-users: DLL hell - Nico Williams - Aug 15, 2013]
 
* [http://www.mail-archive.com/openssl-users@openssl.org/msg71749.html openssl-users: DLL hell - Nico Williams - Aug 15, 2013]
 
** [http://www.mail-archive.com/openssl-dev@openssl.org/msg32867.html openssl-dev: not fork-safe if pids wrap (continuation of above openssl-users thread) - Aug 22, 2013]
 
** [http://www.mail-archive.com/openssl-dev@openssl.org/msg32867.html openssl-dev: not fork-safe if pids wrap (continuation of above openssl-users thread) - Aug 22, 2013]
Line 18: Line 22:
 
* [http://www.mail-archive.com/openssl-dev@openssl.org/msg33012.html openssl-dev: Mixing time into the pool - Ben Laurie - Sept 21, 2013]
 
* [http://www.mail-archive.com/openssl-dev@openssl.org/msg33012.html openssl-dev: Mixing time into the pool - Ben Laurie - Sept 21, 2013]
 
** [https://github.com/openssl/openssl/commit/3cd8547a2018ada88a4303067a2aa15eadc17f39 commit mentioned in above message]
 
** [https://github.com/openssl/openssl/commit/3cd8547a2018ada88a4303067a2aa15eadc17f39 commit mentioned in above message]
 +
* [http://bugs.python.org/issue19227 Python issue 19227: test_multiprocessing_xxx hangs under Gentoo buildbots - Oct 11, 2013]
 +
* [http://www.mail-archive.com/openssl-dev@openssl.org/msg33604.html openssl-dev: Reseed PRNG on PID change - Florian Weimer - Jan 15, 2014]
  
 
== Remediations ==
 
== Remediations ==
Line 25: Line 31:
 
* Don't use <tt>RAND_bytes</tt>
 
* Don't use <tt>RAND_bytes</tt>
 
* Call <tt>RAND_seed</tt> after a fork
 
* Call <tt>RAND_seed</tt> after a fork
 +
* Call <tt>RAND_poll</tt> after a fork
 
* Use a hardware based generator
 
* Use a hardware based generator
 
* Practice hedging cryptography
 
* Practice hedging cryptography
  
The first remediation is to avoid using <tt>RAND_bytes</tt>. Instead, you can use whatever the operating system provides; read directly from <tt>/dev/random</tt>, <tt>/dev/urandom</tt> or <tt>/dev/srandom</tt>; or use <tt>CryptGenRandom</tt> on Windows systems. Avoiding <tt>RAND_bytes</tt> is not practical in practice because the library will use it internally.
+
The first remediation is to avoid using <tt>RAND_bytes</tt>. Instead, you can read directly from <tt>/dev/random</tt>, <tt>/dev/urandom</tt> or <tt>/dev/srandom</tt>; or use <tt>CryptGenRandom</tt> on Windows systems. Avoiding <tt>RAND_bytes</tt> is not practical in practice because the library will use it internally.
 +
 
 +
The second remediation is to call <tt>RAND_seed</tt> or <tt>RAND_add</tt> after a fork. Entropy can be obtained from the operating system by reading from <tt>/dev/random</tt>, <tt>/dev/urandom</tt> or <tt>/dev/srandom</tt>; or using <tt>CryptGenRandom</tt> on Windows systems. For mobile devices with an interactive user, you could even add sensor data from the accelerometer, magnetometer and gyroscopes. This is appropriate for most programs, but might have problems in low entropy environments such as mobile devices and headless servers. Additionally, this could have problems in virtualized environments. For details, see [[Random_Numbers|Random Numbers]].
 +
 
 +
The third remediation is to use <tt>RAND_poll</tt> after a fork. This is used by OpenSSL to seed the generator on startup. The function always reads from <tt>/dev/urandom</tt>, so you will have to seed the generator yourself if you want to use <tt>/dev/random</tt> or <tt>/dev/srandom</tt>. This is appropriate for most programs, and recommended by a number of people familiar with the library. This method has the same potential problems as <tt>RAND_seed</tt> or <tt>RAND_add</tt>. For details, see [[Random_Numbers|Random Numbers]].
 +
 
 +
The fourth remediation is to use a hardware based generator. This is not always practical because hardware is not always present. Additionally, hardware is not usually auditable so some question its unabridged use in the post-Snowden era. For details, see [[Random_Numbers|Random Numbers]] and [[Manual:Engine(3)|OpenSSL engine(3)]] man page.
 +
 
 +
The fifth remediation is to practice hedging cryptography. Hedging uses entropy gathered from a peer during key exchange or key agreement to add to the program's internal entropy pool (for example, the random '''R<sub>A</sub>''' or '''R<sub>B</sub>''' in SSL/TLS). The benefit of hedging is its resilient against fork problems, low entropy environments, and virtual machine playbacks. For details, see [http://static.usenix.org/event/hotos05/final_papers/full_papers/garfinkel/garfinkel.pdf When Virtual is Harder than Real: Resource Allocation Challenges in Virtual Machine Based IT Environments] and [http://www.isoc.org/isoc/conferences/ndss/10/pdf/15.pdf When Good Randomness Goes Bad: Virtual Machine Reset Vulnerabilities and Hedging Deployed Cryptography].
  
The second remediation is to call <tt>RAND_seed</tt> or <tt>RAND_add</tt> after a fork. Entropy can be obtained from the operating system by reading from <tt>/dev/random</tt>, <tt>/dev/urandom</tt> or <tt>/dev/srandom</tt>; or using <tt>CryptGenRandom</tt> on Windows systems. This is appropriate for most programs, but might have problems in low entropy environments such as mobile devices and headless servers. Additionally, this could have problems in virtualized environments. For details, see [[Random_Numbers]].
+
Some of the items above recommend reading from operating system provided facilities, such as <tt>/dev/random</tt> and <tt>/dev/urandom</tt>. Since the random number generator should be in good order before the fork, you should be able to add unique information to diversify the generator states between parent and child. Unique information would include the time the call was made and the process identifier. In fact, Ben Laurie [https://github.com/openssl/openssl/commit/3cd8547a2018ada88a4303067a2aa15eadc17f39 pushed a patch to use time] as the uniqueness (unfortunately, it will be years before it gains widespread adoption in the field).
  
The third remediation is to use a hardware based generator. This is not always practical because hardware is not always present. Additionally, hardware is not usually auditable so some question its unabridged use in a post-Snowden era. For details, see [[Random_Numbers]] and [[Manual:Engine(3)|OpenSSL engine(3)]] man page.
+
The code below can be used to add unique information to the generator after a fork. The call to <tt>time()</tt> ensures the information is unique if the PIDs wrap. The call to <tt>getpid()</tt> ensures that two forks that occur back-to-back on hardware with low resolution timers are unique. Array elements <tt>seed[2]</tt> and <tt>seed[3]</tt> attempt to further enhance the generator's state by using whatever data is on the stack.
  
The fourth remediation is to practice hedging cryptography. Hedging uses entropy gathered from a peer during key exchange or key agreement to add to the program's internal entropy pool. The benefit of hedging is its resilient against fork problems, low entropy entropy environments, and virtual machine playbacks. For details, see [http://static.usenix.org/event/hotos05/final_papers/full_papers/garfinkel/garfinkel.pdf When Virtual is Harder than Real: Resource Allocation Challenges in Virtual Machine Based IT Environments] and [http://www.isoc.org/isoc/conferences/ndss/10/pdf/15.pdf When Good Randomness Goes Bad: Virtual Machine Reset Vulnerabilities and Hedging Deployed Cryptography].
+
<pre>long long seed[4];
 +
seed[0] = (long long)time(NULL);
 +
seed[1] = (long long)getpid();
 +
RAND_seed(seed, sizeof(seed));</pre>

Latest revision as of 18:56, 12 March 2021

One of the most important issues in the proper cryptographic use of random numbers is that random numbers must not be reused. Since the UNIX fork() system call duplicates the entire process state, a random number generator which does not take this issue into account will produce the same sequence of random numbers in both the parent and the child (or in multiple children), leading to cryptographic disaster (i. e. people being able to read your communications).

The situation has changed greatly, starting with OpenSSL 1.1.1 which completely rewrote RNG. The concerns below do not really apply any more.

OpenSSL's default random number generator mixes in the PID, which provides a certain degree of fork safety. However, once the PIDs wrap, new children will start to produce the same random sequence as previous children which had the same PID. This is unlikely to happen in most common cases, but it is not impossible, which makes the issue even more insidious.

The most comprehensive explanation of this problem is probably this blog post:

However, since this issue has been "rediscovered" and discussed multiple times, here are some additional links (some are also linked from the above article)

Remediations[edit]

OpenSSL cannot fix the fork-safety problem because its not in a position to do so. However, there are remediations available and they are listed below.

  • Don't use RAND_bytes
  • Call RAND_seed after a fork
  • Call RAND_poll after a fork
  • Use a hardware based generator
  • Practice hedging cryptography

The first remediation is to avoid using RAND_bytes. Instead, you can read directly from /dev/random, /dev/urandom or /dev/srandom; or use CryptGenRandom on Windows systems. Avoiding RAND_bytes is not practical in practice because the library will use it internally.

The second remediation is to call RAND_seed or RAND_add after a fork. Entropy can be obtained from the operating system by reading from /dev/random, /dev/urandom or /dev/srandom; or using CryptGenRandom on Windows systems. For mobile devices with an interactive user, you could even add sensor data from the accelerometer, magnetometer and gyroscopes. This is appropriate for most programs, but might have problems in low entropy environments such as mobile devices and headless servers. Additionally, this could have problems in virtualized environments. For details, see Random Numbers.

The third remediation is to use RAND_poll after a fork. This is used by OpenSSL to seed the generator on startup. The function always reads from /dev/urandom, so you will have to seed the generator yourself if you want to use /dev/random or /dev/srandom. This is appropriate for most programs, and recommended by a number of people familiar with the library. This method has the same potential problems as RAND_seed or RAND_add. For details, see Random Numbers.

The fourth remediation is to use a hardware based generator. This is not always practical because hardware is not always present. Additionally, hardware is not usually auditable so some question its unabridged use in the post-Snowden era. For details, see Random Numbers and OpenSSL engine(3) man page.

The fifth remediation is to practice hedging cryptography. Hedging uses entropy gathered from a peer during key exchange or key agreement to add to the program's internal entropy pool (for example, the random RA or RB in SSL/TLS). The benefit of hedging is its resilient against fork problems, low entropy environments, and virtual machine playbacks. For details, see When Virtual is Harder than Real: Resource Allocation Challenges in Virtual Machine Based IT Environments and When Good Randomness Goes Bad: Virtual Machine Reset Vulnerabilities and Hedging Deployed Cryptography.

Some of the items above recommend reading from operating system provided facilities, such as /dev/random and /dev/urandom. Since the random number generator should be in good order before the fork, you should be able to add unique information to diversify the generator states between parent and child. Unique information would include the time the call was made and the process identifier. In fact, Ben Laurie pushed a patch to use time as the uniqueness (unfortunately, it will be years before it gains widespread adoption in the field).

The code below can be used to add unique information to the generator after a fork. The call to time() ensures the information is unique if the PIDs wrap. The call to getpid() ensures that two forks that occur back-to-back on hardware with low resolution timers are unique. Array elements seed[2] and seed[3] attempt to further enhance the generator's state by using whatever data is on the stack.

long long seed[4];
seed[0] = (long long)time(NULL);
seed[1] = (long long)getpid();
RAND_seed(seed, sizeof(seed));