Self Test Failures - Revision history

Jwalton: Add info on OpenSSL 1.1.x

2020-02-09T16:17:01Z

Add info on OpenSSL 1.1.x

Jwalton: Tweaked headings.

2016-06-14T10:30:45Z

Tweaked headings.

Jwalton: Added info on running the test

2016-06-14T08:42:39Z

Added info on running the test

Jwalton: Added example of stepping problematic code.

2016-06-14T08:39:38Z

Added example of stepping problematic code.

Jwalton: Add link to FAQ item on bug reporting. TODO: maybe wikify it.

2016-06-14T08:22:51Z

Add link to FAQ item on bug reporting. TODO: maybe wikify it.

Jwalton: Fixed typo.

2016-06-14T08:02:49Z

Fixed typo.

Jwalton: Added section on debugging engines from notes acquired from Matt.

2016-06-14T08:00:55Z

Added section on debugging engines from notes acquired from Matt.

Jwalton: Improved flow and grammar.

2016-06-14T07:56:43Z

Improved flow and grammar.

Jwalton: Add missing command line prompt. its a visual queue for readers.

2016-06-14T07:53:24Z

Add missing command line prompt. its a visual queue for readers.

Jwalton: /* Build the Test */

2016-06-14T07:52:29Z

Build the Test

@@ Line 7: / Line 7: @@
 You can sometimes isolate a bug with <tt>no-asm</tt>, and sometimes with <tt>-O0</tt> or <tt>-O1</tt>. Ideally the bug will be present with no optimizations (<tt>-O0</tt>), but sometimes it takes some compiler analysis to tickle it (<tt>-O1</tt>).
-==Build the Library==
+== OpenSSL 1.1.0 ==
 The first thing you should do after verifying the bug is to create a debug build of the OpenSSL library and attempt to reproduce it. Creating a debug build is as easy as using <tt>-d</tt> configure argument. In the output below, notice <tt>-d</tt> caused the library to reduce optimizations (<tt>-O0</tt>) and increase debugging information (<tt>-g</tt>). However, the debugging information will lack symbolic constants because <tt>-g3</tt> was not used:
@@ Line 31: / Line 108: @@
 After configuring, run <tt>make</tt> as usual.
-==Build the Test==
+===Build the Test===
 You typically run <tt>make test</tt> to validate the library after <tt>make</tt>'ing. Since you already know there's a problem, you can build the problem test with the following. The command below is an example for tracking down a failure in the HMAC test suite
@@ Line 62: / Line 139: @@
 `test' is up to date.</pre>
-== Debug the Test ==
+=== Debug the Test ===
 The executable for <tt>test_hmac</tt> is created from the source <tt>05-test_hmac.t</tt>, and it will be located at <tt>./test/buildtest_hmac</tt>:
@@ Line 128: / Line 205: @@
              jmp     .Lot</pre>
-== Debug an Engine ==
+=== Debug an Engine ===
 If you are testing an OpenSSL engine, then the procedures and paths are slightly different.

@@ Line 7: / Line 7: @@
 You can sometimes isolate a bug with <tt>no-asm</tt>, and sometimes with <tt>-O0</tt> or <tt>-O1</tt>. Ideally the bug will be present with no optimizations (<tt>-O0</tt>), but sometimes it takes some compiler analysis to tickle it (<tt>-O1</tt>).
-==Debug Build==
+==Build the Library==
 The first thing you should do after verifying the bug is to create a debug build of the OpenSSL library and attempt to reproduce it. Creating a debug build is as easy as using <tt>-d</tt> configure argument. In the output below, notice <tt>-d</tt> caused the library to reduce optimizations (<tt>-O0</tt>) and increase debugging information (<tt>-g</tt>). However, the debugging information will lack symbolic constants because <tt>-g3</tt> was not used:
@@ Line 62: / Line 62: @@
 `test' is up to date.</pre>
-== Execute the Test ==
+== Debug the Test ==
 The executable for <tt>test_hmac</tt> is created from the source <tt>05-test_hmac.t</tt>, and it will be located at <tt>./test/buildtest_hmac</tt>:

@@ Line 69: / Line 69: @@
 ./test/buildtest_hmac:  ELF 64-bit LSB executable AMD64 Version 1, dynamically linked, not stripped</pre>
-You can manually execute the test with the following.
+You can manually execute the test with the following. Note that <tt>$PWD</tt> is the root of the OpenSSL distribution, and the libraries are put on path using <tt>LD_LIBRARY_PATH</tt>.
-<pre>TODO</pre>
+<pre>$ LD_LIBRARY_PATH=".:$LD_LIBRARY_PATH" test/hmactest
-And you can run the test under a debugger with the following, As can be seen, the code ends up in a loop between lines 184 and 188.
+You can run the test under a debugger with the following. As can be seen, the execution results in a loop between lines 184 and 189.
 <pre>$ LD_LIBRARY_PATH=".:$LD_LIBRARY_PATH" gdb test/hmactest

@@ Line 69: / Line 69: @@
 ./test/buildtest_hmac:  ELF 64-bit LSB executable AMD64 Version 1, dynamically linked, not stripped</pre>
-You can manually execute the test with the following:
+You can manually execute the test with the following.
 <pre>TODO</pre>
-And you can run the test under a debugger with the following:
+And you can run the test under a debugger with the following, As can be seen, the code ends up in a loop between lines 184 and 188.
-<pre>TODO</pre>
+<pre>$ LD_LIBRARY_PATH=".:$LD_LIBRARY_PATH" gdb test/hmactest
 == Debug an Engine ==

@@ Line 1: / Line 1: @@
 The OpenSSL library includes a test suite that exercises components from both <tt>libcrypto.a</tt> and <tt>libssl.a</tt>. Ensuring the library builds and executes its tests properly is a keystone to using the library.
-This page will show you how to isolate the test and the cause under many conditions. The more information you can supply with a bug report or pull request, the easier it is for the team to identify the failure or accept proposed changes.
+This page will show you how to isolate the test and the cause under many conditions. The more information you can supply with a bug report or pull request, the easier it is for the team to identify the failure or accept proposed changes. Also see [http://www.openssl.org/docs/faq.html#BUILD17 Item 17. I think I've found a bug, what should I do?] in the OpenSSL FAQ.
 Generally speaking, there are two generations of self tests. The first generation is from OpenSSL 1.0.2 and below. The second generation is from OpenSSL 1.1.0 and above. This page focuses on the second generation, or OpenSSL 1.1.0 and above.

@@ Line 33: / Line 33: @@
 ==Build the Test==
-Typically after <tt>make</tt>'ing the library, you run <tt>make test</tt> to validate it. Since you already know there's a problem, you can only build the problem test with the following. The command below builds the HMAC test suite
+You typically run <tt>make test</tt> to validate the library after <tt>make</tt>'ing. Since you already know there's a problem, you can build the problem test with the following. The command below is an example for tracking down a failure in the HMAC test suite
 <pre>$ VERBOSE=1 make TESTS="test_hmac" test

@@ Line 11: / Line 11: @@
 The first thing you should do after verifying the bug is to create a debug build of the OpenSSL library and attempt to reproduce it. Creating a debug build is as easy as using <tt>-d</tt> configure argument. In the output below, notice <tt>-d</tt> caused the library to reduce optimizations (<tt>-O0</tt>) and increase debugging information (<tt>-g</tt>). However, the debugging information will lack symbolic constants because <tt>-g3</tt> was not used:
-<pre>./config -d
+<pre>$ ./config -d
 Operating system: i86pc-whatever-solaris2
 Configuring for solaris64-x86_64-gcc