How To Write Unit Tests For OpenSSL

This is an outline of the basic process and principles to follow when writing unit tests. This document will evolve quite a bit as the community gains experience.

Mechanics

Forking the Repo

For now, people contributing new unit tests for uncovered code (as opposed to submitting tests with new code changes) should fork Mike Bland's GitHub repository and issue GitHub pull requests. Remember to do all development on a topic branch (not master). Tests committed to this repo will occasionally get merged into the master OpenSSL repo.

Rationale per Matt Caswell: We should think about process here. If you are going off to recruit an army of people writing tests then I wonder if it is worth setting up a separate github repository whilst you are building up the tests. We can then merge in from that on a periodic basis. I wouldn't want availability of openssl team committers to be a bottle neck.

Use the Test Template Generator

TODO(mbland): Get template generator checked-in. Maybe have a template generator for each library, e.g. ssl/new-test.sh that has additional setup boilerplate specific to the ssl library.

Use the test/new-test.sh script to generate a skeleton test file.

Until we solve the private-symbol problem on Windows, we will need to wrap our unit test code in the following #ifdef block:

#if !defined(OPENSSL_SYS_WINDOWS)

/* All the test code, including main() */

int main(int argc, char *argv[])                                                
    {
        return EXIT_SUCCESS;                                                    
    }

#endif /* OPENSSL_NO_WINDOWS  */

Add Makefile Targets

The following instructions use the Makefile targets for ssl/heartbeat_test.c as an example.

In the Makefile for the library containing the test, add the test source file to the TEST variable:

# ssl/Makefile
TEST=ssltest.c heartbeat_test.c

In test/Makefile:

• add a variable for the test target near the top of the file, right after the existing test variables
• use the variable to add an executable target to the EXE variable
• use the variable to add an object file target to the OBJ variable
• use the variable to add a source file target to the SRC variable
• add the test target to the alltests target
• add the target to execute the test
• add the target to build the test executable

# test/Makefile
HEARTBEATTEST=  heartbeat_test
EXE=  ... $(HEARTBEATTEST)$(EXE_EXT)
OBJ= ... $(HEARTBEATTEST).o
SRC= ... $(HEARTBEATTEST).c
alltests: \
        ... test_heartbeat

test_heartbeat: $(HEARTBEATTEST)$(EXE_EXT)
  ../util/shlib_wrap.sh ./$(HEARTBEATTEST)

$(HEARTBEATTEST)$(EXE_EXT): $(HEARTBEATTEST).o $(DLIBCRYPTO)
  @target=$(HEARTBEATTEST); $(BUILD_CMD)

Finally, run make depend to automatically generate the header file dependencies.

Style

Follow Pseudo-xUnit Style

The Pseudo-xUnit Pattern is that established by ssl/heartbeat_test.c. This pattern organizes code in a fashion reminiscent of the xUnit family of unit testing frameworks, without actually using a testing framework. This should lower the barrier to entry for people wanting to write unit tests, but enable a relatively easy migration to an xUnit-based framework if we decide to do so one day.