SSL/TLS Client

SSL/TLS Client is sample code for a basic web client that fetches a page. The code shown below omits error checking for brevity, but the sample available for download performs the error checking.

The sample code will set up BIO to fet a page from www.random.org. The code uses TLS (not SSL) and adds the Server Name Indication (SNI) extension from RFC 3546, Transport Layer Security (TLS) Extensions.

If you need features beyond the example below, then you should examine s_client.c in the apps/ directory of the OpenSSL distribution. OpenSSL's s_client implements nearly every client side feature available from the library.

The code below does not perform hostname verification. OpenSSL prior to 1.1.0 does not perform the check, and you must perform the check yourself. The OpenSSL Change Log for OpenSSL 1.1.0 states you can use -verify_name option, and apps.c offers -verify_hostname. But s_client does not respond to either switch, so its unclear how hostname checking will be implemented or invoked for a client. Note (N.B.): hostname verification is marked as experimental, so switches, options, and implementations could change.

Implementation

The code below demonstrates a basic client that uses BIOs and TLS to connect to www.random.org, and fetches 32 bytes of random data through an HTTP request. The sample code is available for download below.

#define HOST_NAME "www.random.org"
#define HOST_PORT "443"
#define HOST_RESOURCE "/cgi-bin/randbyte?nbytes=32&format=h"

long res = 1;

SSL_CTX* ctx = NULL;
BIO *web = NULL, *out = NULL;
SSL *ssl = NULL;

init_openssl_library();

const SSL_METHOD* method = SSLv23_method();
if(!(NULL != method)) handleFailure();

ctx = SSL_CTX_new(method);
if(!(ctx != NULL)) handleFailure();

/* Cannot fail ??? */
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);

/* Cannot fail ??? */
SSL_CTX_set_verify_depth(ctx, 4);

/* Cannot fail ??? */
const long flags = SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_COMPRESSION;
SSL_CTX_set_options(ctx, flags);

res = SSL_CTX_load_verify_locations(ctx, "random-org-chain.pem", NULL);
if(!(1 == res)) handleFailure();

web = BIO_new_ssl_connect(ctx);
if(!(web != NULL)) handleFailure();

res = BIO_set_conn_hostname(web, HOST_NAME ":" HOST_PORT);
if(!(1 == res)) handleFailure();

BIO_get_ssl(web, &ssl);
if(!(ssl != NULL)) handleFailure();

const char* const PREFERRED_CIPHERS = "HIGH:!aNULL:!PSK:!SRP!MD5:!RC4";
res = SSL_set_cipher_list(ssl, PREFERRED_CIPHERS);
if(!(1 == res)) handleFailure();

res = SSL_set_tlsext_host_name(ssl, HOST_NAME);
if(!(1 == res)) handleFailure();

out = BIO_new_fp(stdout, BIO_NOCLOSE);
if(!(NULL != out)) handleFailure();

res = BIO_do_connect(web);
if(!(1 == res)) handleFailure();

res = BIO_do_handshake(web);
if(!(1 == res)) handleFailure();

/* Step 1: verify a server certificate was presented during the negotiation */
X509* cert = SSL_get_peer_certificate(ssl);
if(cert) { X509_free(cert); } /* Free immediately */
if(NULL == cert) handleFailure();

/* Step 2: verify the result of chain verification */
res = SSL_get_verify_result(ssl);
if(!(X509_V_OK == res)) handleFailure();

/* Step 3: hostname verification */
/* An exercise left to the reader */

BIO_puts(web, "GET " HOST_RESOURCE " HTTP/1.1\r\n"
              "Host: " HOST_NAME "\r\n"
              "Connection: close\r\n\r\n");
BIO_puts(out, "\n");

int len = 0;
do
{
  char buff[1536] = {};
  len = BIO_read(web, buff, sizeof(buff));
            
  if(len > 0)
    BIO_write(out, buff, len);

} while (len > 0 || BIO_should_retry(web));

if(out)
  BIO_free(out);

if(web != NULL)
  BIO_free_all(web);

if(NULL != ctx)
  SSL_CTX_free(ctx);

Initialization

The sample program initializes the OpenSSL library with init_openssl_library. init_openssl_library calls three OpenSSL functions.

#if (SSLEAY_VERSION_NUMBER >= 0x0907000L)
# include <openssl/conf.h>
#endif
...

void init_openssl_library(void)
{
  (void)SSL_library_init();

  SSL_load_error_strings();

  /* ERR_load_crypto_strings(); */
  
  OPENSSL_config(NULL);
    
  /* Include <openssl/opensslconf.h> to get this define */
#if defined (OPENSSL_THREADS)
  fprintf(stdout, "Warning: thread locking is not implemented\n");
#endif
}

SSL_library_init performs initialization of libcrypto and libssl, and loads required algorithms. The documents state SSL_library_init always returns 1, so its a useless return value.

SSL_load_error_strings loads error strings from both libcrypto and libssl. There's no need to call ERR_load_crypto_strings.

OpenSSL_add_ssl_algorithms is a #define for SSL_library_init, so the call is omitted.

OPENSSL_config may (or may not) be needed. Internally, OPENSSL_config is called based on a configuration options via OPENSSL_LOAD_CONF. If you are dynamically loading an engine specified in openssl.cnf, then you might need it so you should call it. That is, don't depend upon the OpenSSL library to call it for you.

If you are building a multi-threaded client, you should set the locking callbacks. See threads(3) for details.

A detailed treatment of initialization can be found at Library Initialization.

Context Setup

The sample program uses SSLv23_method and to create a context.

SSLv23_method specifies the protocols used and behavior of the handshake. The method essentially means SSLv2 or above, and includes the TLS protocols. The protocols are further tuned through SSL/TLS options. By using SSLv23_method (and removing the SSL protocols with SSL_OP_NO_SSLv2 and SSL_OP_NO_SSLv3), then you will use TLS v1.0 and above, including TLS v1.2. You will also use a TLS handshake in the TLS Record.

SSL_CTX_new uses the SSLv23_method method to create a new SSL/TLS context object.

If you use, for example TLSv1_method, then you will only use TLS v1.0, and if you use TLSv11_method then you will only use TLS v1.1.

Options (1)

After creating a context with SSLv23_method and SSL_CTX_new, the context object is tuned with the following functions:

SSL_CTX_set_verify
SSL_CTX_set_verify_depth
SSL_CTX_set_options
SSL_CTX_load_verify_locations

SSL_CTX_set_verify sets the SSL_VERIFY_PEER flag and the verify callback so certificate chain Issuer and Subject information can be printed. If you don't want to perform custom processing (such as printing or checking), then don't set the callback. OpenSSL's default checking should be sufficient, so pass NULL to SSL_CTX_set_verify.

There is also a SSL_VERIFY_FAIL_IF_NO_PEER_CERT flag, but it is used for servers and has no effect on clients. If you accidentally use SSL_VERIFY_FAIL_IF_NO_PEER_CERT, then you chain will always verify when call SSL_get_verify_result because the flag is ignored for clients (essentially, 0 is passed for the flag which performs no verification).

SSL_CTX_set_verify_depth sets the chain depth to 4. Chain depth is fairly useless in practice.

SSL_CTX_set_options set the SSL_OP_ALL, SSL_OP_NO_SSLv2, SSL_OP_NO_SSLv3, SSL_OP_NO_COMPRESSION options. In essence, it takes all the bug fixes and work arounds for the various servers, removes the SSL protocols (leaving only TLS protocols), and removes compression. The remaining TLS protocols are TLS 1.0, TLS 1.1, and TLS 1.2.

SSL_CTX_load_verify_locations loads the certificate chain for the random.org site. The site's CA is Comodo, and the chain includes AddTrust External CA Root, COMODO Certification Authority, and COMODO Extended Validation Secure Server CA.

The PEM format means the file is a concatenation of Base64 encoded certificates with the -----BEGIN CERTIFICATE----- prologue (and associated epilogue). If the server sends all certificates required to verify the chain (which it should), then only the AddTrust External CA Root certificate is needed.

The options set on the CTX* can be overridden on a per-connection basis by modifying the SSL* using SSL_set_verify, SSL_set_verify_depth and SSL_set_options (and friends).

SSL BIO

The sample program uses BIOs for input and output. One BIO is used to connect to random.org, and a second BIO is used to print output to stdout.

BIO_new_ssl_connect creates a new BIO chain consisting of an SSL BIO (using ctx) followed by a connect BIO.

BIO_set_conn_hostname is used to set the hostname and port that will be used by the connection.

Options (2)

BIO_get_ssl is used to fetch the SSL connection object created by BIO_new_ssl_connect. The connection object inherits from the context object, and can override the settings on the context. The connection object is tuned with the following functions:

SSL_set_cipher_list
SSL_set_tlsext_host_name

SSL_set_cipher_list sets the cipher list. The list prefers elliptic curves, ephemeral [Diffie-Hellman], AES and SHA. It also removes NULL authentication methods and ciphers; and removes medium-security, low-security and export-grade security ciphers, such as 40-bit RC2. If desired, you could set the options on the context with SSL_CTX_set_cipher_list.

SSL_set_tlsext_host_name uses the TLS SNI extension to set the hostname. If you are connecting to a Server Name Indication-aware server (such as Apache with name-based virtual hosts or IIS 8.0), then you will receive the proper certificate during the handshake.

Cipher Suites

Wireshark and ClientHello

According to openssl ciphers ALL, there are just over 110 cipher suites available. Each cipher suite takes 2 bytes in the ClientHello, so advertising every cipher suite available at the client is going to cause a larger ClientHello. When using SSL_set_cipher_list with the string "HIGH:!aNULL:!PSK:!SRP:!MD5:!RC4", you'll cut the number of cipher suites down to about 50. If you know the server does not support DSA, then you can add "!DSS" and reduce the list further by about 7. Advertising 40 or so ciphers saves about 150 bytes in the ClientHello.

Keeping the ClientHello small is important for older F5 and IronPort devices. Apparently, the devices used fixed sized buffers and choke on large ClientHello's. In fact, a "large hello" was the cause of the TLS padding bug on IronPort devices. See TLS padding breaks ironport on the TLS mailing list for details.

Connection

Wireshark and TLS versions

After setting the connection object options, the sample connects to the site and negotiates a secure channel.

BIO_do_connect
BIO_do_handshake

BIO_do_connect performs the name lookup for the host and standard TCP/IP three way handshake.

BIO_do_handshake performs the SSL/TLS handshake. If you set a callback with SSL_CTX_set_verify or SSL_set_verify, then you callback will be invoked for each certificate in the chain used during the execution of the protocol.

The Wireshark packet capture to the right shows the TLS handshake with the SNI extension encountered during the execution of BIO_do_handshake. OpenSSL 1.0.1e advertises TLSv1.2 as the highest protocol level in its ClientHello.

Callback

SSL_CTX_set_verify specified SSL_VERIFY_PEER and verify_callback, and they ensure the library will invoke the callback for each certificate encountered in the chain. The callback in this example simply prints the current certificate's Issuer and Subject names.

int verify_callback(int preverify, X509_STORE_CTX* x509_ctx)
{
    int depth = X509_STORE_CTX_get_error_depth(x509_ctx);
    int err = X509_STORE_CTX_get_error(x509_ctx);
    
    X509* cert = X509_STORE_CTX_get_current_cert(x509_ctx);
    X509_NAME* iname = cert ? X509_get_issuer_name(cert) : NULL;
    X509_NAME* sname = cert ? X509_get_subject_name(cert) : NULL;
    
    print_cn_name("Issuer (cn)", iname);
    print_cn_name("Subject (cn)", sname);
    
    if(depth == 0) {
        /* If depth is 0, its the server's certificate. Print the SANs too */
        print_san_name("Subject (san)", cert);
    }

    return preverify;
}

The OpenSSL library will pass in the value of its preliminary checking of the certificate through preverify. If you always return 1 regardless of the value of preverify or the actual result of your processing, then SSL_get_verify_result will always return X509_V_OK. That's probably a bad idea for production software.

If you don't need to perform special processing on the chain, then you should forgo the verify_callback altogether by supplying NULL to SSL_CTX_set_verify:

SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);

Verification

Verification consists of three steps and can be tricky. Painting with a broad brush, minimal checking includes: (1) confirm the server has a certificate, (2) confirm the certificate chain verifies back to a trusted root, and (3) confirm the name of the host matches a hostname listed in the server's certificate.

You usually don't perform revocation in real time because it essentially creates a denial of service on your application. That is, your app will hang while downloading a multi-megabyte CRL or contacts a missing OCSP responder. For a detailed treatment of problems with PKI and Revocation, see Peter Gutmann's Engineering Security (Chapters 1 and 8).

For detailed checks, there are five documents of interest:

The detailed checks will reveal, for example, a Certificate Authority's certificate must have a Basic Constraint of CA=TRUE and marked as Critical; that a hostname or IP in the Common Name (CN) field must also be present in the Subject Alternate Name (SAN) field; and an Extended Validation certificate cannot have RFC 1918 private address or wildcard address (i.e., 192.168.1.1 or *.example.com). Some CAs treat the rules like guidelines or suggestions, so you might DoS your application by applying the rules. Again, see Peter Gutmann's Engineering Security for a detailed treatment.

In the end, its probably better to ignore PKI and just use Public Key Pinning (or Certificate Pinning) when a pre-exisiting relationship exists; or use a Perspectives-like system or a Trust-On-First-Use (TOFU) system when there's no a priori relationship (similar to SSH's StrictHostkeyChecking option). See Peter Gutmann's Engineering Security for details of a security diversification strategy (Chapter 4, starting on page 292).

Server Certificate

You must confirm the server provided a certificate. This is because a server might be misconfigured, or the client and server used Anonymous Diffie-Hellman. You do so as follows:

X509* cert = SSL_get_peer_certificate(ssl);
if(cert) { X509_free(cert); }
if(NULL == cert) handleFailure();

If the server has a certificate, then SSL_get_peer_certificate will return a non-NULL value. You don't really need the certificate, so its free'd immediately.

Certificate Chain

You must confirm the server's certificate chains back to a trusted root, and all the certificates in the chain are valid. You do so as follows:

long res = SSL_get_verify_result(ssl);
if(!(X509_V_OK == res)) handleFailure();

SSL_get_verify_result returns the result of verifying the chain. See the earlier warning on doing the wrong thing in the verification callback.

Certificate Names

You must confirm a match between the hostname you contacted and the hostnames listed in the certificate. OpenSSL prior to 1.1.0 does not perform hostname verification, so you will have to perform the checking yourself. The sample code does not offer code at the moment, so you will need to borrow it or implement it.

If you want to borrow the code, take a look at libcurl and the verification procedure in source file ssluse.c. If you implement the code for checking, the sample code shows you how to extract the Common Name (CN) and Subject Alternate Names (SAN) from the certificate in print_cn_name and print_san_name.

Note: matching between the hostname (used in BIO_do_connect ) and names in the certificate (from SSL_get_peer_certificate) must also be validated. For example, a certificate cannot claim to be wildcarded for *.com, *.net, or other Top Level Domains (TLDs). In addition to the TLDs, you also have to country level or ccTLDs, so it can't match *.us, *.cn, *.fed.us, *.公司.cn or similar levels either. Mozilla maintains a list of ccTLDs that are off limits at the Public Suffix List, and there are currently 6136 entries on the list.

Program Output

After all this musing, here's the lousy output you get when running the program:

Downloads

openssl-bio-fetch.tar.gz - The program and Makefile used for this wiki page.