You said: "CRYPTO_cleanup_all_ex_data and ERR_remove_state should be called on each thread, and not just the main thread."
However the FAQ says:
"Brutal" (thread-unsafe) Application-global cleanup functions: ERR_free_strings(), EVP_cleanup() and CRYPTO_cleanup_all_ex_data().'
And the code comments for CRYPTO_cleanup_all_ex_data() say:
/* Release all "ex_data" state to prevent memory leaks. This can't be made * thread-safe without overhauling a lot of stuff, and shouldn't really be * called under potential race-conditions anyway (it's for program shutdown * after all). */