Fix SSL deadlock risk in libpq

In libpq, we set up and pass to OpenSSL callback routines to handle
locking.  When we run out of SSL connections, we try to clean things
up by de-registering the hooks.  Unfortunately, we had a few calls
into the OpenSSL library after these hooks were de-registered during
SSL cleanup which lead to deadlocking.  This moves the thread callback
cleanup to be after all SSL-cleanup related OpenSSL library calls.
I've been unable to reproduce the deadlock with this fix.

In passing, also move the close_SSL call to be after unlocking our
ssl_config mutex when in a failure state.  While it looks pretty
unlikely to be an issue, it could have resulted in deadlocks if we
ended up in this code path due to something other than SSL_new
failing.  Thanks to Heikki for pointing this out.

Back-patch to all supported versions; note that the close_SSL issue
only goes back to 9.0, so that hunk isn't included in the 8.4 patch.

Initially found and reported by Vesa-Matti J Kari; many thanks to
both Heikki and Andres for their help running down the specific
issue and reviewing the patch.

diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 847f2721dd8f994a31ec892e15317cb94e4e1017..a07c44ad14b061915191982eaf83e7cd625efcb9 100644 (file)
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -1430,13 +1430,22 @@ open_client_SSL(PGconn *conn)
 static void
 close_SSL(PGconn *conn)
 {
+       bool destroy_needed = false;
+
        if (conn->ssl)
        {
                DISABLE_SIGPIPE((void) 0);
+
+               /*
+                * We can't destroy everything SSL-related here due to the possible
+                * later calls to OpenSSL routines which may need our thread
+                * callbacks, so set a flag here and check at the end.
+                */
+               destroy_needed = true;
+
                SSL_shutdown(conn->ssl);
                SSL_free(conn->ssl);
                conn->ssl = NULL;
-               pqsecure_destroy();
                /* We have to assume we got EPIPE */
                REMEMBER_EPIPE(true);
                RESTORE_SIGPIPE();
@@ -1456,6 +1465,17 @@ close_SSL(PGconn *conn)
                conn->engine = NULL;
        }
 #endif
+
+       /*
+        * This will remove our SSL locking hooks, if this is the last SSL
+        * connection, which means we must wait to call it until after all
+        * SSL calls have been made, otherwise we can end up with a race
+        * condition and possible deadlocks.
+        *
+        * See comments above destroy_ssl_system().
+        */
+       if (destroy_needed)
+               pqsecure_destroy();
 }
 
 /*