From: Peter Eisentraut <peter@eisentraut.org>
Date: Mon, 23 Sep 2019 18:44:49 +0000 (+0200)
Subject: Add so_reuseport option
X-Git-Tag: pgbouncer_1_12_0~16
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=35b354044a6cb01b203560dd930944e5f84aaae8;p=pgbouncer

Add so_reuseport option

Adds an option to specify whether to set the socket option
`SO_REUSEPORT` on TCP listening sockets.  On some operating systems,
this allows running multiple PgBouncer instances on the same host
listening on the same port and having the kernel distribute the
connections automatically.  This option is a way to get PgBouncer to
use more CPU cores.

closes #345
---

diff --git a/doc/config.md b/doc/config.md
index 4a4e8b2..f1c06b8 100644
--- a/doc/config.md
+++ b/doc/config.md
@@ -734,6 +734,35 @@ PgBouncer for a long time.  One loop processes one `pkt_buf` amount of data.
 
 Default: 5
 
+### so_reuseport
+
+Specifies whether to set the socket option `SO_REUSEPORT` on TCP
+listening sockets.  On some operating systems, this allows running
+multiple PgBouncer instances on the same host listening on the same
+port and having the kernel distribute the connections automatically.
+This option is a way to get PgBouncer to use more CPU cores.
+(PgBouncer is single-threaded and uses one CPU core per instance.)
+
+The behavior in detail depends on the operating system kernel.  As of
+this writing, this setting has the desired effect on (sufficiently
+recent versions of) Linux, DragonFlyBSD, and FreeBSD.  (On FreeBSD, it
+applies the socket option `SO_REUSEPORT_LB` instead.)  Some other
+operating systems support the socket option but it won't have the
+desired effect: It will allow multiple processes to bind to the same
+port but only one of them will get the connections.  See your
+operating system's setsockopt() documentation for details.
+
+On systems that don't support the socket option at all, turning this
+setting on will result in an error.
+
+Each PgBouncer instance on the same host needs different settings for
+at least `unix_socket_dir` and `pidfile`, as well as `logfile` if that
+is used.  Also note that if you make use of this option, you can no
+longer connect to a specific PgBouncer instance via TCP/IP, which
+might have implications for monitoring and metrics collection.
+
+Default: 0
+
 ### tcp_defer_accept
 
 For details on this and other tcp options, please see `man 7 tcp`.
diff --git a/etc/pgbouncer.ini b/etc/pgbouncer.ini
index 2a00596..d72dede 100644
--- a/etc/pgbouncer.ini
+++ b/etc/pgbouncer.ini
@@ -291,6 +291,9 @@ auth_file = /etc/pgbouncer/userlist.txt
 ;; Maximum PostgreSQL protocol packet size.
 ;max_packet_size = 2147483647
 
+;; Set SO_REUSEPORT socket option
+;so_reuseport = 0
+
 ;; networking options, for info: man 7 tcp
 
 ;; Linux: Notify program about new connection only if there is also
diff --git a/include/bouncer.h b/include/bouncer.h
index 95f4a4e..c1ead6a 100644
--- a/include/bouncer.h
+++ b/include/bouncer.h
@@ -486,6 +486,7 @@ extern int cf_reboot;
 extern unsigned int cf_max_packet_size;
 
 extern int cf_sbuf_loopcnt;
+extern int cf_so_reuseport;
 extern int cf_tcp_keepalive;
 extern int cf_tcp_keepcnt;
 extern int cf_tcp_keepidle;
diff --git a/src/main.c b/src/main.c
index 0c0ccd0..13d1107 100644
--- a/src/main.c
+++ b/src/main.c
@@ -76,6 +76,7 @@ int cf_pool_mode = POOL_SESSION;
 /* sbuf config */
 int cf_sbuf_len;
 int cf_sbuf_loopcnt;
+int cf_so_reuseport;
 int cf_tcp_socket_buffer;
 #if defined(TCP_DEFER_ACCEPT) || defined(SO_ACCEPTFILTER)
 int cf_tcp_defer_accept = 1;
@@ -209,6 +210,7 @@ CF_ABS("logfile", CF_STR, cf_logfile, 0, ""),
 CF_ABS("pidfile", CF_STR, cf_pidfile, CF_NO_RELOAD, ""),
 CF_ABS("listen_addr", CF_STR, cf_listen_addr, CF_NO_RELOAD, ""),
 CF_ABS("listen_port", CF_INT, cf_listen_port, CF_NO_RELOAD, "6432"),
+CF_ABS("so_reuseport", CF_INT, cf_so_reuseport, CF_NO_RELOAD, "0"),
 CF_ABS("listen_backlog", CF_INT, cf_listen_backlog, CF_NO_RELOAD, "128"),
 #ifndef WIN32
 CF_ABS("unix_socket_dir", CF_STR, cf_unix_socket_dir, CF_NO_RELOAD, "/tmp"),
diff --git a/src/pooler.c b/src/pooler.c
index 1a74cff..9dd9a3b 100644
--- a/src/pooler.c
+++ b/src/pooler.c
@@ -120,6 +120,30 @@ static bool add_listen(int af, const struct sockaddr *sa, int salen)
 	}
 #endif
 
+	/*
+	 * If configured, set SO_REUSEPORT or equivalent.  If it's not
+	 * enabled, just leave the socket alone.  (We could also unset
+	 * the socket option in that case, but this area is fairly
+	 * unportable, so perhaps better to avoid it.)
+	 */
+	if (af != AF_UNIX && cf_so_reuseport) {
+#if defined(SO_REUSEPORT)
+		int val = 1;
+		errpos = "setsockopt/SO_REUSEPORT";
+		res = setsockopt(sock, SOL_SOCKET, SO_REUSEPORT, &val, sizeof(val));
+		if (res < 0)
+			goto failed;
+#elif defined(SO_REUSEPORT_LB)
+		int val = 1;
+		errpos = "setsockopt/SO_REUSEPORT_LB";
+		res = setsockopt(sock, SOL_SOCKET, SO_REUSEPORT_LB, &val, sizeof(val));
+		if (res < 0)
+			goto failed;
+#else
+		fatal("so_reuseport not supported on this platform");
+#endif
+	}
+
 	/* bind it */
 	errpos = "bind";
 	res = bind(sock, sa, salen);