fixes PowerDNS/pdns/#661

I was doing some performance testing on our 32-way servers and discovered that in the simple case where the test program returns the data straight away, about 80% of the time was being spent in the kernel in a spinlock. I've been debugging this looking at the coprocess code and discovered that this is due to the following line in coprocess.cc:

setbuf(d_fp,0); // no buffering please, confuses select

If this is removed, performance in my particular test case goes from 2000qps with powerdns running at about 2000% cpu to 10000qps with powerdns using about 300% cpu.

Obviously the comment implies that this is not a permanent solution, I guess if the timeout is specified as 0 then the select code won't be executed and so you can disable the setbuf easily enough. However perhaps if the timeout is wanted you could set an alarm() rather than using select?

diff --git a/modules/pipebackend/coprocess.cc b/modules/pipebackend/coprocess.cc
index 8357781209484f420be4fe49b123c5076c98fb26..c6ac83a372e197101ea6c0b2360f084efd85ead7 100644 (file)
--- a/modules/pipebackend/coprocess.cc
+++ b/modules/pipebackend/coprocess.cc
@@ -52,7 +52,8 @@ void CoProcess::launch(const char **argv, int timeout, int infd, int outfd)
     Utility::setCloseOnExec(d_fd2[0]);
     if(!(d_fp=fdopen(d_fd2[0],"r")))
       throw PDNSException("Unable to associate a file pointer with pipe: "+stringerror());
-    setbuf(d_fp,0); // no buffering please, confuses select
+    if( d_timeout)
+      setbuf(d_fp,0); // no buffering please, confuses select
   }
   else if(!d_pid) { // child
     signal(SIGCHLD, SIG_DFL); // silence a warning from perl

diff --git a/pdns/docs/pdns.xml b/pdns/docs/pdns.xml
index 849ce521e81749292112d65d88034d4065d4c74a..b9117ad3ea4db7b0b9848a6963ce582351ee6bf8 100644 (file)
--- a/pdns/docs/pdns.xml
+++ b/pdns/docs/pdns.xml
@@ -17028,6 +17028,14 @@ This setting will make PowerDNS renotify the slaves after an AXFR is *received*
              <para>
                Number of milliseconds to wait for an answer from the backend. If this time is ever exceeded, the backend
                is declared dead and a new process is spawned. Available since version 2.7.
+
+             </para>
+             <para>
+          Note that if you are running pipes over many (8+) cores, if you use
+          the pipe-timeout option you will hit a scaleability bottleneck above
+          about 2000 queries per second. If you set pipe-timeout to 0 this
+          bottleneck will go away and scaling will be linear to number of
+          cores.
              </para>
            </listitem>
          </varlistentry>