-From mscott@sacadia.com Wed Nov 15 14:50:19 2000
-Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id OAA11583
- for <pgman@candle.pha.pa.us>; Wed, 15 Nov 2000 14:50:13 -0500 (EST)
-Received: from localhost (localhost [127.0.0.1])
- by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id LAA09998;
- Wed, 15 Nov 2000 11:35:33 -0800 (PST)
-Date: Wed, 15 Nov 2000 11:35:33 -0800 (PST)
-From: Myron Scott <mscott@sacadia.com>
-X-Sender: mscott@goldengate.kojoworldwide.com.
-To: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>,
- Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
-Subject: Please help with some advice
-Message-ID: <Pine.GSO.4.10.10011151053260.9940-100000@goldengate.kojoworldwide.com.>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-Status: ORr
-
-Dear Sirs,
-
-I have been lurking on the PostgreSQL hackers list for about 3 months now
-and your names comes up more than any with helpful info about the project
-so I was hoping you could help me.
-
-Let me cut to the chase. I have been experimenting with 7.0.2 source to
-see if I could create a mutlti-threaded version of the backend so
-I could link directly from java ( I have a fe<->be protocol that I use for
-my apps). Needless to say I got into much more than I bargained for. I
-now have a version that works and it has some nice benefits that are very
-helpful to a project that I am working on. What I gained was
-
-prepared statements outside of spi
-batched commits (fsync)
-one connection per thread
- multiple threads per process
- multiple processes per installation
-
-I never really intended for anyone else to see the work so I drifted
-pretty far from the original code. I also ended up using Solaris threads
-rather than pthreads, I did my own implementation of the bufmgr.c and
-gram.y, and used Solaris implementation of mutex in place of S_LOCK and
-TAS. I grabbed all global variables and put them in an environment
-variable that is thread local. I also did some really stupid
-things like making TransactionId uint64 and making all my inserts use the
-same oid.
-
-My question is this. I would like to get some critical feedback and
-suggestions about the work from others. What is the best way to go about
-this? I thought about trying to create a project on greatbridge.org
-but I am rather new to open source and the code needs commented properly
-and cleaned up before too many try and look at it.
-
-Any suggestions would be greatly appreciated.
-
-
-Thanks in advance,
-
-Myron Scott
-
-
-
-From mscott@sacadia.com Thu Nov 16 17:19:45 2000
-Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA04315
- for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 17:19:43 -0500 (EST)
-Received: from localhost (localhost [127.0.0.1])
- by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id OAA11449;
- Thu, 16 Nov 2000 14:05:15 -0800 (PST)
-Date: Thu, 16 Nov 2000 14:05:15 -0800 (PST)
-From: Myron Scott <mscott@sacadia.com>
-X-Sender: mscott@goldengate.kojoworldwide.com.
-To: Bruce Momjian <pgman@candle.pha.pa.us>
-cc: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>, Tom Lane <tgl@sss.pgh.pa.us>
-Subject: Re: Please help with some advice
-In-Reply-To: <200011160533.AAA27886@candle.pha.pa.us>
-Message-ID: <Pine.GSO.4.10.10011161401570.11441-100000@goldengate.kojoworldwide.com.>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-Status: OR
-
-Bruce Momjian wrote:
-
->I am curious how you isolated each thread. It seems we pretty much
->assume all our memory is controlled by a single query in the process.
-
-
-I moved all global variables to a thread global variable which is accessed
-by the method GetEnv(). Which looks like this
-
-Env* GetEnv(void) {
- Env* env;
- thr_getspecific(*envkey,(void*)&env);
- return env;
-}
-
-The Env struct includes the CurrentMemoryContext, TopMemoryContext,
-PortalHeapMemory for each instance of a connection (one thread per
-connection). So, for example,
-EndPortalAllocMode uses GetEnv()->CurrentMemoryContext
-
-void
-EndPortalAllocMode()
-{
- PortalHeapMemory context;
-
- AssertState(PortalManagerEnabled);
- AssertState(IsA(GetEnv()->CurrentMemoryContext,
-PortalHeapMemory));
-
- context = (PortalHeapMemory) GetEnv()->CurrentMemoryContext;
- AssertState(PointerIsValid(context->block)); /* XXX
-Trap(...) */
-
- /* free current mode */
- AllocSetReset(&HEAPMEMBLOCK(context)->setData);
- MemoryContextFree((MemoryContext)
-PortalHeapMemoryGetVariableMemory(context),
- context->block);
-
- /* restore previous mode */
- context->block = FixedStackPop(&context->stackData);
-}
-
-
-
-
-From vmikheev@SECTORBASE.COM Thu Nov 16 17:23:22 2000
-Received: from sectorbase2.sectorbase.com ([208.48.122.131])
- by candle.pha.pa.us (8.9.0/8.9.0) with SMTP id RAA04562
- for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 17:23:21 -0500 (EST)
-Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2650.21)
- id <V8XQB5RW>; Thu, 16 Nov 2000 14:05:24 -0800
-Message-ID: <8F4C99C66D04D4118F580090272A7A234D318D@sectorbase1.sectorbase.com>
-From: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
-To: "'Myron Scott'" <mscott@sacadia.com>,
- Bruce Momjian
- <pgman@candle.pha.pa.us>
-Cc: Tom Lane <tgl@sss.pgh.pa.us>
-Subject: RE: Please help with some advice
-Date: Thu, 16 Nov 2000 14:09:30 -0800
-MIME-Version: 1.0
-X-Mailer: Internet Mail Service (5.5.2650.21)
-Content-Type: text/plain;
- charset="iso-8859-1"
-Status: ORr
-
-I think the question do we want to make backend multy-threaded
-should be discussed in hackers.
-
-Vadim
-
-> -----Original Message-----
-> From: Myron Scott [mailto:mscott@sacadia.com]
-> Sent: Thursday, November 16, 2000 2:05 PM
-> To: Bruce Momjian
-> Cc: Mikheev, Vadim; Tom Lane
-> Subject: Re: Please help with some advice
->
->
-> Bruce Momjian wrote:
->
-> >I am curious how you isolated each thread. It seems we pretty much
-> >assume all our memory is controlled by a single query in the process.
->
->
->
-> I moved all global variables to a thread global variable
-> which is accessed
-> by the method GetEnv(). Which looks like this
->
-> Env* GetEnv(void) {
-> Env* env;
-> thr_getspecific(*envkey,(void*)&env);
-> return env;
-> }
->
-> The Env struct includes the CurrentMemoryContext, TopMemoryContext,
-> PortalHeapMemory for each instance of a connection (one thread per
-> connection). So, for example,
-> EndPortalAllocMode uses GetEnv()->CurrentMemoryContext
->
-> void
-> EndPortalAllocMode()
-> {
-> PortalHeapMemory context;
->
-> AssertState(PortalManagerEnabled);
-> AssertState(IsA(GetEnv()->CurrentMemoryContext,
-> PortalHeapMemory));
->
-> context = (PortalHeapMemory) GetEnv()->CurrentMemoryContext;
-> AssertState(PointerIsValid(context->block)); /* XXX
-> Trap(...) */
->
-> /* free current mode */
-> AllocSetReset(&HEAPMEMBLOCK(context)->setData);
-> MemoryContextFree((MemoryContext)
-> PortalHeapMemoryGetVariableMemory(context),
-> context->block);
->
-> /* restore previous mode */
-> context->block = FixedStackPop(&context->stackData);
-> }
->
->
->
-
-From mscott@sacadia.com Thu Nov 16 22:16:38 2000
-Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id WAA14638
- for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 22:16:36 -0500 (EST)
-Received: from localhost (localhost [127.0.0.1])
- by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id TAA11874;
- Thu, 16 Nov 2000 19:04:48 -0800 (PST)
-Date: Thu, 16 Nov 2000 19:04:48 -0800 (PST)
-From: Myron Scott <mscott@sacadia.com>
-X-Sender: mscott@goldengate.kojoworldwide.com.
-To: Bruce Momjian <pgman@candle.pha.pa.us>
-cc: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>, Tom Lane <tgl@sss.pgh.pa.us>
-Subject: Re: Please help with some advice
-In-Reply-To: <200011170156.UAA11438@candle.pha.pa.us>
-Message-ID: <Pine.GSO.4.10.10011161904140.11870-100000@goldengate.kojoworldwide.com.>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-Status: ORr
-
-Thanks very much, I will post to hackers.
-
-Myron
-
-
-
-From pgsql-hackers-owner+M2691@postgresql.org Tue Jan 2 00:30:20 2001
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA08195
- for <pgman@candle.pha.pa.us>; Tue, 2 Jan 2001 00:30:19 -0500 (EST)
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f025UjL33335;
- Tue, 2 Jan 2001 00:30:45 -0500 (EST)
- (envelope-from pgsql-hackers-owner+M2691@postgresql.org)
-Received: from mailsys01.intnet.net (tmail.wwc.com [198.252.32.143] (may be forged))
- by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f025UTL33232
- for <pgsql-hackers@postgresql.org>; Tue, 2 Jan 2001 00:30:32 -0500 (EST)
- (envelope-from mscott@sacadia.com)
-Received: from [206.112.108.0] (HELO sacadia.com)
- by mailsys01.intnet.net (CommuniGate Pro SMTP 3.3.2)
- with ESMTP id 2214231; Tue, 02 Jan 2001 00:29:47 -0500
-Message-ID: <3A5167DB.3050807@sacadia.com>
-Date: Mon, 01 Jan 2001 21:32:11 -0800
-From: Myron Scott <mscott@sacadia.com>
-Reply-To: mscott@sacadia.com
-User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; m18) Gecko/20001108 Netscape6/6.0
+From pgsql-hackers-owner+M13599=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 17:25:32 2001
+Return-path: <pgsql-hackers-owner+M13599=candle.pha.pa.us=pgman@postgresql.org>
+Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8QLPWo07589
+ for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 17:25:32 -0400 (EDT)
+Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
+ by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8QLPf405606
+ for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 16:25:41 -0500 (CDT)
+ (envelope-from pgsql-hackers-owner+M13599=candle.pha.pa.us=pgman@postgresql.org)
+Received: from gromit.dotclick.com (ipn9-f8366.net-resource.net [216.204.83.66])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id f8QKj3h82020
+ for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 16:45:03 -0400 (EDT)
+ (envelope-from markw@mohawksoft.com)
+Received: from mohawksoft.com (IDENT:markw@localhost.localdomain [127.0.0.1])
+ by gromit.dotclick.com (8.9.3/8.9.3) with ESMTP id QAA23693;
+ Wed, 26 Sep 2001 16:43:02 -0400
+Message-ID: <3BB23DD6.E86AF327@mohawksoft.com>
+Date: Wed, 26 Sep 2001 16:43:02 -0400
+From: mlw <markw@mohawksoft.com>
+X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.4.2 i686)
X-Accept-Language: en
MIME-Version: 1.0
-To: "Ross J. Reedstrom" <reedstrm@rice.edu>
-CC: pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] Using Threads?
-References: <004401c058fd$fd498d40$f2356880@tracy> <Pine.GSO.4.10.10012032351040.28161-100000@goldengate.kojoworldwide.com.> <20001204113307.B5871@rice.edu>
-Content-Type: text/plain; charset=us-ascii; format=flowed
-Content-Transfer-Encoding: 7bit
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-For anyone interested,
-
-I have posted my multi-threaded version of PostgreSQL here.
-
-http://www.sacadia.com/mtpg.html
-
-It is based on 7.0.2 and the TAO CORBA ORB which is here.
-
-http://www.cs.wustl.edu/~schmidt/TAO.html
-
-Myron Scott
-mkscott@sacadia.com
-
-
-
-From bright@fw.wintelcom.net Tue Jan 2 03:02:28 2001
-Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA16169
- for <pgman@candle.pha.pa.us>; Tue, 2 Jan 2001 03:02:27 -0500 (EST)
-Received: (from bright@localhost)
- by fw.wintelcom.net (8.10.0/8.10.0) id f0282Vm10623;
- Tue, 2 Jan 2001 00:02:31 -0800 (PST)
-Date: Tue, 2 Jan 2001 00:02:31 -0800
-From: Alfred Perlstein <bright@wintelcom.net>
-To: Bruce Momjian <pgman@candle.pha.pa.us>
-Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] Assuming that TAS() will succeed the first time is verboten
-Message-ID: <20010102000230.C19572@fw.wintelcom.net>
-References: <9850.978067943@sss.pgh.pa.us> <200101020759.CAA15836@candle.pha.pa.us>
-Mime-Version: 1.0
+To: "D. Hageman" <dhageman@dracken.com>,
+ "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Spinlock performance improvement proposal
+References: <Pine.LNX.4.33.0109261330030.1906-100000@typhon.dracken.com>
Content-Type: text/plain; charset=us-ascii
-Content-Disposition: inline
-User-Agent: Mutt/1.2.5i
-In-Reply-To: <200101020759.CAA15836@candle.pha.pa.us>; from pgman@candle.pha.pa.us on Tue, Jan 02, 2001 at 02:59:20AM -0500
-Status: OR
-
-* Bruce Momjian <pgman@candle.pha.pa.us> [010101 23:59] wrote:
-> > Alfred Perlstein <bright@wintelcom.net> writes:
-> > > One trick that may help is calling sched_yield(2) on a lock miss,
-> > > it's a POSIX call and quite new so you'd need a 'configure' test
-> > > for it.
-> >
-> > The author of the current s_lock code seems to have thought that
-> > select() with a zero delay would do the equivalent of sched_yield().
-> > I'm not sure if that's true on very many kernels, if indeed any...
-> >
-> > I doubt we could buy much by depending on sched_yield(); if you want
-> > to assume POSIX facilities, ISTM you might as well go for user-space
-> > semaphores and forget the whole TAS mechanism.
->
->
-> Another issue is that sched_yield brings in the pthreads library/hooks
-> on some OS's, which we certainly want to avoid.
-
-I know it's a major undertaking, but since the work is sort of done,
-have you guys considered the port to solaris threads and seeing about
-making a pthreads port of that?
-
-I know it would probably get you considerable gains under Windows
-at the expense of dropping some really really legacy system.
-
-Or you could do what apache (is rumored) does and have it do either
-threads or processes or both...
-
---
--Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
-"I have the heart of a child; I keep it in a jar on my desk."
-
-From pgsql-hackers-owner+M4275@postgresql.org Mon Feb 5 21:45:00 2001
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA09262
- for <pgman@candle.pha.pa.us>; Mon, 5 Feb 2001 21:44:59 -0500 (EST)
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f162ixx00920;
- Mon, 5 Feb 2001 21:44:59 -0500 (EST)
- (envelope-from pgsql-hackers-owner+M4275@postgresql.org)
-Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
- by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f162fSx00595
- for <pgsql-hackers@postgresql.org>; Mon, 5 Feb 2001 21:41:29 -0500 (EST)
- (envelope-from mscott@sacadia.com)
-Received: from localhost (localhost [127.0.0.1])
- by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id SAA03298
- for <pgsql-hackers@postgresql.org>; Mon, 5 Feb 2001 18:25:05 -0800 (PST)
-Date: Mon, 5 Feb 2001 18:25:05 -0800 (PST)
-From: Myron Scott <mscott@sacadia.com>
-X-Sender: mscott@goldengate.kojoworldwide.com.
-To: pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] Using Threads?
-Message-ID: <Pine.GSO.4.10.10102051823210.3289-100000@goldengate.kojoworldwide.com.>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-I have put a new version of my multi-threaded
-postgresql experiment at
-
-http://www.sacadia.com/mtpg.html
-
-This one actually works. I have added a server
-based on omniORB, a CORBA 2.3 ORB from ATT. It
- is much smaller than TAO and uses the thread per
-connection model. I haven't added the java side
-of the JNI interface yet but the C++ side is there.
-
-It's still not stable but it is much better than
-the last.
-
-Myron Scott
-mkscott@sacadia.com
-
-
-
-
-
-From pgsql-hackers-owner+M4304@postgresql.org Tue Feb 6 10:24:21 2001
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA22027
- for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 10:24:20 -0500 (EST)
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16FOBx97182;
- Tue, 6 Feb 2001 10:24:11 -0500 (EST)
- (envelope-from pgsql-hackers-owner+M4304@postgresql.org)
-Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
- by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16FLWx96814
- for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 10:21:33 -0500 (EST)
- (envelope-from mscott@sacadia.com)
-Received: from localhost (localhost [127.0.0.1])
- by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id HAA04170;
- Tue, 6 Feb 2001 07:05:04 -0800 (PST)
-Date: Tue, 6 Feb 2001 07:05:04 -0800 (PST)
-From: Myron Scott <mscott@sacadia.com>
-X-Sender: mscott@goldengate.kojoworldwide.com.
-To: Karel Zak <zakkr@zf.jcu.cz>
-cc: pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] Using Threads
-In-Reply-To: <Pine.LNX.3.96.1010206101030.20355B-100000@ara.zf.jcu.cz>
-Message-ID: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-
->
-> Sorry I haven't time to see and test your experiment,
-> but I have a question. How you solve memory management?
-> The current mmgr is based on global variable
-> CurrentMemoryContext that is very often changed and used.
-> Use you for this locks? If yes it is probably problematic
-> point for perfomance.
->
-> Karel
->
-
-There are many many globals I had to work around including all the memory
-management stuff. I basically threw everything into and "environment"
-variable which I stored in a thread specific using thr_setspecific.
-
-Performance is acually very good for what I am doing. I was able to batch
-commit transactions which cuts down on fsync calls, use prepared
-statements from my client using CORBA, and the various locking calls for
-the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast. I did
-some performance tests for inserts
-
-20 clients, 900 inserts per client, 1 insert per transaction, 4 different
-tables.
-
-7.0.2 About 10:52 average completion
-multi-threaded 2:42 average completion
-7.1beta3 1:13 average completion
-
-If I increased the number of inserts per transaction, multi-threaded got
-closer to 7.1 for inserts. I haven't tested other other types of
-commands
-yet.
-
-
-Myron Scott
-mkscott@sacadia.com
-
-
-From pgsql-hackers-owner+M4313@postgresql.org Tue Feb 6 12:32:00 2001
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id MAA29163
- for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 12:31:59 -0500 (EST)
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16HVox17454;
- Tue, 6 Feb 2001 12:31:51 -0500 (EST)
- (envelope-from pgsql-hackers-owner+M4313@postgresql.org)
-Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4])
- by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16HV6x17323
- for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 12:31:06 -0500 (EST)
- (envelope-from zakkr@zf.jcu.cz)
-Received: from localhost (zakkr@localhost)
- by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id SAA03980;
- Tue, 6 Feb 2001 18:31:02 +0100
-Date: Tue, 6 Feb 2001 18:31:02 +0100 (CET)
-From: Karel Zak <zakkr@zf.jcu.cz>
-To: Myron Scott <mscott@sacadia.com>
-cc: pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] Using Threads
-In-Reply-To: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
-Message-ID: <Pine.LNX.3.96.1010206182112.3799B-100000@ara.zf.jcu.cz>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-
-On Tue, 6 Feb 2001, Myron Scott wrote:
-
-> There are many many globals I had to work around including all the memory
-> management stuff. I basically threw everything into and "environment"
-> variable which I stored in a thread specific using thr_setspecific.
-
- Yes, it's good. I working on multi-thread application server
-(http://mape.jcu.cz) and I use for this project some things from PG (like
-mmgr), I planning use same solution.
-
-> Performance is acually very good for what I am doing. I was able to batch
-> commit transactions which cuts down on fsync calls, use prepared
-> statements from my client using CORBA, and the various locking calls for
-> the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast. I did
-> some performance tests for inserts
->
-> 20 clients, 900 inserts per client, 1 insert per transaction, 4 different
-> tables.
->
-> 7.0.2 About 10:52 average completion
-> multi-threaded 2:42 average completion
-> 7.1beta3 1:13 average completion
-
-It is very very good for time for 7.1, already look forward to 7.2! :-)
-
- BTW, I not sure if you anytime in future will see threads in
-official PostgreSQL and if you spending time on relevant things (IMHO).
-
- Karel
-
-
-
-
-
-
-From pgsql-hackers-owner+M4304@postgresql.org Tue Feb 6 10:24:21 2001
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA22027
- for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 10:24:20 -0500 (EST)
-Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
- by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16FOBx97182;
- Tue, 6 Feb 2001 10:24:11 -0500 (EST)
- (envelope-from pgsql-hackers-owner+M4304@postgresql.org)
-Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
- by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16FLWx96814
- for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 10:21:33 -0500 (EST)
- (envelope-from mscott@sacadia.com)
-Received: from localhost (localhost [127.0.0.1])
- by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id HAA04170;
- Tue, 6 Feb 2001 07:05:04 -0800 (PST)
-Date: Tue, 6 Feb 2001 07:05:04 -0800 (PST)
-From: Myron Scott <mscott@sacadia.com>
-X-Sender: mscott@goldengate.kojoworldwide.com.
-To: Karel Zak <zakkr@zf.jcu.cz>
-cc: pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] Using Threads
-In-Reply-To: <Pine.LNX.3.96.1010206101030.20355B-100000@ara.zf.jcu.cz>
-Message-ID: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
+Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR
-
->
-> Sorry I haven't time to see and test your experiment,
-> but I have a question. How you solve memory management?
-> The current mmgr is based on global variable
-> CurrentMemoryContext that is very often changed and used.
-> Use you for this locks? If yes it is probably problematic
-> point for perfomance.
->
-> Karel
->
-
-There are many many globals I had to work around including all the memory
-management stuff. I basically threw everything into and "environment"
-variable which I stored in a thread specific using thr_setspecific.
-
-Performance is acually very good for what I am doing. I was able to batch
-commit transactions which cuts down on fsync calls, use prepared
-statements from my client using CORBA, and the various locking calls for
-the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast. I did
-some performance tests for inserts
-
-20 clients, 900 inserts per client, 1 insert per transaction, 4 different
-tables.
-
-7.0.2 About 10:52 average completion
-multi-threaded 2:42 average completion
-7.1beta3 1:13 average completion
-
-If I increased the number of inserts per transaction, multi-threaded got
-closer to 7.1 for inserts. I haven't tested other other types of
-commands
-yet.
-
-
-Myron Scott
-mkscott@sacadia.com
-
-
-From lamar.owen@wgcr.org Thu Jun 28 11:14:10 2001
-Return-path: <lamar.owen@wgcr.org>
-Received: from www.wgcr.org (IDENT:root@www.wgcr.org [206.74.232.194])
- by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5SFE9U18758
- for <pgman@candle.pha.pa.us>; Thu, 28 Jun 2001 11:14:09 -0400 (EDT)
-Received: from lowen.wgcr.org (IDENT:lowen@[10.1.2.3])
- by www.wgcr.org (8.9.3/8.9.3/WGCR) with SMTP id LAA11879;
- Thu, 28 Jun 2001 11:14:14 -0400
-Content-Type: text/plain;
- charset="iso-8859-1"
-From: Lamar Owen <lamar.owen@wgcr.org>
-To: Bruce Momjian <pgman@candle.pha.pa.us>
-Subject: Process weight (was:Re: [GENERAL] Re: Red Hat to support PostgreSQL)
-Date: Thu, 28 Jun 2001 11:14:09 -0400
-X-Mailer: KMail [version 1.2]
-References: <200106272258.f5RMwIb26959@candle.pha.pa.us>
-In-Reply-To: <200106272258.f5RMwIb26959@candle.pha.pa.us>
-MIME-Version: 1.0
-Message-ID: <01062811140902.01118@lowen.wgcr.org>
-Content-Transfer-Encoding: 8bit
-Status: ORr
-
-On Wednesday 27 June 2001 18:58, Bruce Momjian wrote:
-> > I had almost given up on using Postgres for this system because under
-> > Solaris, it just couldn't cut it (MySQL could do the work with one CPU
-> > while Postgres took up even more CPU and required *both* CPUs to be
-> > enabled), but when we moved the system to a Linux box, things worked
-> > much better.
-
-> Ah, back to a PostgreSQL topic. :-)
-
-> My guess on this one is that Solaris is slower for PostgreSQL because
-> process switching is _much_ heavier on Solaris than other OS's. This is
-> because of the way they implemented processes in SVr4. They got quite
-> heavy, almost requiring kernel threads so you weren't switching
-> processes all the time.
-
-Now, the question of the week:
-Is supporting a thread model for an inefficient OS a desirable thing to do,
-when more efficient OS kernels are available such as FreeBSD 4.x and Linux
-2.4? My opinion is that our existing model, when used with a
-connection-pooling frontend, is rather efficient. (Yes, I use a
-connection-pooling frontend. Performance is rather nice, and I don't have to
-have a full backend spawned for every page hit.)
-
-In fact, on a Linux box threads show as processes. While I know that the
-kernel actually supports themin a slightly different manner than processes,
-they have more similarities than differences.
-
-However, even on OS's where threads are supported, the mechanism to support
-those threads must be an efficient one -- not all pthreads libraries are
-created equal. Many are frontends (expensive ones, at that) for plain old
-processes.
-
-Does anyone know of a resource that details the 'weight' of processes for our
-supported platforms? [reply off-list -- I'll be glad to summarize responses
-to HACKERS, ADMIN, or PORTS, as appropriate, if desired.]
---
-Lamar Owen
-WGCR Internet Radio
-1 Peter 4:11
+"D. Hageman" wrote:
+
+> The plan for the new spinlocks does look like it has some potential. My
+> only comment in regards to permformance when we start looking at SMP
+> machines is ... it is my belief that getting a true threaded backend may
+> be the only way to get the full potential out of SMP machines. I see that
+> is one of the things to experiment with on the TODO list and I have seen
+> some people have messed around already with this using Solaris threads.
+> It should probably be attempted with pthreads if PostgreSQL is going to
+> keep some resemblance of cross-platform compatibility. At that time, it
+> would probably be easier to go in and clean up some stuff for the
+> implementation of other TODO items (put in the base framework for more
+> complex future items) as threading the backend would take a little bit of
+> ideology shift.
+
+I can only think of two objectives for threading. (1) running the various
+connections in their own thread instead of their own process. (2) running
+complex queries across multiple threads.
+
+For item (1) I see no value to this. It is a lot of work with no tangible
+benefit. If you have an old fashion pthreads implementation, it will hurt
+performance because are scheduled within the single process's time slice.. If
+you have a newer kernel scheduled implementation, then you will have the same
+scheduling as separate processes. The only thing you will need to do is
+switch your brain from figuring out how to share data, to trying to figure
+out how to isolate data. A multithreaded implementation lacks many of the
+benefits and robustness of a multiprocess implementation.
+
+For item (2) I can see how that could speed up queries in a low utilization
+system, and that would be cool, but in a server that is under load, threading
+the queries probably be less efficient.
+
+
+---------------------------(end of broadcast)---------------------------
+TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
projects / postgresql / commitdiff