+++ /dev/null
-From pgsql-hackers-owner+M46352=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 02:20:11 2003
-Return-path: <pgsql-hackers-owner+M46352=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA37K9511168
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 02:20:10 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGXy7-0002PD-Dn
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 00:13:39 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id C7586D1CA89
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 06:08:20 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 93156-10
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 02:07:49 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id A35A6D1C9FF
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 02:07:46 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP id 657631E1A
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 01:07:45 -0500 (EST)
-Subject: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-Content-Type: text/plain
-Message-ID: <1067839664.3089.173.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 01:07:45 -0500
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-A couple days ago, Manfred Spraul mentioned the posix_fadvise() API on
--hackers:
-
-http://www.opengroup.org/onlinepubs/007904975/functions/posix_fadvise.html
-
-I'm working on making use of posix_fadvise() where appropriate. I can
-think of the following places where this would be useful:
-
-(1) As Manfred originally noted, when we advance to a new XLOG segment,
-we can use POSIX_FADV_DONTNEED to let the kernel know we won't be
-accessing the old WAL segment anymore. I've attached a quick kludge of a
-patch that implements this. I haven't done any benchmarking of it yet,
-though (comments or benchmark results are welcome).
-
-(2) ISTM that we can set POSIX_FADV_RANDOM for *all* indexes, since the
-vast majority of the accesses to them shouldn't be sequential. Are there
-any situations in which this assumption doesn't hold? (Perhaps B+-tree
-bulk loading, or CLUSTER?) Should this be done per-index-AM, or
-globally?
-
-(3) When doing VACUUM, ANALYZE, or large sequential scans (for some
-reasonable definition of "large"), we can use POSIX_FADV_SEQUENTIAL.
-
-(4) Various other components, such as tuplestore, tuplesort, and any
-utility commands that need to scan through an entire user relation for
-some reason. Once we've got the APIs for doing this worked out, it
-should be relatively easy to add other uses of posix_fadvise().
-
-(5) I'm hesitant to make use of POSIX_FADV_DONTNEED in VACUUM, as has
-been suggested elsewhere. The problem is that it's all-or-nothing: if
-the VACUUM happens to look at hot pages, these will be flushed from the
-page cache, so the net result may be a loss.
-
-So what API is desirable for uses 2-4? I'm thinking of adding a new
-function to the smgr API, smgradvise(). Given a Relation and an advice,
-this would:
-
-(a) propagate the advice for this relation to all the open FDs for the
-relation
-
-(b) store the new advice somewhere so that new FDs for the relation can
-have this advice set for them: clients should just be able to call
-smgradvise() without needing to worry if someone else has already called
-smgropen() for the relation in the past. One problem is how to store
-this: I don't think it can be a field of RelationData, since that is
-transient. Any suggestions?
-
-Note that I'm assuming that we don't need to set advice on sub-sections
-of a relation, although the posix_fadvise() API allows it -- does anyone
-think that would be useful?
-
-One potential issue is that when one process calls posix_fadvise() on a
-particular FD, I'd expect that other processes accessing the same file
-will be affected. For example, enabling FADV_SEQUENTIAL while we're
-vacuuming a relation will mean that another client doing a concurrent
-SELECT on the relation will see different readahead behavior. That
-doesn't seem like a major problem though.
-
-BTW, posix_fadvise() is currently only supported on Linux 2.6 w/ a
-recent version of glibc (BSD hackers, if you're listening,
-posix_fadvise() would be a very cool thing to have :P). So we'll need to
-do the appropriate configure magic to ensure we only use it where its
-available. Thankfully, it is a POSIX standard, so I would expect that in
-the years to come it will be available on more platforms.
-
-Any comments would be welcome.
-
--Neil
-
-
-
----------------------------(end of broadcast)---------------------------
-TIP 7: don't forget to increase your free space map settings
-
-From pgsql-hackers-owner+M46354=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 04:16:05 2003
-Return-path: <pgsql-hackers-owner+M46354=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA39G4519850
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 04:16:04 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGY3D-0002fz-QO
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 00:18:55 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id C35A5D1C9FF
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 06:16:01 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 02547-01
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 02:15:31 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id A2D66D1CB3D
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 02:15:30 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP id B7CE81E1A
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 01:15:30 -0500 (EST)
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <1067839664.3089.173.camel@tokyo>
-References: <1067839664.3089.173.camel@tokyo>
-Content-Type: multipart/mixed; boundary="=-FWP1piDRdCKsDZuLvApE"
-Message-ID: <1067840130.3089.177.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 01:15:30 -0500
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
---=-FWP1piDRdCKsDZuLvApE
-Content-Type: text/plain
-Content-Transfer-Encoding: 7bit
-
-On Mon, 2003-11-03 at 01:07, Neil Conway wrote:
-> (1) As Manfred originally noted, when we advance to a new XLOG segment,
-> we can use POSIX_FADV_DONTNEED to let the kernel know we won't be
-> accessing the old WAL segment anymore. I've attached a quick kludge of a
-> patch that implements this. I haven't done any benchmarking of it yet,
-> though (comments or benchmark results are welcome).
-
-Woops, the patch is attached.
-
--Neil
-
-
---=-FWP1piDRdCKsDZuLvApE
-Content-Disposition: attachment; filename=xlog-fadvise-1.patch
-Content-Type: text/x-patch; name=xlog-fadvise-1.patch; charset=ANSI_X3.4-1968
-Content-Transfer-Encoding: 7bit
-
-Index: src/backend/access/transam/xlog.c
-===================================================================
-RCS file: /var/lib/cvs/pgsql-server/src/backend/access/transam/xlog.c,v
-retrieving revision 1.125
-diff -c -r1.125 xlog.c
-*** src/backend/access/transam/xlog.c 27 Sep 2003 18:16:35 -0000 1.125
---- src/backend/access/transam/xlog.c 3 Nov 2003 02:46:57 -0000
-***************
-*** 1043,1048 ****
---- 1043,1060 ----
- */
- if (openLogFile >= 0)
- {
-+ /*
-+ * Let the kernel know that we're not going to need
-+ * this WAL segment anymore, so there's no need to
-+ * keep it in the I/O cache
-+ */
-+ if (posix_fadvise(openLogFile, 0, 0, POSIX_FADV_DONTNEED) != 0)
-+ {
-+ ereport(WARNING,
-+ (errcode_for_file_access(),
-+ errmsg("could not posix_fadvise() log file %u: %m", openLogId)));
-+ }
-+
- if (close(openLogFile) != 0)
- ereport(PANIC,
- (errcode_for_file_access(),
-***************
-*** 1159,1164 ****
---- 1171,1188 ----
- if (openLogFile >= 0 &&
- !XLByteInPrevSeg(LogwrtResult.Write, openLogId, openLogSeg))
- {
-+ /*
-+ * Let the kernel know that we're not going to need
-+ * this WAL segment anymore, so there's no need to
-+ * keep it in the I/O cache
-+ */
-+ if (posix_fadvise(openLogFile, 0, 0, POSIX_FADV_DONTNEED) != 0)
-+ {
-+ ereport(WARNING,
-+ (errcode_for_file_access(),
-+ errmsg("could not posix_fadvise() log file %u: %m", openLogId)));
-+ }
-+
- if (close(openLogFile) != 0)
- ereport(PANIC,
- (errcode_for_file_access(),
-
---=-FWP1piDRdCKsDZuLvApE
-Content-Type: text/plain
-Content-Disposition: inline
-Content-Transfer-Encoding: 8bit
-MIME-Version: 1.0
-
-
----------------------------(end of broadcast)---------------------------
-TIP 6: Have you searched our list archives?
-
- http://archives.postgresql.org
-
---=-FWP1piDRdCKsDZuLvApE--
-
-From pgsql-hackers-owner+M46358=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 04:30:38 2003
-Return-path: <pgsql-hackers-owner+M46358=pgman=candle.pha.pa.us@postgresql.org>
-Received: from hosting.commandprompt.com (222.commandprompt.com [207.173.200.222])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA39UY522930
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 04:30:36 -0500 (EST)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by hosting.commandprompt.com (8.11.6/8.11.6) with ESMTP id hA39UMm25323
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 01:30:32 -0800
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id D53FED1CB31
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 09:24:28 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 21316-02
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 05:23:58 -0400 (AST)
-Received: from fuji.krosing.net (silmet.estpak.ee [194.126.97.78])
- by svr1.postgresql.org (Postfix) with ESMTP id 5B0FED1CAC2
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 05:23:56 -0400 (AST)
-Received: from fuji.krosing.net (localhost.localdomain [127.0.0.1])
- by fuji.krosing.net (8.12.8/8.12.8) with ESMTP id hA39La7Q002784;
- Mon, 3 Nov 2003 11:21:36 +0200
-Received: (from hannu@localhost)
- by fuji.krosing.net (8.12.8/8.12.8/Submit) id hA39LaAZ002782;
- Mon, 3 Nov 2003 11:21:36 +0200
-X-Authentication-Warning: fuji.krosing.net: hannu set sender to hannu@tm.ee using -f
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Hannu Krosing <hannu@tm.ee>
-To: Neil Conway <neilc@samurai.com>
-cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <1067839664.3089.173.camel@tokyo>
-References: <1067839664.3089.173.camel@tokyo>
-Content-Type: text/plain
-Content-Transfer-Encoding: 7bit
-Message-ID: <1067851295.2580.12.camel@fuji.krosing.net>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 11:21:36 +0200
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-Neil Conway kirjutas E, 03.11.2003 kell 08:07:
-> A couple days ago, Manfred Spraul mentioned the posix_fadvise() API on
-> -hackers:
->
-> http://www.opengroup.org/onlinepubs/007904975/functions/posix_fadvise.html
->
-> I'm working on making use of posix_fadvise() where appropriate. I can
-> think of the following places where this would be useful:
->
-> (1) As Manfred originally noted, when we advance to a new XLOG segment,
-> we can use POSIX_FADV_DONTNEED to let the kernel know we won't be
-> accessing the old WAL segment anymore. I've attached a quick kludge of a
-> patch that implements this. I haven't done any benchmarking of it yet,
-> though (comments or benchmark results are welcome).
->
-> (2) ISTM that we can set POSIX_FADV_RANDOM for *all* indexes, since the
-> vast majority of the accesses to them shouldn't be sequential. Are there
-> any situations in which this assumption doesn't hold? (Perhaps B+-tree
-> bulk loading, or CLUSTER?) Should this be done per-index-AM, or
-> globally?
-
-Perhaps we could do it for all _leaf_ nodes, the root and intermediate
-nodes are usually better kept in cache.
-
-> (3) When doing VACUUM, ANALYZE, or large sequential scans (for some
-> reasonable definition of "large"), we can use POSIX_FADV_SEQUENTIAL.
-
-perhaps just sequential scans without "large" ?
-
-> (4) Various other components, such as tuplestore, tuplesort, and any
-> utility commands that need to scan through an entire user relation for
-> some reason. Once we've got the APIs for doing this worked out, it
-> should be relatively easy to add other uses of posix_fadvise().
->
-> (5) I'm hesitant to make use of POSIX_FADV_DONTNEED in VACUUM, as has
-> been suggested elsewhere. The problem is that it's all-or-nothing: if
-> the VACUUM happens to look at hot pages, these will be flushed from the
-> page cache, so the net result may be a loss.
-
-True. POSIX_FADV_DONTNEED should be only used if the page was retrieved
-by VACUUM.
-
-> So what API is desirable for uses 2-4? I'm thinking of adding a new
-> function to the smgr API, smgradvise(). Given a Relation and an advice,
-> this would:
->
-> (a) propagate the advice for this relation to all the open FDs for the
-> relation
->
-> (b) store the new advice somewhere so that new FDs for the relation can
-> have this advice set for them: clients should just be able to call
-> smgradvise() without needing to worry if someone else has already called
-> smgropen() for the relation in the past. One problem is how to store
-> this: I don't think it can be a field of RelationData, since that is
-> transient. Any suggestions?
-
-also, you may want to restore old FADV* after you are done - just
-running one seqscan should probably not leave the relation in
-POSIX_FADV_SEQUENTIAL mode forever.
-
-> Note that I'm assuming that we don't need to set advice on sub-sections
-> of a relation, although the posix_fadvise() API allows it -- does anyone
-> think that would be useful?
->
-> One potential issue is that when one process calls posix_fadvise() on a
-> particular FD, I'd expect that other processes accessing the same file
-> will be affected. For example, enabling FADV_SEQUENTIAL while we're
-> vacuuming a relation will mean that another client doing a concurrent
-> SELECT on the relation will see different readahead behavior. That
-> doesn't seem like a major problem though.
->
-> BTW, posix_fadvise() is currently only supported on Linux 2.6 w/ a
-> recent version of glibc (BSD hackers, if you're listening,
-> posix_fadvise() would be a very cool thing to have :P). So we'll need to
-> do the appropriate configure magic to ensure we only use it where its
-> available. Thankfully, it is a POSIX standard, so I would expect that in
-> the years to come it will be available on more platforms.
->
-> Any comments would be welcome.
->
-> -Neil
->
->
->
-> ---------------------------(end of broadcast)---------------------------
-> TIP 7: don't forget to increase your free space map settings
-
----------------------------(end of broadcast)---------------------------
-TIP 5: Have you checked our extensive FAQ?
-
- http://www.postgresql.org/docs/faqs/FAQ.html
-
-From pgsql-hackers-owner+M46361=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 12:20:11 2003
-Return-path: <pgsql-hackers-owner+M46361=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3HK8528457
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 12:20:10 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGfAs-0000gy-1V
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 07:55:18 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 82330D1B524
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 13:50:36 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 54341-10
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 09:50:08 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id 56261D1B57F
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 09:50:04 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP
- id 80F521DDE; Mon, 3 Nov 2003 08:50:00 -0500 (EST)
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: Hannu Krosing <hannu@tm.ee>
-cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <1067851295.2580.12.camel@fuji.krosing.net>
-References: <1067839664.3089.173.camel@tokyo>
- <1067851295.2580.12.camel@fuji.krosing.net>
-Content-Type: text/plain
-Message-ID: <1067867399.3089.219.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 08:50:00 -0500
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-On Mon, 2003-11-03 at 04:21, Hannu Krosing wrote:
-> Neil Conway kirjutas E, 03.11.2003 kell 08:07:
-> > (2) ISTM that we can set POSIX_FADV_RANDOM for *all* indexes, since the
-> > vast majority of the accesses to them shouldn't be sequential.
->
-> Perhaps we could do it for all _leaf_ nodes, the root and intermediate
-> nodes are usually better kept in cache.
-
-POSIX_FADV_RANDOM doesn't effect the page cache, it just determines how
-aggressive the kernel is when doing readahead (at least on Linux, but
-I'd expect to see other kernels implement similar behavior). In other
-words, using FADV_RANDOM shouldn't decrease the chance that interior
-B+-tree nodes are kept in the page cache.
-
-> True. POSIX_FADV_DONTNEED should be only used if the page was retrieved
-> by VACUUM.
-
-Right -- we'd like pages touched by VACUUM to be flushed from the page
-cache if that page wasn't previously in *either* the PostgreSQL buffer
-pool or the kernel's page cache. We can implement the former easily
-enough, but I don't see any feasible way to do the latter: on a high-end
-machine with gigabytes of RAM but a relatively small shared_buffers
-(which is the configuration we recommend), there may be plenty of hot
-pages that aren't in the PostgreSQL buffer pool but are in the page
-cache.
-
-> also, you may want to restore old FADV* after you are done - just
-> running one seqscan should probably not leave the relation in
-> POSIX_FADV_SEQUENTIAL mode forever.
-
-Right, I forgot to mention that. The API doesn't provide a means to get
-the current advice for an FD. So when we're finished doing whatever
-operation we set some advice for, we'll need to just reset the file to
-FADV_NORMAL and hope that it doesn't overrule some advise just set by
-someone else. Either that, or we can manually keep track of all the
-advise we're setting ourselves, but that seems a hassle.
-
--Neil
-
-
-
----------------------------(end of broadcast)---------------------------
-TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
-
-From pgsql-hackers-owner+M46362=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 11:38:34 2003
-Return-path: <pgsql-hackers-owner+M46362=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3GcW524671
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 11:38:33 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGfZS-0001Yo-Ot
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 08:20:42 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id E7744D1CA72
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 14:17:05 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 72987-02
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 10:16:37 -0400 (AST)
-Received: from mail.libertyrms.com (unknown [209.167.124.227])
- by svr1.postgresql.org (Postfix) with ESMTP id E0B34D1B57D
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 10:16:33 -0400 (AST)
-Received: from [10.1.2.130] (helo=dba2)
- by mail.libertyrms.com with esmtp (Exim 3.22 #3 (Debian))
- id 1AGfVW-00055W-00
- for <pgsql-hackers@postgresql.org>; Mon, 03 Nov 2003 09:16:38 -0500
-Received: by dba2 (Postfix, from userid 1019)
- id DCCBACD8C; Mon, 3 Nov 2003 09:16:37 -0500 (EST)
-Date: Mon, 3 Nov 2003 09:16:37 -0500
-From: Andrew Sullivan <andrew@libertyrms.info>
-To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-Message-ID: <20031103141637.GB12457@libertyrms.info>
-Mail-Followup-To: Andrew Sullivan <andrew@libertyrms.info>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-References: <1067839664.3089.173.camel@tokyo> <1067851295.2580.12.camel@fuji.krosing.net> <1067867399.3089.219.camel@tokyo>
-MIME-Version: 1.0
-Content-Type: text/plain; charset=us-ascii
-Content-Disposition: inline
-In-Reply-To: <1067867399.3089.219.camel@tokyo>
-User-Agent: Mutt/1.5.4i
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-On Mon, Nov 03, 2003 at 08:50:00AM -0500, Neil Conway wrote:
-
-> pool or the kernel's page cache. We can implement the former easily
-> enough, but I don't see any feasible way to do the latter: on a high-end
-> machine with gigabytes of RAM but a relatively small shared_buffers
-> (which is the configuration we recommend), there may be plenty of hot
-
-I wonder if the limitations that are on one's ability to evaluate
-effectively what is in the OS's filesystem cache is the real reason
-all those Other systems (of Databases, Big, too) have stayed with
-their old design of managing it all themselves (raw filesystems and
-all the buffering handled by the back end). Maybe that's not just an
-historical argument whereby they happen to have the code around.
-After all, it can't be cheap to maintain. Not that I'm advocating
-writing such a system -- I sure couldn't do the work, to begin with.
-
-A
-
-
---
-----
-Andrew Sullivan 204-4141 Yonge Street
-Afilias Canada Toronto, Ontario Canada
-<andrew@libertyrms.info> M2P 2A8
- +1 416 646 3304 x110
-
-
----------------------------(end of broadcast)---------------------------
-TIP 9: the planner will ignore your desire to choose an index scan if your
- joining column's datatypes do not match
-
-From pgsql-hackers-owner+M46363=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 12:41:32 2003
-Return-path: <pgsql-hackers-owner+M46363=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3HfU500821
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 12:41:31 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGfuP-0001rv-3W
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 08:42:21 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 77F1ED1CA56
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 14:38:57 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 73581-06
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 10:38:29 -0400 (AST)
-Received: from sss.pgh.pa.us (unknown [192.204.191.242])
- by svr1.postgresql.org (Postfix) with ESMTP id EE4C8D1B923
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 10:38:19 -0400 (AST)
-Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
- by sss.pgh.pa.us (8.12.10/8.12.10) with ESMTP id hA3EcO19013973;
- Mon, 3 Nov 2003 09:38:24 -0500 (EST)
-To: Neil Conway <neilc@samurai.com>
-cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-In-Reply-To: <1067839664.3089.173.camel@tokyo>
-References: <1067839664.3089.173.camel@tokyo>
-Comments: In-reply-to Neil Conway <neilc@samurai.com>
- message dated "Mon, 03 Nov 2003 01:07:45 -0500"
-Date: Mon, 03 Nov 2003 09:38:23 -0500
-Message-ID: <13972.1067870303@sss.pgh.pa.us>
-From: Tom Lane <tgl@sss.pgh.pa.us>
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-Neil Conway <neilc@samurai.com> writes:
-> So what API is desirable for uses 2-4? I'm thinking of adding a new
-> function to the smgr API, smgradvise().
-
-It's a little premature to be inventing APIs when you have no evidence
-that this will make any useful performance difference. I'd recommend a
-quick hack to get proof of concept before you bother with nice APIs.
-
-> Given a Relation and an advice, this would:
-> (a) propagate the advice for this relation to all the open FDs for the
-> relation
-
-"All"? You cannot affect the FDs being used by other backends. It's
-fairly unclear to me what the posix_fadvise function is really going
-to do for files that are being accessed by multiple processes. For
-instance, is there any value in setting POSIX_FADV_DONTNEED on a WAL
-file, given that every other backend is going to have that same file
-open? I would expect that rational kernel behavior would be to ignore
-this advice unless it's set by the last backend to have the file open
---- but I'm not sure we can synchronize the closing of old WAL segments
-well enough to know which backend is the last to close the file.
-
-A related problem is that the smgr uses the same FD to access the same
-relation no matter how many scans are in progress. Think about a
-complex query that is doing both a seqscan and an indexscan on the same
-relation (a self-join could easily do this). You'd really need to
-change this if you want POSIX_FADV_SEQUENTIAL and POSIX_FADV_RANDOM to
-get set usefully.
-
-In short I think you need to do some more thinking about what the scope
-of the advice flags is going to be ...
-
-> (b) store the new advice somewhere so that new FDs for the relation can
-> have this advice set for them: clients should just be able to call
-> smgradvise() without needing to worry if someone else has already called
-> smgropen() for the relation in the past. One problem is how to store
-> this: I don't think it can be a field of RelationData, since that is
-> transient. Any suggestions?
-
-Something Vadim had wanted to do for years is to decouple the smgr and
-lower levels from the existing Relation cache, and have a low-level
-notion of "open relation" that only requires having the "RelFileNode"
-value to open it. This would allow eliminating the concept of blind
-write, which would be a Very Good Thing. It would make sense to
-associate the advice setting with such low-level relations. One
-possible way to handle the multiple-scan issue is to make the desired
-advice part of the low-level open() call, so that you actually have
-different low-level relations for seq and random access to a relation.
-Not sure if this works cleanly when you take into account issues like
-smgrunlink, but it's something to think about.
-
- regards, tom lane
-
----------------------------(end of broadcast)---------------------------
-TIP 2: you can get off all lists at once with the unregister command
- (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
-
-From pgsql-hackers-owner+M46366=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 16:16:06 2003
-Return-path: <pgsql-hackers-owner+M46366=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3LG4520809
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 16:16:05 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGgI8-0002Y4-7P
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 09:06:52 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id B00B3D1CA79
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 15:02:23 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 75791-08
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 11:01:55 -0400 (AST)
-Received: from sss.pgh.pa.us (unknown [192.204.191.242])
- by svr1.postgresql.org (Postfix) with ESMTP id 65D22D1CAFC
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 11:01:51 -0400 (AST)
-Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
- by sss.pgh.pa.us (8.12.10/8.12.10) with ESMTP id hA3F1t19014250;
- Mon, 3 Nov 2003 10:01:55 -0500 (EST)
-To: Neil Conway <neilc@samurai.com>
-cc: Hannu Krosing <hannu@tm.ee>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-In-Reply-To: <1067867399.3089.219.camel@tokyo>
-References: <1067839664.3089.173.camel@tokyo> <1067851295.2580.12.camel@fuji.krosing.net> <1067867399.3089.219.camel@tokyo>
-Comments: In-reply-to Neil Conway <neilc@samurai.com>
- message dated "Mon, 03 Nov 2003 08:50:00 -0500"
-Date: Mon, 03 Nov 2003 10:01:55 -0500
-Message-ID: <14249.1067871715@sss.pgh.pa.us>
-From: Tom Lane <tgl@sss.pgh.pa.us>
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-Neil Conway <neilc@samurai.com> writes:
-> POSIX_FADV_RANDOM doesn't effect the page cache, it just determines how
-> aggressive the kernel is when doing readahead (at least on Linux, but
-> I'd expect to see other kernels implement similar behavior).
-
-I would expect POSIX_FADV_SEQUENTIAL to reduce the chance that a page
-will be kept in buffer cache after it's been used.
-
- regards, tom lane
-
----------------------------(end of broadcast)---------------------------
-TIP 5: Have you checked our extensive FAQ?
-
- http://www.postgresql.org/docs/faqs/FAQ.html
-
-From pgsql-hackers-owner+M46367=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 11:29:59 2003
-Return-path: <pgsql-hackers-owner+M46367=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3GTw523888
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 11:29:59 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGgzl-0003cP-FZ
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 09:51:57 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 891D0D1CB32
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 15:45:26 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 85721-04
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 11:44:59 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id 0A282D1CB2C
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 11:44:55 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP
- id 235771FAE; Mon, 3 Nov 2003 10:44:44 -0500 (EST)
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: Tom Lane <tgl@sss.pgh.pa.us>
-cc: Hannu Krosing <hannu@tm.ee>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <14249.1067871715@sss.pgh.pa.us>
-References: <1067839664.3089.173.camel@tokyo>
- <1067851295.2580.12.camel@fuji.krosing.net>
- <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us>
-Content-Type: text/plain
-Message-ID: <1067874283.3089.241.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 10:44:43 -0500
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-On Mon, 2003-11-03 at 10:01, Tom Lane wrote:
-> Neil Conway <neilc@samurai.com> writes:
-> > POSIX_FADV_RANDOM doesn't effect the page cache, it just determines how
-> > aggressive the kernel is when doing readahead (at least on Linux, but
-> > I'd expect to see other kernels implement similar behavior).
->
-> I would expect POSIX_FADV_SEQUENTIAL to reduce the chance that a page
-> will be kept in buffer cache after it's been used.
-
-I don't think that can be reasonably implied from the POSIX text, which
-is merely:
-
-POSIX_FADV_SEQUENTIAL
- Specifies that the application expects to access the specified
- data sequentially from lower offsets to higher offsets.
-
-The present Linux implementation doesn't do this, AFAICS -- all it does
-it increase the readahead for this file:
-
- http://lxr.linux.no/source/mm/fadvise.c?v=2.6.0-test7
-
--Neil
-
-
-
----------------------------(end of broadcast)---------------------------
-TIP 9: the planner will ignore your desire to choose an index scan if your
- joining column's datatypes do not match
-
-From pgsql-hackers-owner+M46369=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 11:17:50 2003
-Return-path: <pgsql-hackers-owner+M46369=pgman=candle.pha.pa.us@postgresql.org>
-Received: from hosting.commandprompt.com (222.commandprompt.com [207.173.200.222])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3GHm522584
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 11:17:49 -0500 (EST)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by hosting.commandprompt.com (8.11.6/8.11.6) with ESMTP id hA3GHYm21291
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 08:17:45 -0800
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id CC4D5D1CB1B
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 16:12:10 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 87278-03
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 12:11:39 -0400 (AST)
-Received: from sss.pgh.pa.us (unknown [192.204.191.242])
- by svr1.postgresql.org (Postfix) with ESMTP id 5B0AED1B56D
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 12:11:37 -0400 (AST)
-Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
- by sss.pgh.pa.us (8.12.10/8.12.10) with ESMTP id hA3GBa19024628;
- Mon, 3 Nov 2003 11:11:36 -0500 (EST)
-To: Neil Conway <neilc@samurai.com>
-cc: Hannu Krosing <hannu@tm.ee>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-In-Reply-To: <1067874283.3089.241.camel@tokyo>
-References: <1067839664.3089.173.camel@tokyo> <1067851295.2580.12.camel@fuji.krosing.net> <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us> <1067874283.3089.241.camel@tokyo>
-Comments: In-reply-to Neil Conway <neilc@samurai.com>
- message dated "Mon, 03 Nov 2003 10:44:43 -0500"
-Date: Mon, 03 Nov 2003 11:11:36 -0500
-Message-ID: <24627.1067875896@sss.pgh.pa.us>
-From: Tom Lane <tgl@sss.pgh.pa.us>
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-Neil Conway <neilc@samurai.com> writes:
-> On Mon, 2003-11-03 at 10:01, Tom Lane wrote:
->> I would expect POSIX_FADV_SEQUENTIAL to reduce the chance that a page
->> will be kept in buffer cache after it's been used.
-
-> I don't think that can be reasonably implied from the POSIX text, which
-> is merely:
-
-> POSIX_FADV_SEQUENTIAL
-> Specifies that the application expects to access the specified
-> data sequentially from lower offsets to higher offsets.
-
-Why not? The advice says that you're going to access the data
-sequentially in the forward direction. If you're not going to back up,
-there is no point in keeping pages in cache after they've been read.
-
-A reasonable implementation of the POSIX semantics would need to balance
-this consideration against the likelihood that some other process would
-want to access some of these pages later. But I would certainly expect
-it to reduce the probability of keeping the pages in cache.
-
-> The present Linux implementation doesn't do this, AFAICS --
-
-So it only does part of what it could do. No surprise...
-
- regards, tom lane
-
----------------------------(end of broadcast)---------------------------
-TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
-
-From pgsql-hackers-owner+M46371=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 17:03:33 2003
-Return-path: <pgsql-hackers-owner+M46371=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3M3V525067
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 17:03:32 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGi7n-00058i-6q
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 11:04:19 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id A01ADD1CAC3
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 16:59:59 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 95778-05
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 12:59:28 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id 88C61D1CAC1
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 12:59:27 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP
- id BC4611FA6; Mon, 3 Nov 2003 11:59:25 -0500 (EST)
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: Tom Lane <tgl@sss.pgh.pa.us>
-cc: Hannu Krosing <hannu@tm.ee>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <24627.1067875896@sss.pgh.pa.us>
-References: <1067839664.3089.173.camel@tokyo>
- <1067851295.2580.12.camel@fuji.krosing.net>
- <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us>
- <1067874283.3089.241.camel@tokyo> <24627.1067875896@sss.pgh.pa.us>
-Content-Type: text/plain
-Message-ID: <1067878764.3089.369.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 11:59:24 -0500
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-On Mon, 2003-11-03 at 11:11, Tom Lane wrote:
-> Why not? The advice says that you're going to access the data
-> sequentially in the forward direction. If you're not going to back up,
-> there is no point in keeping pages in cache after they've been read.
-
-The advice says: "I'm going to read this data sequentially, going
-forward." It doesn't say: "I'm only going to read the data once, and
-then not access it again" (ISTM that's what FADV_NOREUSE is for). For
-example, the following is a perfectly reasonable sequential access
-pattern:
-
- a,b,c,a,b,c,a,b,c,a,b,c
-
-(i.e. repeatedly scanning through a large file, say for a data-analysis
-app that does multiple passes over the input data). It might not be a
-particularly common database reference pattern, but just because an app
-is doing a sequential read says little about the temporal locality of
-references to the pages in question.
-
--Neil
-
-
-
----------------------------(end of broadcast)---------------------------
-TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
-
-From pgsql-hackers-owner+M46373=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 12:24:42 2003
-Return-path: <pgsql-hackers-owner+M46373=pgman=candle.pha.pa.us@postgresql.org>
-Received: from hosting.commandprompt.com (222.commandprompt.com [207.173.200.222])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3HOd529168
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 12:24:40 -0500 (EST)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by hosting.commandprompt.com (8.11.6/8.11.6) with ESMTP id hA3HOBm27594
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 09:24:35 -0800
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 13798D1B557
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 17:18:13 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 05139-02
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 13:17:42 -0400 (AST)
-Received: from fuji.krosing.net (silmet.estpak.ee [194.126.97.78])
- by svr1.postgresql.org (Postfix) with ESMTP id A1A62D1B4FE
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 13:17:40 -0400 (AST)
-Received: from fuji.krosing.net (localhost.localdomain [127.0.0.1])
- by fuji.krosing.net (8.12.8/8.12.8) with ESMTP id hA3HHerb002608;
- Mon, 3 Nov 2003 19:17:40 +0200
-Received: (from hannu@localhost)
- by fuji.krosing.net (8.12.8/8.12.8/Submit) id hA3HHehZ002606;
- Mon, 3 Nov 2003 19:17:40 +0200
-X-Authentication-Warning: fuji.krosing.net: hannu set sender to hannu@tm.ee using -f
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Hannu Krosing <hannu@tm.ee>
-To: Neil Conway <neilc@samurai.com>
-cc: Tom Lane <tgl@sss.pgh.pa.us>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <1067878764.3089.369.camel@tokyo>
-References: <1067839664.3089.173.camel@tokyo>
- <1067851295.2580.12.camel@fuji.krosing.net>
- <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us>
- <1067874283.3089.241.camel@tokyo> <24627.1067875896@sss.pgh.pa.us>
- <1067878764.3089.369.camel@tokyo>
-Content-Type: text/plain
-Content-Transfer-Encoding: 7bit
-Message-ID: <1067879859.2414.27.camel@fuji.krosing.net>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 19:17:40 +0200
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-Neil Conway kirjutas E, 03.11.2003 kell 18:59:
-> On Mon, 2003-11-03 at 11:11, Tom Lane wrote:
-> > Why not? The advice says that you're going to access the data
-> > sequentially in the forward direction. If you're not going to back up,
-> > there is no point in keeping pages in cache after they've been read.
->
-> The advice says: "I'm going to read this data sequentially, going
-> forward." It doesn't say: "I'm only going to read the data once, and
-> then not access it again" (ISTM that's what FADV_NOREUSE is for).
-
-They seem like independent features.
-
-Can you use combinations like ( FADV_NOREUS | FADV_SEQUENTIAL )
-
-(I obviously have'nt read the spec)
-
-----------------
-Hannu
-
-
----------------------------(end of broadcast)---------------------------
-TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
-
-From pgsql-hackers-owner+M46376=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 14:03:58 2003
-Return-path: <pgsql-hackers-owner+M46376=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3J3t508443
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 14:03:56 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGjpp-0007xC-6K
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 12:53:53 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 84EB3D1CAF9
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 18:47:18 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 15987-04
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 14:46:47 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id B951FD1B53F
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 14:46:46 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP
- id E47C61FB5; Mon, 3 Nov 2003 13:46:46 -0500 (EST)
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: Hannu Krosing <hannu@tm.ee>
-cc: Tom Lane <tgl@sss.pgh.pa.us>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <1067879859.2414.27.camel@fuji.krosing.net>
-References: <1067839664.3089.173.camel@tokyo>
- <1067851295.2580.12.camel@fuji.krosing.net>
- <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us>
- <1067874283.3089.241.camel@tokyo> <24627.1067875896@sss.pgh.pa.us>
- <1067878764.3089.369.camel@tokyo>
- <1067879859.2414.27.camel@fuji.krosing.net>
-Content-Type: text/plain
-Message-ID: <1067885206.3089.476.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 13:46:46 -0500
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-On Mon, 2003-11-03 at 12:17, Hannu Krosing wrote:
-> Can you use combinations like ( FADV_NOREUS | FADV_SEQUENTIAL )
-
-You can do an fadvise() for FADV_SEQUENTIAL, and then another fadvise()
-for FADV_NOREUSE.
-
--Neil
-
-
-
----------------------------(end of broadcast)---------------------------
-TIP 4: Don't 'kill -9' the postmaster
-
-From pgsql-hackers-owner+M46378=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 14:32:05 2003
-Return-path: <pgsql-hackers-owner+M46378=pgman=candle.pha.pa.us@postgresql.org>
-Received: from hosting.commandprompt.com (222.commandprompt.com [207.173.200.222])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3JW3511090
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 14:32:04 -0500 (EST)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by hosting.commandprompt.com (8.11.6/8.11.6) with ESMTP id hA3JVYm07352
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 11:31:53 -0800
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id E5BF3D1B541
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 19:26:06 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 17405-10
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 15:25:37 -0400 (AST)
-Received: from dbl.q-ag.de (dbl.q-ag.de [80.146.160.66])
- by svr1.postgresql.org (Postfix) with ESMTP id A9477D1B908
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 15:25:29 -0400 (AST)
-Received: from colorfullife.com (dbl [127.0.0.1])
- by dbl.q-ag.de (8.12.3/8.12.3/Debian-6.6) with ESMTP id hA3JP0N9002667;
- Mon, 3 Nov 2003 20:25:01 +0100
-Message-ID: <3FA6AB8B.8060902@colorfullife.com>
-Date: Mon, 03 Nov 2003 20:24:59 +0100
-From: Manfred Spraul <manfred@colorfullife.com>
-User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030701
-X-Accept-Language: en-us, en
-MIME-Version: 1.0
-To: Neil Conway <neilc@samurai.com>
-cc: Tom Lane <tgl@sss.pgh.pa.us>, Hannu Krosing <hannu@tm.ee>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-References: <1067839664.3089.173.camel@tokyo> <1067851295.2580.12.camel@fuji.krosing.net> <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us> <1067874283.3089.241.camel@tokyo>
-In-Reply-To: <1067874283.3089.241.camel@tokyo>
-Content-Type: text/plain; charset=ISO-8859-1; format=flowed
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-Status: OR
-
-Neil Conway wrote:
-
->The present Linux implementation doesn't do this, AFAICS -- all it does
->it increase the readahead for this file:
->
->
-AFAIK Linux uses a modified LRU that automatically puts pages that were
-touched only once at a lower priority than frequently accessed pages.
-
-Neil: what about calling posix_fadvise for the whole file immediately
-after issue_xlog_fsync() in XLogWrite? According to the comment, it's
-guaranteed that this will happen only once.
-Or: add an posix_fadvise into issue_xlog_fsync(), for the range just
-sync'ed.
-
-Btw, how much xlog traffic does a busy postgres site generate?
-
---
- Manfred
-
-
----------------------------(end of broadcast)---------------------------
-TIP 9: the planner will ignore your desire to choose an index scan if your
- joining column's datatypes do not match
-
-From pgsql-hackers-owner+M46381=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 21:41:18 2003
-Return-path: <pgsql-hackers-owner+M46381=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA42fG527858
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 21:41:17 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGl6T-0001bk-22
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 14:15:09 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 444D7D1B541
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 20:11:01 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 35524-02
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 16:10:31 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id 602CDD1CA8E
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 16:10:29 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP
- id 87D611FC7; Mon, 3 Nov 2003 15:10:29 -0500 (EST)
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: Tom Lane <tgl@sss.pgh.pa.us>
-cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <13972.1067870303@sss.pgh.pa.us>
-References: <1067839664.3089.173.camel@tokyo>
- <13972.1067870303@sss.pgh.pa.us>
-Content-Type: text/plain
-Message-ID: <1067890228.3089.532.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 15:10:29 -0500
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-On Mon, 2003-11-03 at 09:38, Tom Lane wrote:
-> Neil Conway <neilc@samurai.com> writes:
-> > Given a Relation and an advice, this would:
-> > (a) propagate the advice for this relation to all the open FDs for the
-> > relation
->
-> "All"? You cannot affect the FDs being used by other backends.
-
-Sorry, I meant just the FDs opened by this backend.
-
-> It's fairly unclear to me what the posix_fadvise function is really
-> going to do for files that are being accessed by multiple processes.
-
-In a thread on lkml[1], Andrew Morton comments:
-
- Note that it applies to a file descriptor. If
- posix_fadvise(FADV_DONTNEED) is called against a file
- descriptor, and someone else has an fd open against the same
- file, that other user gets their foot shot off. That's OK.
-
-I would imagine that by "getting their foot" shot off, Andrew is saying
-that FADV_DONTNEED by one process affects any other processes accessing
-the same file via a different FD. If I'm misunderstanding what's going
-on here, please let me know.
-
-> For instance, is there any value in setting POSIX_FADV_DONTNEED on a
-> WAL file, given that every other backend is going to have that same
-> file open?
-
-My understanding is that yes, there is value in doing this, for the
-reasons mentioned above.
-
-> A related problem is that the smgr uses the same FD to access the same
-> relation no matter how many scans are in progress.
-
-Interesting ... I'll have to think some more about this. Thanks for the
-suggestions and comments.
-
--Neil
-
-[1] - http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/0361.html
-
-The rest of the thread includes an interesting discussion -- I recommend
-reading it. The lkml folks actually speculate about what we (OSS DBMS
-developers) would find useful in fadvise(), amusingly enough... The
-thread starts here:
-
-http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/0230.html
-
-Finally, Andrew Morton provides some more clarification on what happens
-when multiple processes are accessing a file that is fadvise()'d:
-
-http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/0476.html
-
-
-
----------------------------(end of broadcast)---------------------------
-TIP 4: Don't 'kill -9' the postmaster
-
-From pgsql-hackers-owner+M46385=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 17:57:53 2003
-Return-path: <pgsql-hackers-owner+M46385=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA3Mvp502402
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 17:57:52 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGlJr-0002JW-HW
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 14:28:59 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 761FFD1B541
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 20:23:41 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 35080-07
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 16:23:11 -0400 (AST)
-Received: from bob.samurai.com (bob.samurai.com [205.207.28.75])
- by svr1.postgresql.org (Postfix) with ESMTP id 56553D1B8E4
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 16:23:09 -0400 (AST)
-Received: from 6-allhosts (d226-89-59.home.cgocable.net [24.226.89.59])
- by bob.samurai.com (Postfix) with ESMTP
- id 36EBC1F7A; Mon, 3 Nov 2003 15:23:10 -0500 (EST)
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-From: Neil Conway <neilc@samurai.com>
-To: Manfred Spraul <manfred@colorfullife.com>
-cc: Tom Lane <tgl@sss.pgh.pa.us>, Hannu Krosing <hannu@tm.ee>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-In-Reply-To: <3FA6AB8B.8060902@colorfullife.com>
-References: <1067839664.3089.173.camel@tokyo>
- <1067851295.2580.12.camel@fuji.krosing.net>
- <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us>
- <1067874283.3089.241.camel@tokyo> <3FA6AB8B.8060902@colorfullife.com>
-Content-Type: text/plain
-Message-ID: <1067890989.3089.540.camel@tokyo>
-MIME-Version: 1.0
-X-Mailer: Ximian Evolution 1.4.5
-Date: Mon, 03 Nov 2003 15:23:09 -0500
-Content-Transfer-Encoding: 7bit
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-On Mon, 2003-11-03 at 14:24, Manfred Spraul wrote:
-> Neil: what about calling posix_fadvise for the whole file immediately
-> after issue_xlog_fsync() in XLogWrite? According to the comment, it's
-> guaranteed that this will happen only once.
-> Or: add an posix_fadvise into issue_xlog_fsync(), for the range just
-> sync'ed.
-
-I'll try those, in case it makes any difference. My guess/hope is that
-it won't (as mentioned earlier), but we'll see.
-
-> Btw, how much xlog traffic does a busy postgres site generate?
-
-No idea. Can anyone recommend what kind of benchmark would be be
-appropriate?
-
--Neil
-
-
-
----------------------------(end of broadcast)---------------------------
-TIP 5: Have you checked our extensive FAQ?
-
- http://www.postgresql.org/docs/faqs/FAQ.html
-
-From pgsql-hackers-owner+M46392=pgman=candle.pha.pa.us@postgresql.org Mon Nov 3 23:04:29 2003
-Return-path: <pgsql-hackers-owner+M46392=pgman=candle.pha.pa.us@postgresql.org>
-Received: from noon.pghoster.com ([64.246.0.64])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id hA444O504242
- for <pgman@candle.pha.pa.us>; Mon, 3 Nov 2003 23:04:28 -0500 (EST)
-Received: from svr1.postgresql.org ([200.46.204.71] helo=postgresql.org)
- by noon.pghoster.com with esmtp (Exim 4.20)
- id 1AGoLI-0007lm-9Z
- for pgman@candle.pha.pa.us; Mon, 03 Nov 2003 17:42:40 -0600
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.2])
- by svr1.postgresql.org (Postfix) with ESMTP id 2A5ADD1CA7C
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Mon, 3 Nov 2003 23:38:33 +0000 (GMT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024)
- with ESMTP id 67058-04
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Mon, 3 Nov 2003 19:38:04 -0400 (AST)
-Received: from sss.pgh.pa.us (unknown [192.204.191.242])
- by svr1.postgresql.org (Postfix) with ESMTP id 157C4D1B914
- for <pgsql-hackers@postgresql.org>; Mon, 3 Nov 2003 19:38:01 -0400 (AST)
-Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
- by sss.pgh.pa.us (8.12.10/8.12.10) with ESMTP id hA3Nc119013157;
- Mon, 3 Nov 2003 18:38:01 -0500 (EST)
-To: Neil Conway <neilc@samurai.com>
-cc: Hannu Krosing <hannu@tm.ee>,
- PostgreSQL Hackers <pgsql-hackers@postgresql.org>
-Subject: Re: [HACKERS] adding support for posix_fadvise()
-In-Reply-To: <1067878764.3089.369.camel@tokyo>
-References: <1067839664.3089.173.camel@tokyo> <1067851295.2580.12.camel@fuji.krosing.net> <1067867399.3089.219.camel@tokyo> <14249.1067871715@sss.pgh.pa.us> <1067874283.3089.241.camel@tokyo> <24627.1067875896@sss.pgh.pa.us> <1067878764.3089.369.camel@tokyo>
-Comments: In-reply-to Neil Conway <neilc@samurai.com>
- message dated "Mon, 03 Nov 2003 11:59:24 -0500"
-Date: Mon, 03 Nov 2003 18:38:01 -0500
-Message-ID: <13156.1067902681@sss.pgh.pa.us>
-From: Tom Lane <tgl@sss.pgh.pa.us>
-X-Virus-Scanned: by amavisd-new at postgresql.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
-X-AntiAbuse: Primary Hostname - noon.pghoster.com
-X-AntiAbuse: Original Domain - candle.pha.pa.us
-X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
-X-AntiAbuse: Sender Address Domain - postgresql.org
-Status: OR
-
-Neil Conway <neilc@samurai.com> writes:
-> On Mon, 2003-11-03 at 11:11, Tom Lane wrote:
->> Why not? The advice says that you're going to access the data
->> sequentially in the forward direction. If you're not going to back up,
->> there is no point in keeping pages in cache after they've been read.
-
-> The advice says: "I'm going to read this data sequentially, going
-> forward." It doesn't say: "I'm only going to read the data once, and
-> then not access it again" (ISTM that's what FADV_NOREUSE is for).
-
-I'd believe that interpretation if the spec specifically allowed for
-applying multiple "advice" values to the same fd. However, given the
-way the API is written, it sure looks like the intention is that only
-the most recent advice value is valid for any one (portion of a) file.
-If the intention was that you could specify both FADV_SEQUENTIAL and
-FADV_NOREUSE, the usual Unix-y way to handle it would have been to
-define these constants as bit mask values and specify that the parameter
-to the syscall is a bitwise OR of multiple flags. The way you are
-interpreting it, there is no way to cancel an FADV_NOREUSE setting,
-since there is no value that is the opposite setting.
-
- regards, tom lane
-
----------------------------(end of broadcast)---------------------------
-TIP 6: Have you searched our list archives?
-
- http://archives.postgresql.org
-
projects / postgresql / commitdiff