--- /dev/null
+From pgsql-performance-owner+M3897@postgresql.org Sat Oct 4 19:50:57 2003
+Return-path: <pgsql-performance-owner+M3897@postgresql.org>
+Received: from svr5.postgresql.org (svr5.postgresql.org [64.117.225.181])
+ by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id h94NotQ08911
+ for <pgman@candle.pha.pa.us>; Sat, 4 Oct 2003 19:50:56 -0400 (EDT)
+Received: from postgresql.org (svr1.postgresql.org [64.117.224.193])
+ by svr5.postgresql.org (Postfix) with ESMTP
+ id DB0F072DC9E; Sat, 4 Oct 2003 23:50:50 +0000 (GMT)
+X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org
+Received: from localhost (unknown [64.117.224.130])
+ by svr1.postgresql.org (Postfix) with ESMTP id 70DDDD1B4EC
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>; Sat, 4 Oct 2003 23:50:42 +0000 (GMT)
+Received: from svr1.postgresql.org ([64.117.224.193])
+ by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024)
+ with ESMTP id 14368-03
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>;
+ Sat, 4 Oct 2003 20:49:56 -0300 (ADT)
+Received: from news.hub.org (unknown [64.117.224.194])
+ by svr1.postgresql.org (Postfix) with ESMTP id 2FBF7D1B4F0
+ for <pgsql-performance@postgresql.org>; Sat, 4 Oct 2003 20:49:53 -0300 (ADT)
+Received: from news.hub.org (host-64-117-224-194.altec1.com [64.117.224.194] (may be forged))
+ by news.hub.org (8.12.9/8.12.9) with ESMTP id h94NnqQh076664
+ for <pgsql-performance@postgresql.org>; Sat, 4 Oct 2003 23:49:52 GMT
+ (envelope-from news@news.hub.org)
+Received: (from news@localhost)
+ by news.hub.org (8.12.9/8.12.9/Submit) id h94NaQEP075478
+ for pgsql-performance@postgresql.org; Sat, 4 Oct 2003 23:36:26 GMT
+From: Christopher Browne <cbbrowne@acm.org>
+X-Newsgroups: comp.databases.postgresql.performance
+Subject: Re: [PERFORM] count(*) slow on large tables
+Date: Sat, 04 Oct 2003 19:33:46 -0400
+Organization: cbbrowne Computing Inc
+Lines: 77
+Message-ID: <m3u16ovaqt.fsf@wolfe.cbbrowne.com>
+References: <200310041556.h94Fuek24423@candle.pha.pa.us> <6743.1065286173@sss.pgh.pa.us>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+X-Complaints-To: usenet@news.hub.org
+X-message-flag: Outlook is rather hackable, isn't it?
+X-Home-Page: http://www.cbbrowne.com/info/
+X-Affero: http://svcs.affero.net/rm.php?r=cbbrowne
+User-Agent: Gnus/5.1003 (Gnus v5.10.3) XEmacs/21.4 (Reasonable Discussion, linux)
+Cancel-Lock: sha1:lLXE17xNVoXrMYZPn8CzzK9g1mc=
+To: pgsql-performance@postgresql.org
+X-Virus-Scanned: by amavisd-new at postgresql.org
+X-Mailing-List: pgsql-performance
+Precedence: bulk
+Sender: pgsql-performance-owner@postgresql.org
+Status: OR
+
+Quoth tgl@sss.pgh.pa.us (Tom Lane):
+> Bruce Momjian <pgman@candle.pha.pa.us> writes:
+>> We do have a TODO item:
+>> * Consider using MVCC to cache count(*) queries with no WHERE clause
+>
+>> The idea is to cache a recent count of the table, then have
+>> insert/delete add +/- records to the count. A COUNT(*) would get the
+>> main cached record plus any visible +/- records. This would allow the
+>> count to return the proper value depending on the visibility of the
+>> requesting transaction, and it would require _no_ heap or index scan.
+>
+> ... and it would give the wrong answers. Unless the cache is somehow
+> snapshot-aware, so that it can know which other transactions should be
+> included in your count.
+
+[That's an excellent summary that Bruce did of what came out of the
+previous discussion...]
+
+If this "cache" was a table, itself, the visibility of its records
+should be identical to that of the visibility of the "real" records.
++/- records would become visible when the transaction COMMITed, at the
+very same time the source records became visible.
+
+I thought, at one point, that it would be a slick idea for "record
+compression" to take place automatically; when you do a COUNT(*), the
+process would include compressing multiple records down to one.
+Unfortunately, that turns out to be Tremendously Evil if the same
+COUNT(*) were being concurrently processed in multiple transactions.
+Both would repeat much the same work, and this would ultimately lead
+to one of the transactions aborting. [I recently saw this effect
+occur, um, a few times...]
+
+For this not to have Evil Effects on unsuspecting transactions, we
+would instead require some process analagous to VACUUM, where a single
+transaction would be used to compress the "counts table" down to one
+record per table. Being independent of "user transactions," it could
+safely compress the data without injuring unsuspecting transactions.
+
+But in most cases, the cost of this would be pretty prohibitive.
+Every transaction that adds a record to a table leads to a record
+being added to table "pg_exact_row_counts". If transactions typically
+involve adding ONE row to any given table, this effectively doubles
+the update traffic. Ouch. That means that in a _real_
+implementation, it would make sense to pick and choose the tables that
+would be so managed.
+
+In my earlier arguing of "You don't really want that!", while I may
+have been guilty of engaging in a _little_ hyperbole, I was certainly
+_not_ being facetious overall. At work, we tell the developers "avoid
+doing COUNT(*) inside ordinary transactions!", and that is certainly
+NOT facetious comment. I recall a case a while back where system
+performance was getting brutalized by a lurking COUNT(*). (Combined
+with some other pathological behaviour, of course!) And note that
+this wasn't a query that the TODO item could address; it was of the
+form "SELECT COUNT(*) FROM SOME_TABLE WHERE OWNER = VALUE;"
+
+As you have commented elsewhere in the thread, much of the time, the
+point of asking for COUNT(*) is often to get some idea of table size,
+where the precise number isn't terribly important in comparison with
+getting general magnitude. Improving the ability to get approximate
+values would be of some value.
+
+I would further argue that "SELECT COUNT(*) FROM TABLE" isn't
+particularly useful even when precision _is_ important. If I'm
+working on reports that would be used to reconcile things, the queries
+I use are a whole lot more involved than that simple form. It is far
+more likely that I'm using a GROUP BY.
+
+It is legitimate to get wishful and imagine that it would be nice if
+we could get the value of that query "instantaneously." It is also
+legitimate to think that the effort required to implement that might
+be better used on improving other things.
+--
+(reverse (concatenate 'string "ac.notelrac.teneerf" "@" "454aa"))
+http://www3.sympatico.ca/cbbrowne/
+"very few people approach me in real life and insist on proving they
+are drooling idiots." -- Erik Naggum, comp.lang.lisp
+
+---------------------------(end of broadcast)---------------------------
+TIP 5: Have you checked our extensive FAQ?
+
+ http://www.postgresql.org/docs/faqs/FAQ.html
+
+From josh@agliodbs.com Sun Oct 5 14:59:07 2003
+Return-path: <josh@agliodbs.com>
+Received: from davinci.ethosmedia.com (vista1-228.percepticon.net [209.128.84.228] (may be forged))
+ by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id h95Ix5Q17861
+ for <pgman@candle.pha.pa.us>; Sun, 5 Oct 2003 14:59:06 -0400 (EDT)
+Received: from [63.195.55.98] (HELO spooky)
+ by davinci.ethosmedia.com (CommuniGate Pro SMTP 4.0.2)
+ with ESMTP id 3728969; Sun, 05 Oct 2003 11:59:26 -0700
+Content-Type: text/plain;
+ charset="iso-8859-1"
+From: Josh Berkus <josh@agliodbs.com>
+Organization: Aglio Database Solutions
+To: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
+Subject: Re: [PERFORM] count(*) slow on large tables
+Date: Sun, 5 Oct 2003 11:57:21 -0700
+User-Agent: KMail/1.4.3
+cc: Christopher Browne <cbbrowne@libertyrms.info>,
+ pgsql-performance@postgresql.org
+References: <200310041819.h94IJkV07596@candle.pha.pa.us>
+In-Reply-To: <200310041819.h94IJkV07596@candle.pha.pa.us>
+MIME-Version: 1.0
+Content-Transfer-Encoding: 8bit
+Message-ID: <200310051157.21555.josh@agliodbs.com>
+Status: OR
+
+Bruce,
+
+> OK, I beefed up the TODO:
+>
+> * Use a fixed row count and a +/- count with MVCC visibility rules
+> to allow fast COUNT(*) queries with no WHERE clause(?)
+>
+> I can always give the details if someone asks. It doesn't seem complex
+> enough for a separate TODO.detail item.
+
+Hmmm ... this doesn't seem effort-worthy to me. How often does anyone do
+COUNT with no where clause, except GUIs that give you a record count? (of
+course, as always, if someone wants to code it, feel free ...)
+
+And for those GUIs, wouldn't it be 97% as good to run an ANALYZE and give the
+approximate record counts for large tables?
+
+As for counts with a WHERE clause, this is obviously up to the user. Joe
+Conway and I tested using a C trigger to track some COUNT ... GROUP BY values
+for large tables based on additive numbers. It worked fairly well for
+accuracy, but the performance penalty on data writes was significant ... 8%
+to 25% penalty for UPDATES, depending on the frequency and batch size (>
+frequency > batch size --> > penalty)
+
+It's possible that this could be improved through some mechanism more tightly
+integrated with the source code. However,the coding effort would be
+significant ( 12-20 hours ) and it's possible that there would be no
+improvement, which is why we didn't do it.
+
+We also discussed an asynchronous aggregates collector that would work
+something like the statistics collector, and keep pre-programmmed aggregate
+data, updating during "low-activity" periods. This would significantly
+reduce the performance penalty, but at the cost of accuracy ... that is, a
+1%-5% variance on high-activity tables would be unavoidable, and all cached
+aggregates would have to be recalculated on database restart, significantly
+slowing down startup. Again, we felt that the effort-result payoff was not
+worthwhile.
+
+Overall, I think the stuff we already have planned ... the hash aggregates in
+7.4 and Tom's suggestion of adding an indexable flag to pg_aggs ... are far
+more likely to yeild useful fruit than any caching plan.
+
+--
+Josh Berkus
+Aglio Database Solutions
+San Francisco
+
+From pgsql-performance-owner+M3915@postgresql.org Mon Oct 6 02:08:33 2003
+Return-path: <pgsql-performance-owner+M3915@postgresql.org>
+Received: from svr5.postgresql.org (svr5.postgresql.org [64.117.225.181])
+ by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id h9668VQ15914
+ for <pgman@candle.pha.pa.us>; Mon, 6 Oct 2003 02:08:32 -0400 (EDT)
+Received: from postgresql.org (svr1.postgresql.org [64.117.224.193])
+ by svr5.postgresql.org (Postfix) with ESMTP
+ id DC70672E71E; Mon, 6 Oct 2003 06:08:24 +0000 (GMT)
+X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org
+Received: from localhost (unknown [64.117.224.130])
+ by svr1.postgresql.org (Postfix) with ESMTP id BFA49D1B4F6
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>; Mon, 6 Oct 2003 06:07:33 +0000 (GMT)
+Received: from svr1.postgresql.org ([64.117.224.193])
+ by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024)
+ with ESMTP id 90800-06
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>;
+ Mon, 6 Oct 2003 03:06:44 -0300 (ADT)
+Received: from smtp.pspl.co.in (unknown [202.54.11.65])
+ by svr1.postgresql.org (Postfix) with ESMTP id 9033ED1B4EB
+ for <pgsql-performance@postgresql.org>; Mon, 6 Oct 2003 03:06:41 -0300 (ADT)
+Received: (from root@localhost)
+ by smtp.pspl.co.in (8.12.9/8.12.9) id h966AmTk013993
+ for <pgsql-performance@postgresql.org>; Mon, 6 Oct 2003 11:40:49 +0530
+Received: from persistent.co.in (daithan.intranet.pspl.co.in [192.168.7.161])
+ (authenticated bits=0)
+ by persistent.co.in (8.12.9/8.12.9) with ESMTP id h966AlYM013922
+ for <pgsql-performance@postgresql.org>; Mon, 6 Oct 2003 11:40:48 +0530
+Message-ID: <3F81066C.90402@persistent.co.in>
+Date: Mon, 06 Oct 2003 11:36:36 +0530
+From: Shridhar Daithankar <shridhar_daithankar@persistent.co.in>
+Organization: Persistent Systems Pvt. Ltd.
+User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b) Gecko/20030917 Thunderbird/0.3a
+X-Accept-Language: en-us, en
+MIME-Version: 1.0
+To: pgsql-performance@postgresql.org
+Subject: Re: [PERFORM] count(*) slow on large tables
+References: <200310041819.h94IJkV07596@candle.pha.pa.us>
+In-Reply-To: <200310041819.h94IJkV07596@candle.pha.pa.us>
+Content-Type: text/plain; charset=us-ascii; format=flowed
+Content-Transfer-Encoding: 7bit
+X-Virus-Scanned: by amavisd-new at postgresql.org
+X-Mailing-List: pgsql-performance
+Precedence: bulk
+Sender: pgsql-performance-owner@postgresql.org
+Status: OR
+
+Bruce Momjian wrote:
+> OK, I beefed up the TODO:
+>
+> * Use a fixed row count and a +/- count with MVCC visibility rules
+> to allow fast COUNT(*) queries with no WHERE clause(?)
+>
+> I can always give the details if someone asks. It doesn't seem complex
+> enough for a separate TODO.detail item.
+
+May I propose alternate approach for this optimisation?
+
+- Postgresql allows to maintain user defined variables in shared memory.
+- These variables obey transactions but do not get written to disk at all.
+- There should be a facility to detect whether such a variable is initialized or
+not.
+
+How it will help? This is in addition to trigger proposal that came up earlier.
+With triggers it's not possible to make values visible across backends unless
+trigger updates a table, which eventually leads to vacuum/dead tuples problem.
+
+1. User creates a trigger to check updates/inserts for certain conditions.
+2. It updates the count as and when required.
+3. If the trigger detects the count is not initialized, it would issue the same
+query first time. There is no avoiding this issue.
+
+Besides providing facility of resident variables could be used imaginatively as
+well.
+
+Does this make sense? IMO this is more generalised approach over all.
+
+Just a thought.
+
+ Shridhar
+
+
+
+
+---------------------------(end of broadcast)---------------------------
+TIP 6: Have you searched our list archives?
+
+ http://archives.postgresql.org
+
+From pgsql-performance-owner+M3938@postgresql.org Mon Oct 6 16:04:10 2003
+Return-path: <pgsql-performance-owner+M3938@postgresql.org>
+Received: from svr5.postgresql.org (svr5.postgresql.org [64.117.225.181])
+ by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id h96K49i20610
+ for <pgman@candle.pha.pa.us>; Mon, 6 Oct 2003 16:04:10 -0400 (EDT)
+Received: from postgresql.org (svr1.postgresql.org [64.117.224.193])
+ by svr5.postgresql.org (Postfix) with ESMTP
+ id 9B73272DC4D; Mon, 6 Oct 2003 20:04:08 +0000 (GMT)
+X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org
+Received: from localhost (unknown [64.117.224.130])
+ by svr1.postgresql.org (Postfix) with ESMTP id 3770CD1B567
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>; Mon, 6 Oct 2003 15:11:08 +0000 (GMT)
+Received: from svr1.postgresql.org ([64.117.224.193])
+ by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024)
+ with ESMTP id 81338-08
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>;
+ Mon, 6 Oct 2003 12:10:22 -0300 (ADT)
+Received: from main.gmane.org (main.gmane.org [80.91.224.249])
+ by svr1.postgresql.org (Postfix) with ESMTP id E71D7D1B51E
+ for <pgsql-performance@postgresql.org>; Mon, 6 Oct 2003 12:10:21 -0300 (ADT)
+Received: from root by main.gmane.org with local (Exim 3.35 #1 (Debian))
+ id 1A6X08-0003KO-00
+ for <pgsql-performance@postgresql.org>; Mon, 06 Oct 2003 17:10:20 +0200
+X-Injected-Via-Gmane: http://gmane.org/
+To: pgsql-performance@postgresql.org
+Received: from sea.gmane.org ([80.91.224.252])
+ by main.gmane.org with esmtp (Exim 3.35 #1 (Debian))
+ id 1A6Wxn-0003Hh-00
+ for <gmane-comp-db-postgresql-performance@m.gmane.org>; Mon, 06 Oct 2003 17:07:55 +0200
+Received: from news by sea.gmane.org with local (Exim 3.35 #1 (Debian))
+ id 1A6Wxn-0006U8-00
+ for <gmane-comp-db-postgresql-performance@m.gmane.org>; Mon, 06 Oct 2003 17:07:55 +0200
+From: Harald Fuchs <nospam@sap.com>
+Subject: Re: [PERFORM] count(*) slow on large tables
+Date: 06 Oct 2003 17:08:36 +0200
+Organization: Linux Private Site
+Lines: 21
+Message-ID: <pupthae74b.fsf@srv.protecting.net>
+References: <20031002191547.GZ87525@rlx11.zapatec.com> <20031002193905.GD18417@wolff.to> <3F7C98B8.C892D0E5@nsd.ca> <60brszcng5.fsf@dev6.int.libertyrms.info> <20031002223313.GE87525@rlx11.zapatec.com> <m3vfr7f4z1.fsf@wolfe.cbbrowne.com> <20031003042754.GH87525@rlx11.zapatec.com> <3F7D172E.3060107@persistent.co.in>
+Reply-To: hf99@protecting.net
+MIME-Version: 1.0
+Content-Type: text/plain; charset=US-ASCII
+X-Complaints-To: usenet@sea.gmane.org
+X-No-Archive: yes
+User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7
+X-Virus-Scanned: by amavisd-new at postgresql.org
+X-Mailing-List: pgsql-performance
+Precedence: bulk
+Sender: pgsql-performance-owner@postgresql.org
+Status: OR
+
+In article <3F7D172E.3060107@persistent.co.in>,
+Shridhar Daithankar <shridhar_daithankar@persistent.co.in> writes:
+
+> Dror Matalon wrote:
+>> I smell a religious war in the aii:-). Can you go several days in a
+>> row without doing select count(*) on any
+>> of your tables? I suspect that this is somewhat a domain specific
+>> issue. In some areas
+>> you don't need to know the total number of rows in your tables, in
+>> others you do.
+
+> If I were you, I would have an autovacuum daemon running and rather
+> than doing select count(*), I would look at stats generated by
+> vacuums. They give approximate number of tuples and it should be good
+> enough it is accurate within a percent.
+
+The stats might indeed be a good estimate presumed there were not many
+changes since the last VACUUM. But how about a variant of COUNT(*)
+using an index? It would not be quite exact since it might contain
+tuples not visible in the current transaction, but it might be a much
+better estimate than the stats.
+
+
+---------------------------(end of broadcast)---------------------------
+TIP 2: you can get off all lists at once with the unregister command
+ (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
+
+From pgsql-performance-owner+M3930@postgresql.org Mon Oct 6 13:03:02 2003
+Return-path: <pgsql-performance-owner+M3930@postgresql.org>
+Received: from svr4.postgresql.org (svr4.postgresql.org [64.117.224.192])
+ by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id h96H30Q06466
+ for <pgman@candle.pha.pa.us>; Mon, 6 Oct 2003 13:03:00 -0400 (EDT)
+Received: from postgresql.org (svr1.postgresql.org [64.117.224.193])
+ by svr4.postgresql.org (Postfix) with ESMTP
+ id 314A01CB46D6; Mon, 6 Oct 2003 17:02:55 +0000 (GMT)
+X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org
+Received: from localhost (unknown [64.117.224.130])
+ by svr1.postgresql.org (Postfix) with ESMTP id E83D7D1B4F2
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>; Mon, 6 Oct 2003 17:02:38 +0000 (GMT)
+Received: from svr1.postgresql.org ([64.117.224.193])
+ by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024)
+ with ESMTP id 03671-08
+ for <pgsql-performance-postgresql.org@localhost.postgresql.org>;
+ Mon, 6 Oct 2003 14:01:53 -0300 (ADT)
+Received: from perrin.nxad.com (internal.nxad.com [69.1.70.251])
+ by svr1.postgresql.org (Postfix) with ESMTP id DEADDD1B4EC
+ for <pgsql-performance@postgresql.org>; Mon, 6 Oct 2003 14:01:51 -0300 (ADT)
+Received: by perrin.nxad.com (Postfix, from userid 1001)
+ id 64CEC21068; Mon, 6 Oct 2003 10:01:36 -0700 (PDT)
+Date: Mon, 6 Oct 2003 10:01:36 -0700
+From: Sean Chittenden <sean@chittenden.org>
+To: Shridhar Daithankar <shridhar_daithankar@persistent.co.in>
+cc: pgsql-performance@postgresql.org
+Subject: Re: [PERFORM] count(*) slow on large tables
+Message-ID: <20031006170136.GB94718@perrin.nxad.com>
+References: <200310041819.h94IJkV07596@candle.pha.pa.us> <3F81066C.90402@persistent.co.in>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+In-Reply-To: <3F81066C.90402@persistent.co.in>
+X-PGP-Key: finger seanc@FreeBSD.org
+X-PGP-Fingerprint: 3849 3760 1AFE 7B17 11A0 83A6 DD99 E31F BC84 B341
+X-Web-Homepage: http://sean.chittenden.org/
+User-Agent: Mutt/1.5.4i
+X-Virus-Scanned: by amavisd-new at postgresql.org
+X-Mailing-List: pgsql-performance
+Precedence: bulk
+Sender: pgsql-performance-owner@postgresql.org
+Status: OR
+
+> How it will help? This is in addition to trigger proposal that came
+> up earlier. With triggers it's not possible to make values visible
+> across backends unless trigger updates a table, which eventually
+> leads to vacuum/dead tuples problem.
+>
+> 1. User creates a trigger to check updates/inserts for certain conditions.
+> 2. It updates the count as and when required.
+> 3. If the trigger detects the count is not initialized, it would issue the
+> same query first time. There is no avoiding this issue.
+>
+> Besides providing facility of resident variables could be used
+> imaginatively as well.
+>
+> Does this make sense? IMO this is more generalised approach over all.
+
+I do this _VERY_ frequently in my databases, only I have my stored
+procs do the aggregate in a predefined MVCC table that's always there.
+Here's a denormalized version for public consumption/thought:
+
+CREATE TABLE global.dba_aggregate_cache (
+ dbl TEXT NOT NULL, -- The database location, doesn't need to be
+ -- qualified (ex: schema.table.col)
+ op TEXT NOT NULL, -- The operation, SUM, COUNT, etc.
+ qual TEXT, -- Any kind of conditional, such as a where clause
+ val_int INT, -- Whatever the value is, of type INT
+ val_bigint BIGINT, -- Whatever the value is, of type BIGINT
+ val_text TEXT, -- Whatever the value is, of type TEXT
+ val_bytea BYTEA, -- Whatever the value is, of type BYTEA
+);
+CREATE UNIQUE INDEX dba_aggregate_cache_dbl_op_udx ON global.dba_aggregate_cache(dbl,op);
+
+Then, I use a function to retrieve this value instead of a SELECT
+COUNT(*).
+
+SELECT public.cache_count('dbl','qual'); -- In this case, the op is COUNT
+SELECT public.cache_count('dbl'); -- Returns the COUNT for the table listed in the dbl
+
+Then, I create 4 or 5 functions (depends on the op I'm performing):
+
+1) A private function that _doesn't_ run as security definer, that
+ populates the global.dba_aggregate_cache row if it's empty.
+2) A STABLE function for SELECTs, if the row doesn't exist, then it
+ calls function #1 to populate its existence.
+3) A STABLE function for INSERTs, if the row doesn't exist, then it
+ calls function #1 to populate its existence, then adds the
+ necessary bits to make it accurate.
+4) A STABLE function for DELETEs, if the row doesn't exist, then it
+ calls function #1 to populate its existence, then deletes the
+ necessary bits to make it accurate.
+5) A STABLE function for UPDATEs, if the row doesn't exist, then it
+ calls function #1 to populate its existence, then updates the
+ necessary bits to make it accurate. It's not uncommon for me to
+ not have an UPDATE function/trigger.
+
+Create triggers for functions 2-5, and test away. It's MVCC,
+searching through a table that's INDEX'ed for a single row is
+obviously vastly faster than a seqscan/aggregate. If I need any kind
+of an aggregate to be fast, I use this system with a derivation of the
+above table. The problem with it being that I have to retrain others
+to use cache_count(), or some other function instead of using
+COUNT(*).
+
+That said, it'd be nice if there were a way to tell PostgreSQL to do
+the above for you and teach COUNT(*), SUM(*), or other aggregates to
+use an MVCC backed cache similar to the above. If people want their
+COUNT's to be fast, then they have to live with the INSERT, UPDATE,
+DELETE cost. The above doesn't work with anything complex such as
+join's, but it's certainly a start and I think satisfies everyone's
+gripes other than the tuple churn that _does_ happen (*nudge nudge*,
+pg_autovacuum could be integrated into the backend to handle this).
+Those worried about performance, the pages that are constantly being
+recycled would likely stay in disk cache (PG or the OS). There's
+still some commit overhead, but still... no need to over optimize by
+requiring the table to be stored in the out dated, slow, and over used
+shm (also, *nudge nudge*).
+
+Anyway, let me throw that out there as a solution that I use and it
+works quite well. I didn't explain the use of the qual column, but I
+think those who grasp the above way of handling things probably grok
+how to use the qual column in a dynamically executed query.
+
+CREATE AGGREGATE CACHE anyone?
+
+-sc
+
+--
+Sean Chittenden
+
+---------------------------(end of broadcast)---------------------------
+TIP 7: don't forget to increase your free space map settings
+
projects / postgresql / commitdiff