Peter Eisentraut [Tue, 18 Mar 2008 16:24:50 +0000 (16:24 +0000)]
Catch all errors in for and while loops in makefiles. Don't ignore any
errors in any commands, including in various clean targets that have so far
been handled inconsistently. make -i is available to ignore all errors in
a consistent and official way.
Tom Lane [Tue, 18 Mar 2008 03:54:52 +0000 (03:54 +0000)]
Advance multiple array keys rightmost-first instead of leftmost-first
during a bitmap index scan. This cannot affect the query results
(since we're just dumping the TIDs into a bitmap) but it might offer
some advantage in locality of access to the index. Per Greg Stark.
Bruce Momjian [Tue, 18 Mar 2008 02:37:05 +0000 (02:37 +0000)]
Add TODO:
> o Recreate pg_xlog/archive_status/ if it doesn't exist after
> restoring from a PITR backup
>
> http://archives.postgresql.org/pgsql-hackers/2007-12/msg00487.php
Tom Lane [Tue, 18 Mar 2008 01:49:44 +0000 (01:49 +0000)]
Fix our printf implementation to follow spec: if a star parameter
value for a precision is negative, act as though precision weren't
specified at all, that is the whole .* part of the format spec should
be ignored. Our previous coding took it as .0 which is certainly
wrong. Per report from Kris Jurka and local testing.
Possibly this should be back-patched, but it would be good to get
some more testing first; in any case there are no known cases where
there's really a problem on the backend side.
Bruce Momjian [Tue, 18 Mar 2008 00:43:01 +0000 (00:43 +0000)]
Add to TODO:
>
> * Consider Cartesian joins when both relations are needed to form an
> indexscan qualification for a third relation
>
> http://archives.postgresql.org/pgsql-performance/2007-12/msg00090.php
Bruce Momjian [Tue, 18 Mar 2008 00:23:41 +0000 (00:23 +0000)]
Add URL for:
o Allow COPY to report error lines and continue
This requires the use of a savepoint before each COPY line is
processed, with ROLLBACK on COPY failure.
> http://archives.postgresql.org/pgsql-hackers/2007-12/msg00572.php
Bruce Momjian [Mon, 17 Mar 2008 23:56:30 +0000 (23:56 +0000)]
Add to TODO:
>
> * Allow SSL key file permission checks to be optionally disabled when
> sharing SSL keys with other applications
>
> http://archives.postgresql.org/pgsql-bugs/2007-12/msg00069.php
Bruce Momjian [Mon, 17 Mar 2008 22:59:01 +0000 (22:59 +0000)]
Add to TODO:
> * Consider if CommandCounterIncrement() can avoid its
> AcceptInvalidationMessages() call
>
> http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php
Bruce Momjian [Mon, 17 Mar 2008 21:49:11 +0000 (21:49 +0000)]
Add TODO:
>
> o Fix server restart problem when the server was shutdown during
> a PITR backup
>
> http://archives.postgresql.org/pgsql-hackers/2007-11/msg00800.php
>
Peter Eisentraut [Mon, 17 Mar 2008 19:44:41 +0000 (19:44 +0000)]
Enable probes to work with Mac OS X Leopard and other OSes that will
support DTrace in the future.
Switch from using DTRACE_PROBEn macros to the dynamically generated macros.
Use "dtrace -h" to create a header file that contains the dynamically
generated macros to be used in the source code instead of the DTRACE_PROBEn
macros. A dummy header file is generated for builds without DTrace support.
Bruce Momjian [Mon, 17 Mar 2008 18:22:44 +0000 (18:22 +0000)]
Add:
>
> o Allow Kerberos to disable stripping of realms so we can
> check the username@realm against multiple realms
>
> http://archives.postgresql.org/pgsql-hackers/2007-11/msg00009.php
Tom Lane [Mon, 17 Mar 2008 03:45:36 +0000 (03:45 +0000)]
Grab some low-hanging fruit in the new hash index build code.
oprofile shows that a nontrivial amount of time is being spent in
repeated calls to index_getprocinfo, which really only needs to be
called once. So do that, and inline _hash_datum2hashkey to make it
work.
Tom Lane [Mon, 17 Mar 2008 02:18:55 +0000 (02:18 +0000)]
Fix TransactionIdIsCurrentTransactionId() to use binary search instead of
linear search when checking child-transaction XIDs. This makes for an
important speedup in transactions that have large numbers of children,
as in a recent example from Craig Ringer. We can also get rid of an
ugly kluge that represented lists of TransactionIds as lists of OIDs.
Tom Lane [Sun, 16 Mar 2008 23:15:08 +0000 (23:15 +0000)]
When creating a large hash index, pre-sort the index entries by estimated
bucket number, so as to ensure locality of access to the index during the
insertion step. Without this, building an index significantly larger than
available RAM takes a very long time because of thrashing. On the other
hand, sorting is just useless overhead when the index does fit in RAM.
We choose to sort when the initial index size exceeds effective_cache_size.
This is a revised version of work by Tom Raney and Shreya Bhargava.
Alvaro Herrera [Sun, 16 Mar 2008 19:47:34 +0000 (19:47 +0000)]
Modify interactions between sinval.c and sinvaladt.c. The code that actually
deals with the queue, including locking etc, is all in sinvaladt.c. This means
that the struct definition of the queue, and the queue pointer, are now
internal "implementation details" inside sinvaladt.c.
Per my proposal dated 25-Jun-2007 and followup discussion.
Tom Lane [Sat, 15 Mar 2008 20:46:31 +0000 (20:46 +0000)]
Change hash index creation so that rather than always establishing exactly
two buckets at the start, we create a number of buckets appropriate for the
estimated size of the table. This avoids a lot of expensive bucket-split
actions during initial index build on an already-populated table.
This is one of the two core ideas of Tom Raney and Shreya Bhargava's patch
to reduce hash index build time. I'm committing it separately to make it
easier for people to test the effects of this separately from the effects
of their other core idea (pre-sorting the index entries by bucket number).
Tom Lane [Fri, 14 Mar 2008 23:49:28 +0000 (23:49 +0000)]
Fix inappropriately-timed memory context switch in autovacuum_do_vac_analyze.
This accidentally failed to fail before 8.3, because the context we were
switching back to was long-lived anyway; but it sure looks risky as can be
now. Well spotted by Pavan Deolasee.
Alvaro Herrera [Fri, 14 Mar 2008 17:25:59 +0000 (17:25 +0000)]
Fix vacuum so that autovacuum is really not cancelled when doing an emergency
job (i.e. to prevent Xid wraparound problems.) Bug reported by ITAGAKI
Takahiro in 20080314103837.63D3.52131E4D@oss.ntt.co.jp, though I didn't use his
patch.
Tom Lane [Thu, 13 Mar 2008 18:31:56 +0000 (18:31 +0000)]
Fix varstr_cmp's special case for UTF8 encoding on Windows so that strings
that are reported as "equal" by wcscoll() are checked to see if they really
are bitwise equal, and are sorted per strcmp() if not. We made this happen
a couple of years ago in the regular code path, but it unaccountably got
left out of the Windows/UTF8 case (probably brain fade on my part at the
time). As in the prior set of changes, affected users may need to reindex
indexes on textual columns.
Backpatch as far as 8.2, which is the oldest release we are still supporting
on Windows.
Tom Lane [Thu, 13 Mar 2008 18:00:32 +0000 (18:00 +0000)]
Fix heap_page_prune's problem with failing to send cache invalidation
messages if the calling transaction aborts later on. Collapsing out line
pointer redirects is a done deal as soon as we complete the page update,
so syscache *must* be notified even if the VACUUM FULL as a whole doesn't
complete. To fix, add some functionality to inval.c to allow the pending
inval messages to be sent immediately while heap_page_prune is still
running. The implementation is a bit chintzy: it will only work in the
context of VACUUM FULL. But that's all we need now, and it can always be
extended later if needed. Per my trouble report of a week ago.
Tom Lane [Wed, 12 Mar 2008 23:58:27 +0000 (23:58 +0000)]
Fix pg_plan_queries() to restore the previous setting of ActiveSnapshot
(probably NULL) before exiting. Up to now it's just left the variable as it
set it, which means that after we're done processing the current client
message, ActiveSnapshot is probably pointing at garbage (because this function
is typically run in MessageContext which will get reset). There doesn't seem
to have been any code path in which that mattered before 8.3, but now the
plancache module might try to use the stale value if the next client message
is a Bind for a prepared statement that is in need of replanning. Per report
from Alex Hunsaker.
Tom Lane [Wed, 12 Mar 2008 20:11:46 +0000 (20:11 +0000)]
Fix LISTEN/NOTIFY race condition reported by Laurent Birtz, by postponing
pg_listener modifications commanded by LISTEN and UNLISTEN until the end
of the current transaction. This allows us to hold the ExclusiveLock on
pg_listener until after commit, with no greater risk of deadlock than there
was before. Aside from fixing the race condition, this gets rid of a
truly ugly kludge that was there before, namely having to ignore
HeapTupleBeingUpdated failures during NOTIFY. There is a small potential
incompatibility, which is that if a transaction issues LISTEN or UNLISTEN
and then looks into pg_listener before committing, it won't see any resulting
row insertion or deletion, where before it would have. It seems unlikely
that anyone would be depending on that, though.
This patch also disallows LISTEN and UNLISTEN inside a prepared transaction.
That case had some pretty undesirable properties already, such as possibly
allowing pg_listener entries to be made for PIDs no longer present, so
disallowing it seems like a better idea than trying to maintain the behavior.
Bruce Momjian [Wed, 12 Mar 2008 18:33:54 +0000 (18:33 +0000)]
Add for Win32 TODO:
> o Diagnose problem where shared memory can sometimes not be
> attached by postmaster children
>
> http://archives.postgresql.org/pgsql-general/2007-08/msg01377.php
>
Bruce Momjian [Wed, 12 Mar 2008 01:48:08 +0000 (01:48 +0000)]
Add URL for:
* Improve speed with indexes
For large table adjustments during VACUUM FULL, it is faster to cluster
or reindex rather than update the index. Also, index updates can bloat
the index.
Bruce Momjian [Tue, 11 Mar 2008 22:31:02 +0000 (22:31 +0000)]
Add to pl/pgsql:
>
> o Improve logic of determining if an identifier is a a
> variable or column name
>
> http://archives.postgresql.org/pgsql-hackers/2007-07/msg00436.php
Bruce Momjian [Tue, 11 Mar 2008 20:46:20 +0000 (20:46 +0000)]
< * Consider adding buffers the BGW finds reusable to the free list
> * Consider adding buffers the background writer finds reusable to the
> free list
>
> * Consider wither increasing BM_MAX_USAGE_COUNT improves performance
>
> http://archives.postgresql.org/pgsql-hackers/2007-06/msg01007.php
Tom Lane [Tue, 11 Mar 2008 20:20:35 +0000 (20:20 +0000)]
Make TransactionIdIsInProgress check transam.c's single-item XID status cache
before it goes groveling through the ProcArray. In situations where the same
recently-committed transaction ID is checked repeatedly by tqual.c, this saves
a lot of shared-memory searches. And it's cheap enough that it shouldn't
hurt noticeably when it doesn't help.
Concept and patch by Simon, some minor tweaking and comment-cleanup by Tom.
Bruce Momjian [Tue, 11 Mar 2008 19:44:39 +0000 (19:44 +0000)]
Add:
>
> * Change memory allocation for multi-byte functions so memory is
> allocated inside conversion functions
>
> Currently we preallocate memory based on worst-case usage.
Bruce Momjian [Tue, 11 Mar 2008 19:34:12 +0000 (19:34 +0000)]
Add another URL for:
* Consider increasing the number of default statistics target, and
reduce statistics target overhead
Also consider having a larger statistics target for indexed columns
and expression indexes
<
> http://archives.postgresql.org/pgsql-general/2007-06/msg00542.php
Bruce Momjian [Tue, 11 Mar 2008 19:32:03 +0000 (19:32 +0000)]
Add URL for:
* Consider increasing the number of default statistics target, and
reduce statistics target overhead
Also consider having a larger statistics target for indexed columns
and expression indexes
> http://archives.postgresql.org/pgsql-general/2007-05/msg01228.php
>
Bruce Momjian [Tue, 11 Mar 2008 18:30:50 +0000 (18:30 +0000)]
Add:
>
> * Consider increasing the number of default statistics target, and
> reduce statistics target overhead
>
> Also consider having a larger statistics target for indexed columns
> and expression indexes
Bruce Momjian [Tue, 11 Mar 2008 15:45:05 +0000 (15:45 +0000)]
Add:
>
> * Consider adding buffers the BGW finds reusable to the free list
>
> http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php
>
> * Automatically tune bgwriter_delay based on activity rather then using a
> fixed interval
>
> http://archives.postgresql.org/pgsql-hackers/2007-04/msg00781.php
Tom Lane [Mon, 10 Mar 2008 21:50:16 +0000 (21:50 +0000)]
Use -fwrapv in CFLAGS if we are using a version of gcc that accepts this flag.
This prevents compiler optimizations that assume overflow won't occur, which
breaks numerous overflow tests that we need to have working. It is known
that gcc 4.3 causes problems and possible that 4.1 does. Per my proposal
of some time ago and a recent report from Kris Jurka.
Backpatch as far as 8.0, which is as far as the patch conveniently goes.
7.x was pretty short of overflow tests anyway, so it may not matter there,
even assuming that anyone cares whether 7.x builds on recent gcc.
Tom Lane [Mon, 10 Mar 2008 20:06:27 +0000 (20:06 +0000)]
Provide a build-time option to store large relations as single files, rather
than dividing them into 1GB segments as has been our longtime practice. This
requires working support for large files in the operating system; at least for
the time being, it won't be the default.
projects / postgresql / log