Tom Lane [Fri, 21 Mar 2008 01:31:43 +0000 (01:31 +0000)]
Get rid of a bunch of #ifdef HAVE_INT64_TIMESTAMP conditionals by inventing
a new typedef TimeOffset to represent an intermediate time value. It's
either int64 or double as appropriate, and in most usages will be measured
in microseconds or seconds the same as Timestamp. We don't call it
Timestamp, though, since the value doesn't necessarily represent an absolute
time instant.
Tom Lane [Thu, 20 Mar 2008 21:42:48 +0000 (21:42 +0000)]
Arrange for an explicit cast applied to an ARRAY[] constructor to be applied
directly to all the member expressions, instead of the previous implementation
where the ARRAY[] constructor would infer a common element type and then we'd
coerce the finished array after the fact. This has a number of benefits,
one being that we can allow an empty ARRAY[] construct so long as its
element type is specified by such a cast.
Tom Lane [Thu, 20 Mar 2008 17:36:58 +0000 (17:36 +0000)]
Support a --no-tablespaces option in pg_dump/pg_dumpall/pg_restore, so that
dumps can be loaded into databases without the same tablespaces that the
source had. The option acts by suppressing all "SET default_tablespace"
commands, and also CREATE TABLESPACE commands in pg_dumpall's case.
Gavin Roy, with documentation and minor fixes by me.
Add the missing cyrillic "Yo" characters ('e' and 'E' with two dots) to the
ISO_8859-5 <-> MULE_INTERNAL conversion tables.
This was discovered when trying to convert a string containing those characters
from ISO_8859-5 to Windows-1251, because we use MULE_INTERNAL/KOI8R as an
intermediate encoding between those two.
While the missing "Yo" was just an omission in the conversion tables, there are
a few other characters like the "Numero" sign ("No" as a single character) that
exists in all the other cyrillic encodings (win1251, ISO_8859-5 and cp866), but
not in KOI8R. Added comments about that.
Alvaro Herrera [Wed, 19 Mar 2008 14:18:21 +0000 (14:18 +0000)]
We no longer need a snapshot set after opening the finishing transaction: this
is redundant because autovacuum now always analyzes a single table per
transaction.
Tom Lane [Wed, 19 Mar 2008 02:40:37 +0000 (02:40 +0000)]
Fix regexp substring matching (substring(string from pattern)) for the corner
case where there is a match to the pattern overall but the user has specified
a parenthesized subexpression and that subexpression hasn't got a match.
An example is substring('foo' from 'foo(bar)?'). This should return NULL,
since (bar) isn't matched, but it was mistakenly returning the whole-pattern
match instead (ie, 'foo'). Per bug #4044 from Rui Martins.
This has been broken since the beginning; patch in all supported versions.
The old behavior was sufficiently inconsistent that it's impossible to believe
anyone is depending on it.
Tatsuo Ishii [Wed, 19 Mar 2008 00:29:35 +0000 (00:29 +0000)]
Fix tps calculation when -C supplied. Per Yoshiyuki Asaba.
Change Copyright owner from mine to PostgreSQL Global Development Group
Fix minor message typo
Bruce Momjian [Tue, 18 Mar 2008 23:35:21 +0000 (23:35 +0000)]
Spit items:
* Experiment with multi-threaded backend better I/O utilization
This would allow a single query to make use of multiple I/O channels
simultaneously. One idea is to create a background reader that can
pre-fetch sequential and index scan pages needed by other backends.
This could be expanded to allow concurrent reads from multiple devices
in a partitioned table.
* Experiment with multi-threaded backend better CPU utilization
This would allow several CPUs to be used for a single query, such as
for sorting or query execution.
Tom Lane [Tue, 18 Mar 2008 22:04:14 +0000 (22:04 +0000)]
Arrange to "inline" SQL functions that appear in a query's FROM clause,
are declared to return set, and consist of just a single SELECT. We
can replace the FROM-item with a sub-SELECT and then optimize much as
if we were dealing with a view. Patch from Richard Rowell, cleaned up
by me.
Bruce Momjian [Tue, 18 Mar 2008 18:40:42 +0000 (18:40 +0000)]
Add to TODO:
>
> * Consider not storing a NULL bitmap on disk if all the NULLs are
> trailing
>
> http://archives.postgresql.org/pgsql-hackers/2007-12/msg00624.php
> http://archives.postgresql.org/pgsql-patches/2007-12/msg00109.php
>
Peter Eisentraut [Tue, 18 Mar 2008 16:24:50 +0000 (16:24 +0000)]
Catch all errors in for and while loops in makefiles. Don't ignore any
errors in any commands, including in various clean targets that have so far
been handled inconsistently. make -i is available to ignore all errors in
a consistent and official way.
Tom Lane [Tue, 18 Mar 2008 03:54:52 +0000 (03:54 +0000)]
Advance multiple array keys rightmost-first instead of leftmost-first
during a bitmap index scan. This cannot affect the query results
(since we're just dumping the TIDs into a bitmap) but it might offer
some advantage in locality of access to the index. Per Greg Stark.
Bruce Momjian [Tue, 18 Mar 2008 02:37:05 +0000 (02:37 +0000)]
Add TODO:
> o Recreate pg_xlog/archive_status/ if it doesn't exist after
> restoring from a PITR backup
>
> http://archives.postgresql.org/pgsql-hackers/2007-12/msg00487.php
Tom Lane [Tue, 18 Mar 2008 01:49:44 +0000 (01:49 +0000)]
Fix our printf implementation to follow spec: if a star parameter
value for a precision is negative, act as though precision weren't
specified at all, that is the whole .* part of the format spec should
be ignored. Our previous coding took it as .0 which is certainly
wrong. Per report from Kris Jurka and local testing.
Possibly this should be back-patched, but it would be good to get
some more testing first; in any case there are no known cases where
there's really a problem on the backend side.
Bruce Momjian [Tue, 18 Mar 2008 00:43:01 +0000 (00:43 +0000)]
Add to TODO:
>
> * Consider Cartesian joins when both relations are needed to form an
> indexscan qualification for a third relation
>
> http://archives.postgresql.org/pgsql-performance/2007-12/msg00090.php
Bruce Momjian [Tue, 18 Mar 2008 00:23:41 +0000 (00:23 +0000)]
Add URL for:
o Allow COPY to report error lines and continue
This requires the use of a savepoint before each COPY line is
processed, with ROLLBACK on COPY failure.
> http://archives.postgresql.org/pgsql-hackers/2007-12/msg00572.php
Bruce Momjian [Mon, 17 Mar 2008 23:56:30 +0000 (23:56 +0000)]
Add to TODO:
>
> * Allow SSL key file permission checks to be optionally disabled when
> sharing SSL keys with other applications
>
> http://archives.postgresql.org/pgsql-bugs/2007-12/msg00069.php
Bruce Momjian [Mon, 17 Mar 2008 22:59:01 +0000 (22:59 +0000)]
Add to TODO:
> * Consider if CommandCounterIncrement() can avoid its
> AcceptInvalidationMessages() call
>
> http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php
Bruce Momjian [Mon, 17 Mar 2008 21:49:11 +0000 (21:49 +0000)]
Add TODO:
>
> o Fix server restart problem when the server was shutdown during
> a PITR backup
>
> http://archives.postgresql.org/pgsql-hackers/2007-11/msg00800.php
>
Peter Eisentraut [Mon, 17 Mar 2008 19:44:41 +0000 (19:44 +0000)]
Enable probes to work with Mac OS X Leopard and other OSes that will
support DTrace in the future.
Switch from using DTRACE_PROBEn macros to the dynamically generated macros.
Use "dtrace -h" to create a header file that contains the dynamically
generated macros to be used in the source code instead of the DTRACE_PROBEn
macros. A dummy header file is generated for builds without DTrace support.
Bruce Momjian [Mon, 17 Mar 2008 18:22:44 +0000 (18:22 +0000)]
Add:
>
> o Allow Kerberos to disable stripping of realms so we can
> check the username@realm against multiple realms
>
> http://archives.postgresql.org/pgsql-hackers/2007-11/msg00009.php
Tom Lane [Mon, 17 Mar 2008 03:45:36 +0000 (03:45 +0000)]
Grab some low-hanging fruit in the new hash index build code.
oprofile shows that a nontrivial amount of time is being spent in
repeated calls to index_getprocinfo, which really only needs to be
called once. So do that, and inline _hash_datum2hashkey to make it
work.
Tom Lane [Mon, 17 Mar 2008 02:18:55 +0000 (02:18 +0000)]
Fix TransactionIdIsCurrentTransactionId() to use binary search instead of
linear search when checking child-transaction XIDs. This makes for an
important speedup in transactions that have large numbers of children,
as in a recent example from Craig Ringer. We can also get rid of an
ugly kluge that represented lists of TransactionIds as lists of OIDs.
Tom Lane [Sun, 16 Mar 2008 23:15:08 +0000 (23:15 +0000)]
When creating a large hash index, pre-sort the index entries by estimated
bucket number, so as to ensure locality of access to the index during the
insertion step. Without this, building an index significantly larger than
available RAM takes a very long time because of thrashing. On the other
hand, sorting is just useless overhead when the index does fit in RAM.
We choose to sort when the initial index size exceeds effective_cache_size.
This is a revised version of work by Tom Raney and Shreya Bhargava.
Alvaro Herrera [Sun, 16 Mar 2008 19:47:34 +0000 (19:47 +0000)]
Modify interactions between sinval.c and sinvaladt.c. The code that actually
deals with the queue, including locking etc, is all in sinvaladt.c. This means
that the struct definition of the queue, and the queue pointer, are now
internal "implementation details" inside sinvaladt.c.
Per my proposal dated 25-Jun-2007 and followup discussion.
Tom Lane [Sat, 15 Mar 2008 20:46:31 +0000 (20:46 +0000)]
Change hash index creation so that rather than always establishing exactly
two buckets at the start, we create a number of buckets appropriate for the
estimated size of the table. This avoids a lot of expensive bucket-split
actions during initial index build on an already-populated table.
This is one of the two core ideas of Tom Raney and Shreya Bhargava's patch
to reduce hash index build time. I'm committing it separately to make it
easier for people to test the effects of this separately from the effects
of their other core idea (pre-sorting the index entries by bucket number).
Tom Lane [Fri, 14 Mar 2008 23:49:28 +0000 (23:49 +0000)]
Fix inappropriately-timed memory context switch in autovacuum_do_vac_analyze.
This accidentally failed to fail before 8.3, because the context we were
switching back to was long-lived anyway; but it sure looks risky as can be
now. Well spotted by Pavan Deolasee.
Alvaro Herrera [Fri, 14 Mar 2008 17:25:59 +0000 (17:25 +0000)]
Fix vacuum so that autovacuum is really not cancelled when doing an emergency
job (i.e. to prevent Xid wraparound problems.) Bug reported by ITAGAKI
Takahiro in 20080314103837.63D3.52131E4D@oss.ntt.co.jp, though I didn't use his
patch.
Tom Lane [Thu, 13 Mar 2008 18:31:56 +0000 (18:31 +0000)]
Fix varstr_cmp's special case for UTF8 encoding on Windows so that strings
that are reported as "equal" by wcscoll() are checked to see if they really
are bitwise equal, and are sorted per strcmp() if not. We made this happen
a couple of years ago in the regular code path, but it unaccountably got
left out of the Windows/UTF8 case (probably brain fade on my part at the
time). As in the prior set of changes, affected users may need to reindex
indexes on textual columns.
Backpatch as far as 8.2, which is the oldest release we are still supporting
on Windows.
Tom Lane [Thu, 13 Mar 2008 18:00:32 +0000 (18:00 +0000)]
Fix heap_page_prune's problem with failing to send cache invalidation
messages if the calling transaction aborts later on. Collapsing out line
pointer redirects is a done deal as soon as we complete the page update,
so syscache *must* be notified even if the VACUUM FULL as a whole doesn't
complete. To fix, add some functionality to inval.c to allow the pending
inval messages to be sent immediately while heap_page_prune is still
running. The implementation is a bit chintzy: it will only work in the
context of VACUUM FULL. But that's all we need now, and it can always be
extended later if needed. Per my trouble report of a week ago.
Tom Lane [Wed, 12 Mar 2008 23:58:27 +0000 (23:58 +0000)]
Fix pg_plan_queries() to restore the previous setting of ActiveSnapshot
(probably NULL) before exiting. Up to now it's just left the variable as it
set it, which means that after we're done processing the current client
message, ActiveSnapshot is probably pointing at garbage (because this function
is typically run in MessageContext which will get reset). There doesn't seem
to have been any code path in which that mattered before 8.3, but now the
plancache module might try to use the stale value if the next client message
is a Bind for a prepared statement that is in need of replanning. Per report
from Alex Hunsaker.
Tom Lane [Wed, 12 Mar 2008 20:11:46 +0000 (20:11 +0000)]
Fix LISTEN/NOTIFY race condition reported by Laurent Birtz, by postponing
pg_listener modifications commanded by LISTEN and UNLISTEN until the end
of the current transaction. This allows us to hold the ExclusiveLock on
pg_listener until after commit, with no greater risk of deadlock than there
was before. Aside from fixing the race condition, this gets rid of a
truly ugly kludge that was there before, namely having to ignore
HeapTupleBeingUpdated failures during NOTIFY. There is a small potential
incompatibility, which is that if a transaction issues LISTEN or UNLISTEN
and then looks into pg_listener before committing, it won't see any resulting
row insertion or deletion, where before it would have. It seems unlikely
that anyone would be depending on that, though.
This patch also disallows LISTEN and UNLISTEN inside a prepared transaction.
That case had some pretty undesirable properties already, such as possibly
allowing pg_listener entries to be made for PIDs no longer present, so
disallowing it seems like a better idea than trying to maintain the behavior.
Bruce Momjian [Wed, 12 Mar 2008 18:33:54 +0000 (18:33 +0000)]
Add for Win32 TODO:
> o Diagnose problem where shared memory can sometimes not be
> attached by postmaster children
>
> http://archives.postgresql.org/pgsql-general/2007-08/msg01377.php
>
Bruce Momjian [Wed, 12 Mar 2008 01:48:08 +0000 (01:48 +0000)]
Add URL for:
* Improve speed with indexes
For large table adjustments during VACUUM FULL, it is faster to cluster
or reindex rather than update the index. Also, index updates can bloat
the index.
Bruce Momjian [Tue, 11 Mar 2008 22:31:02 +0000 (22:31 +0000)]
Add to pl/pgsql:
>
> o Improve logic of determining if an identifier is a a
> variable or column name
>
> http://archives.postgresql.org/pgsql-hackers/2007-07/msg00436.php
Bruce Momjian [Tue, 11 Mar 2008 20:46:20 +0000 (20:46 +0000)]
< * Consider adding buffers the BGW finds reusable to the free list
> * Consider adding buffers the background writer finds reusable to the
> free list
>
> * Consider wither increasing BM_MAX_USAGE_COUNT improves performance
>
> http://archives.postgresql.org/pgsql-hackers/2007-06/msg01007.php