Tom Lane [Wed, 26 Sep 2007 01:10:42 +0000 (01:10 +0000)]
In the integer-datetimes case, date2timestamp and date2timestamptz need
to check for overflow because the legal range of type date is actually
wider than timestamp's. Problem found by Neil Conway.
Tom Lane [Wed, 26 Sep 2007 00:32:46 +0000 (00:32 +0000)]
Use SYSV semaphores rather than POSIX on Darwin >= 6.0 (i.e., OS X 10.2
and up), per Chris Marcellino. This avoids consuming O(N^2) file
descriptors to support N backends. Tests suggest it's about a wash for
small installations, but large ones would have a problem.
Tom Lane [Tue, 25 Sep 2007 22:21:55 +0000 (22:21 +0000)]
Change on-disk representation of NUMERIC datatype so that the sign_dscale
word comes before the weight instead of after. This will allow future
binary-compatible extension of the representation to support compact formats,
as discussed on pgsql-hackers around 2007/06/18. The reason to do it now is
that we've already pretty well broken any chance of simple in-place upgrade
from 8.2 to 8.3, but it's possible that 8.3 to 8.4 (or whenever we get around
to squeezing NUMERIC) could otherwise be data-compatible.
Tom Lane [Tue, 25 Sep 2007 22:11:48 +0000 (22:11 +0000)]
Dept. of second thoughts: fix loop in BgBufferSync so that the exit when
bgwriter_lru_maxpages is exceeded leaves the loop variables in the
expected state. In the original coding, we'd fail to advance
next_to_clean, causing that buffer to be probably-uselessly rechecked next
time, and also have an off-by-one idea of the number of buffers scanned.
Tom Lane [Tue, 25 Sep 2007 20:03:38 +0000 (20:03 +0000)]
Just-in-time background writing strategy. This code avoids re-scanning
buffers that cannot possibly need to be cleaned, and estimates how many
buffers it should try to clean based on moving averages of recent allocation
requests and density of reusable buffers. The patch also adds a couple
more columns to pg_stat_bgwriter to help measure the effectiveness of the
bgwriter.
Greg Smith, building on his own work and ideas from several other people,
in particular a much older patch from Itagaki Takahiro.
Avoid having autovacuum read pgstats data too many times in quick succession.
This is problematic for the autovac launcher when there are many databases,
so we keep data for a full second before reading it again.
Reduce the size of memory allocations by lazy vacuum when processing a small
table, by allocating just enough for a hardcoded number of dead tuples per
page. The current estimate is 200 dead tuples per page.
Per reports from Jeff Amiel, Erik Jones and Marko Kreen, and subsequent
discussion.
CVS: ----------------------------------------------------------------------
CVS: Enter Log. Lines beginning with `CVS:' are removed automatically
CVS:
CVS: Committing in .
CVS:
CVS: Modified Files:
CVS: commands/vacuumlazy.c
CVS: ----------------------------------------------------------------------
Andrew Dunstan [Mon, 24 Sep 2007 01:29:30 +0000 (01:29 +0000)]
Remove "convert 'blah' using conversion_name" facility, because if it
produces text it is an encoding hole and if not it's incompatible
with the spec, whatever the spec means (which we're not sure about anyway).
Andrew Dunstan [Sun, 23 Sep 2007 21:52:56 +0000 (21:52 +0000)]
Add perl replacements for build.bat and vcregress.bat. In due course
the .bat files will be altered to become tiny wrappers for these scripts,
and one or two other .bat files will disappear.
Tom Lane [Sun, 23 Sep 2007 18:50:38 +0000 (18:50 +0000)]
TransactionIdIsInProgress can skip scanning the ProcArray if the target XID is
later than latestCompletedXid, per Florian Pflug. Also some minor
improvements in the XIDCACHE_DEBUG code --- make sure each call of
TransactionIdIsInProgress is counted one way or another.
Tom Lane [Sun, 23 Sep 2007 15:58:58 +0000 (15:58 +0000)]
Temporarily modify tsearch regression tests to suppress notice that comes
out at erratic times, because it is creating a totally unacceptable level
of noise in our buildfarm results. This patch can be reverted when and if
the code is fixed to not issue notices during cache reload events.
Tom Lane [Sat, 22 Sep 2007 21:36:40 +0000 (21:36 +0000)]
Fix cost estimates for EXISTS subqueries that are evaluated as initPlans
(because they are uncorrelated with the immediate parent query). We were
charging the full run cost to the parent node, disregarding the fact that
only one row need be fetched for EXISTS. While this would only be a
cosmetic issue in most cases, it might possibly affect planning outcomes
if the parent query were itself a subquery to some upper query.
Per recent discussion with Steve Crawford.
Andrew Dunstan [Sat, 22 Sep 2007 20:38:10 +0000 (20:38 +0000)]
Replace calls to external dir program with perlish globs and File::Find
calls. Fixes complaint fron Hannes Eder, whose environment found a different
dir program.
Tom Lane [Sat, 22 Sep 2007 19:10:44 +0000 (19:10 +0000)]
Document the translations from Postgres message severity levels to
syslog and eventlog severity levels, per suggestion from Josh Drake.
Also, some wordsmithing for the csvlog documentation.
Tom Lane [Sat, 22 Sep 2007 18:19:18 +0000 (18:19 +0000)]
Fix erroneous Assert() in syslogger process start in EXEC_BACKEND case,
per ITAGAKI Takahiro. Also, rewrite syslogger_forkexec() in hopes of
eliminating the confusion in the first place.
Tom Lane [Sat, 22 Sep 2007 04:40:03 +0000 (04:40 +0000)]
Although I'd misdiagnosed the reason for the recent failures on
buildfarm member grebe, I see no reason to revert the 1-byte-header-friendly
changes I made in varlena.c. Instead, tweak the code a little bit to
get more advantage out of that.
Andrew Dunstan [Sat, 22 Sep 2007 03:58:34 +0000 (03:58 +0000)]
Go back to using a separate method for doing ILIKE for single byte
character encodings that doesn't involve calling lower(). This should
cure the performance regression in this case complained of by Guillaume
Smet. It still leaves the horrid performance for multi-byte encodings
introduced in 8.2, but there's no obvious solution for that in sight.
Tom Lane [Sat, 22 Sep 2007 00:36:38 +0000 (00:36 +0000)]
Fix varlena.c routines to allow 1-byte-header text values. This is now
demonstrably necessary for text_substring() since regexp_split functions
may pass it such a value; and we might as well convert the whole file
at once. Per buildfarm results (though I wonder why most machines aren't
showing a failure).
Tom Lane [Fri, 21 Sep 2007 22:52:52 +0000 (22:52 +0000)]
Fix regex, LIKE, and some other second-rank text-manipulation functions
to not cause needless copying of text datums that have 1-byte headers.
Greg Stark, in response to performance gripe from Guillaume Smet and
ITAGAKI Takahiro.
Tom Lane [Fri, 21 Sep 2007 21:25:42 +0000 (21:25 +0000)]
Improve handling of prune/no-prune decisions by storing a page's oldest
unpruned XMAX in its header. At the cost of 4 bytes per page, this keeps us
from performing heap_page_prune when there's no chance of pruning anything.
Seems to be necessary per Heikki's preliminary performance testing.
Tom Lane [Fri, 21 Sep 2007 18:24:28 +0000 (18:24 +0000)]
Change tqual.c tests to use !TransactionIdIsCurrentTransactionId, rather than
TransactionIdDidAbort, when handling the case that xmin is one of the current
transaction's XIDs and the tuple has been deleted. xmax must also be one of
the current transaction's XIDs, since no one else can see it yet, and it's
cheaper to look at local state than shared state to find out if xmax aborted.
Per an idea of Heikki's.
Tom Lane [Fri, 21 Sep 2007 17:36:53 +0000 (17:36 +0000)]
Make some simple performance improvements in TransactionIdIsInProgress().
For XIDs of our own transaction and subtransactions, it's cheaper to ask
TransactionIdIsCurrentTransactionId() than to look in shared memory.
Also, the xids[] work array is always the same size within any given
process, so malloc it just once instead of doing a palloc/pfree on every
call; aside from being faster this lets us get rid of some goto's, since
we no longer have any end-of-function pfree to do. Both ideas by Heikki.
Tom Lane [Fri, 21 Sep 2007 00:30:49 +0000 (00:30 +0000)]
Insert a hack in pl/tcl to disable Tcl's built-in Notifier subsystem, which
has a bad habit of launching multiple threads within the backend and thereby
causing all kinds of havoc. Fortunately, we don't need it, and recent Tcl
versions provide an easy way to disable it. Diagnosis and fix by
Steve Marshall, Paul Bayer, and Doug Knight of WSI Corporation.
Bruce Momjian [Thu, 20 Sep 2007 18:54:19 +0000 (18:54 +0000)]
Done:
> * -Consider shrinking expired tuples to just their headers
> * -Allow heap reuse of UPDATEd rows if no indexed columns are changed,
> and old and new versions are on the same heap page
Not needed anymore:
< * Reuse index tuples that point to heap tuples that are not visible to
< anyone?
Tom Lane [Thu, 20 Sep 2007 17:56:33 +0000 (17:56 +0000)]
HOT updates. When we update a tuple without changing any of its indexed
columns, and the new version can be stored on the same heap page, we no longer
generate extra index entries for the new version. Instead, index searches
follow the HOT-chain links to ensure they find the correct tuple version.
In addition, this patch introduces the ability to "prune" dead tuples on a
per-page basis, without having to do a complete VACUUM pass to recover space.
VACUUM is still needed to clean up dead index entries, however.
Pavan Deolasee, with help from a bunch of other people.
Neil Conway [Wed, 19 Sep 2007 22:31:48 +0000 (22:31 +0000)]
Prevent corr() from returning the wrong results for negative correlation
values. The previous coding essentially assumed that x = sqrt(x*x), which
does not hold for x < 0.
Thanks to Jie Zhang at Greenplum and Gavin Sherry for reporting this
issue.
Andrew Dunstan [Tue, 18 Sep 2007 17:41:17 +0000 (17:41 +0000)]
Close previously open holes for invalidly encoded data to enter the
database via builtin functions, as recently discussed on -hackers.
chr() now returns a character in the database encoding. For UTF8 encoded databases
the argument is treated as a Unicode code point. For other multi-byte encodings
the argument must designate a strict ascii character, or an error is raised,
as is also the case if the argument is 0.
ascii() is adjusted so that it remains the inverse of chr().
The two argument form of convert() is gone, and the three argument form now
takes a bytea first argument and returns a bytea. To cover this loss three new
functions are introduced:
. convert_from(bytea, name) returns text - converts the first argument from the
named encoding to the database encoding
. convert_to(text, name) returns bytea - converts the first argument from the
database encoding to the named encoding
. length(bytea, name) returns int - gives the length of the first argument in
characters in the named encoding
Tom Lane [Sun, 16 Sep 2007 16:33:04 +0000 (16:33 +0000)]
Remove Assert(BgWriterShmem != NULL), which is rather pointless since
we'd dump core anyway immediately afterward if it were null; and it
seems to confuse some versions of icc into generating bad code.
Per report from Sergey Koposov. Patched in HEAD only, for the moment,
since this is only likely to affect developers.
Tom Lane [Sun, 16 Sep 2007 15:56:20 +0000 (15:56 +0000)]
Fix overflow in extract(epoch from interval) for intervals exceeding 68 years.
Seems to have been introduced in 8.1 by careless SECS_PER_DAY
search-and-replace.
Tom Lane [Sun, 16 Sep 2007 02:37:46 +0000 (02:37 +0000)]
Fix aboriginal mistake in lazy VACUUM's code for truncating away
no-longer-needed pages at the end of a table. We thought we could throw away
pages containing HEAPTUPLE_DEAD tuples; but this is not so, because such
tuples very likely have index entries pointing at them, and we wouldn't have
removed the index entries. The problem only emerges in a somewhat unlikely
race condition: the dead tuples have to have been inserted by a transaction
that later aborted, and this has to have happened between VACUUM's initial
scan of the page and then rechecking it for empty in count_nondeletable_pages.
But that timespan will include an index-cleaning pass, so it's not all that
hard to hit. This seems to explain a couple of previously unsolved bug
reports.
Bruce Momjian [Fri, 14 Sep 2007 18:50:48 +0000 (18:50 +0000)]
Add:
>
> * Make it possible to use RETURNING together with conditional DO INSTEAD
> rules, such as for partitioning setups
>
> http://archives.postgresql.org/pgsql-hackers/2007-09/msg00577.php
Bruce Momjian [Fri, 14 Sep 2007 04:25:24 +0000 (04:25 +0000)]
Add tsearch tab completion support:
* adds a few missing words to some commands (like adding GIN as a valid
index type or OWNED BY for ALTER SEQUENCE,...)
* support for ALTER TABLE foo ENABLE/DISABLE REPLICA TRIGGER/RULE
* autocomplete CREATE DATABASE foo TEMPLATE (mostly done to prevent
conflicts with the TEMPLATE keyword for text search)
* support for ALTER/CREATE/DROP TEXT SEARCH as well as COMMENT ON TEXT
SEARCH and the corresponding psql backslash commands.
This proved a little more difficult than expected due to the fact that
words_after_create[] is used for two purposes - one is to provide a list
of words that follow immediatly after CREATE (or DROP) and the other
purpose is to use it for autocompleting anywhere in the statement if the
word in that struct is found with a query.
Since TEXT SEARCH CONFIGURATION|DICTIONARY|TEMPLATE|PARSER results in 3
words instead of one (as all the other words in that list are) I added a
flag to the struct to tell create_command_generator() to skip that entry
for autocompleting immediatly after CREATE which feels like a dirty
hack (but that holds true for a lot of code in tab-complete.c).
Tom Lane [Fri, 14 Sep 2007 03:25:31 +0000 (03:25 +0000)]
Remove ill-considered (not to mention undocumented) attempt to make
contrib/intarray's GIN opclass override the built-in default. Per bug #3048
and other complaints.
Tom Lane [Wed, 12 Sep 2007 22:10:26 +0000 (22:10 +0000)]
Redefine the lp_flags field of item pointers as having four states, rather
than two independent bits (one of which was never used in heap pages anyway,
or at least hadn't been in a very long time). This gives us flexibility to
add the HOT notions of redirected and dead item pointers without requiring
anything so klugy as magic values of lp_off and lp_len. The state values
are chosen so that for the states currently in use (pre-HOT) there is no
change in the physical representation.
Tom Lane [Tue, 11 Sep 2007 19:50:25 +0000 (19:50 +0000)]
Give ICC its preferred, documented form of the no-strict-aliasing
switch, viz '-fno-alias'. Since we have a separate code path here
anyway, it's just as easy to cooperate.
Tom Lane [Tue, 11 Sep 2007 17:15:33 +0000 (17:15 +0000)]
Make sure that open hash table scans are cleaned up when bgwriter tries to
recover from elog(ERROR). Problem was created by introduction of hash seq
search tracking awhile back, and affects all branches that have bgwriter;
in HEAD the disease has snuck into autovacuum and walwriter too. (Not sure
that the latter two use hash_seq_search at the moment, but surely they might
someday.) Per report from Sergey Koposov.
Tom Lane [Tue, 11 Sep 2007 16:17:46 +0000 (16:17 +0000)]
Include hash table name in all the internal-error elog messages in
dynahash.c. Sergey Koposov's current open problem shows the possible
usefulness of this, and it doesn't add much code.
Add regression tests for ispell, synonym and thesaurus dictionaries.
Rename synonym.syn.sample and thesaurs.ths.sample to
synonym_sample.syn and thesaurs_sample.ths accordingly to be able to use they
in regression test.