Tom Lane [Thu, 27 Aug 2009 20:08:03 +0000 (20:08 +0000)]
Modify the definition of window-function PARTITION BY and ORDER BY clauses
so that their elements are always taken as simple expressions over the
query's input columns. It originally seemed like a good idea to make them
act exactly like GROUP BY and ORDER BY, right down to the SQL92-era behavior
of accepting output column names or numbers. However, that was not such a
great idea, for two reasons:
1. It permits circular references, as exhibited in bug #5018: the output
column could be the one containing the window function itself. (We actually
had a regression test case illustrating this, but nobody thought twice about
how confusing that would be.)
2. It doesn't seem like a good idea for, eg, "lead(foo) OVER (ORDER BY foo)"
to potentially use two completely different meanings for "foo".
Accordingly, narrow down the behavior of window clauses to use only the
SQL99-compliant interpretation that the expressions are simple expressions.
Alvaro Herrera [Thu, 27 Aug 2009 17:18:44 +0000 (17:18 +0000)]
Fix handling of autovacuum reloptions.
In the original coding, setting a single reloption would cause default
values to be used for all the other reloptions. This is a problem
particularly for autovacuum reloptions.
Tom Lane [Thu, 27 Aug 2009 16:59:38 +0000 (16:59 +0000)]
Make it reasonably safe to use pg_ctl to start the postmaster from a boot-time
script.
To do this, have pg_ctl pass down its parent shell's PID in an environment
variable PG_GRANDPARENT_PID, and teach CreateLockFile() to disregard that PID
as a false match if it finds it in postmaster.pid. This allows us to cope
with one level of postgres-owned shell process even with pg_ctl in the way,
so it's just as safe as starting the postmaster directly. You still have to
be careful about how you write the initscript though.
Adjust the comments in contrib/start-scripts/ to not deprecate use of
pg_ctl. Also, fix the ROTATELOGS option in the OSX script, which was
indulging in exactly the sort of unsafe coding that renders this fix
pointless :-(. A pipe inside the "sudo" will probably result in more
than one postgres-owned process hanging around.
In the checkpoint written at the end of archive recovery, the WAL page header
was incorrectly initialized with timeline ID 0. That rendered the WAL page
unrecoverable, making a subsequent archive recovery stop at that point.
ThisTimeLineID needs to be initialized before calling AdvanceXLInsertBuffer().
This fixes bug #5011 reported by James Bardin. Backpatch to 8.4, as the bug
was introduced by the changes to use of bgwriter for writing the
end-of-archive-recovery checkpoint. Patch by Tom Lane.
Peter Eisentraut [Wed, 26 Aug 2009 22:24:44 +0000 (22:24 +0000)]
Update of install-sh, mkinstalldirs, and associated configury
Update install-sh to that from Autoconf 2.63, plus our Darwin-specific
changes (which I simplified a bit). install-sh is now able to install
multiple files in one run, so we could simplify our makefiles sometime.
install-sh also now has a -d option to create directories, so we don't need
mkinstalldirs anymore.
Use AC_PROG_MKDIR_P in configure.in, so we can use mkdir -p when available
instead of install-sh -d. For consistency with the rest of the world,
the corresponding make variable has been renamed from $(mkinstalldirs) to
$(MKDIR_P).
Peter Eisentraut [Tue, 25 Aug 2009 12:44:59 +0000 (12:44 +0000)]
Enhanced error context support in PL/Python
Extract the "while creating return value" and "while modifying trigger
row" parts of some error messages into another layer of error context.
This will simplify the upcoming patch to improve data type support, but
it can stand on its own.
Peter Eisentraut [Tue, 25 Aug 2009 08:14:42 +0000 (08:14 +0000)]
Use generic attribute management in PL/Python
Switch the implementation of the plan and result types to generic attribute
management, as described at <http://docs.python.org/extending/newtypes.html>.
This modernizes and simplifies the code a bit and prepares for Python 3.1,
where the old way doesn't work anymore.
Peter Eisentraut [Mon, 24 Aug 2009 20:25:25 +0000 (20:25 +0000)]
Make PL/Python tests more compatible with Python 3
This changes a bunch of incidentially used constructs in the PL/Python
regression tests to equivalent constructs in cases where Python 3 no longer
supports the old syntax. Support for older Python versions is unchanged.
Tom Lane [Mon, 24 Aug 2009 20:08:32 +0000 (20:08 +0000)]
Try to make silent_mode behave somewhat reasonably.
Instead of sending stdout/stderr to /dev/null after forking away from the
terminal, send them to postmaster.log within the data directory. Since
this opens the door to indefinite logfile bloat, recommend even more
strongly that log output be redirected when using silent_mode.
Move the postmaster's initial calls of load_hba() and load_ident() down
to after we have started the log collector, if we are going to. This
is so that errors reported by them will appear in the "usual" place.
Reclassify silent_mode as a LOGGING_WHERE, not LOGGING_WHEN, parameter,
since it's got absolutely nothing to do with the latter category.
In passing, fix some obsolete references to -S ... this option hasn't
had that switch letter for a long time.
Back-patch to 8.4, since as of 8.4 load_hba() and load_ident() are more
picky (and thus more likely to fail) than they used to be. This entire
change was driven by a complaint about those errors disappearing into
the bit bucket.
Alvaro Herrera [Mon, 24 Aug 2009 17:23:02 +0000 (17:23 +0000)]
Avoid calling kill() in a postmaster signal handler.
This causes problems when the system load is high, per report from Zdenek
Kotala in <1250860954.1239.114.camel@localhost>; instead of calling kill
directly, have the signal handler set a flag which is checked in ServerLoop.
This way, the handler can return before being called again by a subsequent
signal sent from the autovacuum launcher. Also, increase the sleep in the
launcher in this failure path to 1 second.
Backpatch to 8.3, which is when the signalling between autovacuum
launcher/postmaster was introduced.
Also, add a couple of ReleasePostmasterChildSlot calls in error paths; this
part backpatched to 8.4 which is when the child slot stuff was introduced.
Tom Lane [Mon, 24 Aug 2009 16:18:13 +0000 (16:18 +0000)]
Fix inclusions of readline/editline header files so that we only attempt to
#include the version of history.h that is in the same directory as the
readline.h we are using. This avoids problems in some scenarios where both
readline and editline are installed. Report and patch by Zdenek Kotala.
Tom Lane [Mon, 24 Aug 2009 03:10:16 +0000 (03:10 +0000)]
Run the "tablespace" regression test first not last. The former placement
renders useless one of the few test methodologies we have for WAL replay,
which is to intentionally crash the system just after completing the
regression tests and see if it recovers to the expected database state.
The reason is that DROP TABLESPACE forces a checkpoint, so there's essentially
no WAL available for replay after the tests complete.
Tom Lane [Mon, 24 Aug 2009 02:18:32 +0000 (02:18 +0000)]
Fix a violation of WAL coding rules in the recent patch to include an
"all tuples visible" flag in heap page headers. The flag update *must*
be applied before calling XLogInsert, but heap_update and the tuple
moving routines in VACUUM FULL were ignoring this rule. A crash and
replay could therefore leave the flag incorrectly set, causing rows
to appear visible in seqscans when they should not be. This might explain
recent reports of data corruption from Jeff Ross and others.
In passing, do a bit of editorialization on comments in visibilitymap.c.
Tom Lane [Sun, 23 Aug 2009 19:23:41 +0000 (19:23 +0000)]
Make TRUNCATE do truncate-in-place when processing a relation that was created
or previously truncated in the current (sub)transaction. This is safe since
if the (sub)transaction later rolls back, we'd just discard the rel's current
physical file anyway. This avoids unreasonable growth in the number of
transient files when a relation is repeatedly truncated. Per a performance
gripe a couple weeks ago from Todd Cook.
Tom Lane [Sun, 23 Aug 2009 18:26:08 +0000 (18:26 +0000)]
Tweak ExecIndexEvalRuntimeKeys to forcibly detoast any toasted comparison
values before they get passed to the index access method. This avoids
repeated detoastings that will otherwise ensue as the comparison value
is examined by various index support functions. We have seen a couple of
reports of cases where repeated detoastings result in an order-of-magnitude
slowdown, so it seems worth adding a bit of extra logic to prevent this.
I had previously proposed trying to avoid duplicate detoastings in general,
but this fix takes care of what seems the most important case in practice
with very little effort or risk.
Back-patch to 8.4 so that the PostGIS folk won't have to wait a year to
have this fix in a production release. (The issue exists further back,
of course, but the code's diverged enough to make backpatching further a
higher-risk action. Also it appears that the possible gains may be limited
in prior releases because of different handling of lossy operators.)
Tom Lane [Tue, 18 Aug 2009 23:40:20 +0000 (23:40 +0000)]
Allow mixing of traditional and SQL:2008 LIMIT/OFFSET syntax. Being rigid
about it doesn't simplify the grammar at all, and it does invite confusion
among those who only read the SELECT syntax summary and not the full details.
Per gripe from Jaime Casanova.
Tom Lane [Tue, 18 Aug 2009 21:23:14 +0000 (21:23 +0000)]
Fix overflow for INTERVAL 'x ms' where x is more than a couple million,
and integer datetimes are in use. Per bug report from Hubert Depesz
Lubaczewski.
Tom Lane [Tue, 18 Aug 2009 16:00:50 +0000 (16:00 +0000)]
Print the actual DB encoding in the unaccent regression test.
This is to help make it more obvious what the problem is, if the
encoding isn't what the test expects.
Teodor Sigaev [Tue, 18 Aug 2009 10:30:41 +0000 (10:30 +0000)]
Introduce filtering dictionary support to tsearch. Propagate --nolocale option
to CREATE DATABASE command in pg_regress to allow correct checking of
locale-sensitive contrib modules.
Peter Eisentraut [Mon, 17 Aug 2009 21:29:30 +0000 (21:29 +0000)]
Make version.sgml depend on configure instead of Makefile.global. This
cheats a bit, but it avoids unsatisfied dependencies in distribution
tarballs. (found by make distcheck)
Tom Lane [Mon, 17 Aug 2009 20:34:31 +0000 (20:34 +0000)]
Department of marginal improvements: teach tupconvert.c to avoid doing a
physical conversion when there are dropped columns in the same places in
the input and output tupdescs. This avoids possible performance loss from
the recent patch to improve dropped-column handling, in some cases where
the old code would have worked.
Tom Lane [Sun, 16 Aug 2009 19:55:21 +0000 (19:55 +0000)]
Fix imprecise documentation of random(): it never returns 1.0.
This was changed in 8.2 but the documentation was not corrected.
Per gripe from Sam Mason.
Tom Lane [Sun, 16 Aug 2009 18:14:34 +0000 (18:14 +0000)]
Fix incorrect encoding-aware name truncation in makeArrayTypeName().
truncate_identifier won't do anything if the passed-in strlen is already
less than NAMEDATALEN, which it always would be given the strlcpy usage.
This has been broken since the arrays-of-composite-types code went in.
Arguably truncate_identifier is suffering from excessive optimization
and should always process the string, but for the moment I'll take the
more localized patch.
Tom Lane [Sat, 15 Aug 2009 16:16:01 +0000 (16:16 +0000)]
Remove Wisconsin benchmark files.
This test is clearly not being used anymore, since it's been broken for
long periods of time without anyone noticing. Per discussion, it's not
worth keeping in our source tree.
Peter Eisentraut [Fri, 14 Aug 2009 13:42:16 +0000 (13:42 +0000)]
PL/Python regression tests for data type handling
Add some checks on various data types are converted into and out of Python.
This is extracted from Caleb Welton's patch for improved bytea support,
but much expanded.
Peter Eisentraut [Fri, 14 Aug 2009 13:12:21 +0000 (13:12 +0000)]
Domain support in PL/Python
When examining what Python type to convert a PostgreSQL type to on input,
look at the base type of the input type, otherwise all domains end up
defaulting to string.
Tom Lane [Thu, 13 Aug 2009 16:53:09 +0000 (16:53 +0000)]
Put back adjust_appendrel_attrs()'s code for dealing with RestrictInfo.
I mistakenly removed it last month, thinking it was no longer needed ---
but it is still needed for dealing with joininfo lists. Fortunately this
bit of brain fade hadn't made it into any released versions yet.
Tom Lane [Wed, 12 Aug 2009 23:00:12 +0000 (23:00 +0000)]
Improve error message for the case where a requested foreign key constraint
does match some unique index on the referenced table, but that index is
only deferrably unique. We were doing this nicely for the
default-to-primary-key case, but were being lazy for the other case.
Tom Lane [Wed, 12 Aug 2009 20:53:31 +0000 (20:53 +0000)]
Allow backends to start up without use of the flat-file copy of pg_database.
To make this work in the base case, pg_database now has a nailed-in-cache
relation descriptor that is initialized using hardwired knowledge in
relcache.c. This means pg_database is added to the set of relations that
need to have a Schema_pg_xxx macro maintained in pg_attribute.h. When this
path is taken, we'll have to do a seqscan of pg_database to find the row
we need.
In the normal case, we are able to do an indexscan to find the database's row
by name. This is made possible by storing a global relcache init file that
describes only the shared catalogs and their indexes (and therefore is usable
by all backends in any database). A new backend loads this cache file,
finds its database OID after an indexscan on pg_database, and then loads
the local relcache init file for that database.
This change should effectively eliminate number of databases as a factor
in backend startup time, even with large numbers of databases. However,
the real reason for doing it is as a first step towards getting rid of
the flat files altogether. There are still several other sub-projects
to be tackled before that can happen.
Tom Lane [Wed, 12 Aug 2009 18:23:49 +0000 (18:23 +0000)]
Fix old bug in log_autovacuum_min_duration code: it was relying on being able
to access a Relation entry it had just closed. I happened to be testing with
CLOBBER_CACHE_ALWAYS, which made this a guaranteed core dump (at least on
machines where sprintf %s isn't forgiving of a NULL pointer). It's probably
quite unlikely that it would fail in the field, but a bug is a bug. Fix by
moving the relation_close call down past the logging action.
Peter Eisentraut [Wed, 12 Aug 2009 16:37:26 +0000 (16:37 +0000)]
Split the plpython regression test into test cases arranged by topic, instead
of the previous monolithic setup-create-run sequence, that was apparently
inherited from a previous test infrastructure, but makes working with the
tests and adding new ones weird.
Peter Eisentraut [Wed, 12 Aug 2009 16:32:35 +0000 (16:32 +0000)]
Document the minimum required Python version.
It turns out that Python 2.2 is the oldest version that PL/Python compiles
with, apparently related to the introduction of iterators. Might as well
document this.
Tom Lane [Mon, 10 Aug 2009 16:10:19 +0000 (16:10 +0000)]
Adjust extract(epoch) example to clarify that it includes fractional
seconds, per gripe from Richard Neill. Also, add a cross-reference to
the to_timestamp function.
Documentation files in HTML and man formats are now prepared for
distribution using the distprep make target, like everything else. They
are placed in doc/src/sgml/html and manX and installed from there by
make install, if present. The business with the tarballs in the tarball
is gone.
Tom Lane [Fri, 7 Aug 2009 22:48:34 +0000 (22:48 +0000)]
Modify parallel pg_restore to track pending and ready items by means of
two new lists, rather than repeatedly rescanning the main TOC list.
This avoids a potential O(N^2) slowdown, although you'd need a *lot*
of tables to make that really significant; and it might simplify future
improvements in the scheduling algorithm by making the set of ready
items more easily inspectable. The original thought that it would
in itself result in a more efficient job dispatch order doesn't seem
to have been borne out in testing, but it seems worth doing anyway.
Test coverage support now covers the entire source tree, including
contrib, instead of just src/backend. In a related but independent
development, the commands make coverage and make coverage-html can be run
in any directory.
This turned out to be much easier than feared. Besides a few ad hoc fixes
to pass the make target down the tree, change all affected makefiles to
list their directories in the SUBDIRS variable, changed from variants like
DIRS and WANTED_DIRS. MSVC build fix was attempted as well.
Tom Lane [Fri, 7 Aug 2009 20:16:11 +0000 (20:16 +0000)]
Try to defend against the possibility that libpq is still in COPY_IN state
when we reach the post-COPY "pump it dry" error recovery code that was added
2006-11-24. Per a report from Neil Best, there is at least one code path
in which this occurs, leading to an infinite loop in code that's supposed
to be making it more robust not less so. A reasonable response seems to be
to call PQputCopyEnd() again, so let's try that.
Back-patch to all versions that contain the cleanup loop.
Tom Lane [Fri, 7 Aug 2009 19:29:49 +0000 (19:29 +0000)]
rm_cleanup functions need to be allowed to write WAL entries. This oversight
appears to explain the recent reports of "PANIC: cannot make new WAL entries
during recovery".
Fast shutdown stop should forcibly disconnect any active backends, even
if a smart shutdown is already in progress. Backpatch to 8.3, this was broken
in the patch that introduced "dead-end backends".
Per report by Itagaki Takahiro, patch by Fujii Masao.
Tom Lane [Thu, 6 Aug 2009 20:44:32 +0000 (20:44 +0000)]
Improve plpgsql's ability to cope with rowtypes containing dropped columns,
by supporting conversions in places that used to demand exact rowtype match.
Since this issue is certain to come up elsewhere (in fact, already has,
in ExecEvalConvertRowtype), factor out the support code into new core
functions for tuple conversion. I chose to put these in a new source
file since heaptuple.c is already overly long.
Heavily revised version of a patch by Pavel Stehule.
Magnus Hagander [Thu, 6 Aug 2009 09:50:22 +0000 (09:50 +0000)]
Avoid terminating the postmaster on a number of "can't happen" cases during
backend startup on Win32. Instead, log the error and just forget about
the potentially dangling process, since we can't do anything about it anyway.
Improve error messages in md.c. When a filesystem operation like open() or
fsync() fails, say "file" rather than "relation" when printing the filename.
This makes messages that display block numbers a bit confusing. For example,
in message 'could not read block 150000 of file "base/1234/5678.1"', 150000
is the block number from the beginning of the relation, ie. segment 0, not
150000th block within that segment. Per discussion, users aren't usually
interested in the exact location within the file, so we can live with that.
To ease constructing error messages, add FilePathName(File) function to
return the pathname of a virtual fd.
Joe Conway [Wed, 5 Aug 2009 16:11:07 +0000 (16:11 +0000)]
Implement dblink_get_notify().
Adds the ability to retrieve async notifications using dblink,
via the addition of the function dblink_get_notify(). Original patch
by Marcus Kempe, suggestions by Tom Lane and Alvaro Herrera, patch
review and adjustments by Joe Conway.
This switches the man page building process to use the DocBook XSL stylesheet
toolchain. The previous targets for Docbook2X are removed. configure has been
updated to look for the new tools. The Documentation appendix contains the
new build instructions. There are also a few isolated tweaks in the
documentation to improve places that came out strangely in the man pages.
Tom Lane [Tue, 4 Aug 2009 21:56:09 +0000 (21:56 +0000)]
Fix pg_dump to do the right thing when escaping the contents of large objects.
The previous implementation got it right in most cases but failed in one:
if you pg_dump into an archive with standard_conforming_strings enabled, then
pg_restore to a script file (not directly to a database), the script will set
standard_conforming_strings = on but then emit large object data as
nonstandardly-escaped strings.
At the moment the code is made to emit hex-format bytea strings when dumping
to a script file. We might want to change to old-style escaping for backwards
compatibility, but that would be slower and bulkier. If we do, it's just a
matter of reimplementing appendByteaLiteral().
This has been broken for a long time, but given the lack of field complaints
I'm not going to worry about back-patching.