Tom Lane [Thu, 17 Jun 2010 16:41:25 +0000 (16:41 +0000)]
Don't allow walsender to send WAL data until it's been safely fsync'd on the
master. Otherwise a subsequent crash could cause the master to lose WAL that
has already been applied on the slave, resulting in the slave being out of
sync and soon corrupt. Per recent discussion and an example from Robert Haas.
Peter Eisentraut [Wed, 16 Jun 2010 00:54:16 +0000 (00:54 +0000)]
Refactor sprintf calls with computed format strings into multiple calls with
constant format strings, so that the compiler can more easily check the
formats for correctness.
Tom Lane [Tue, 15 Jun 2010 20:29:01 +0000 (20:29 +0000)]
Change the interpretation of the primary_key_attnums parameter of
dblink_build_sql_insert() and related functions. Now the column numbers
are treated as logical not physical column numbers. This will provide saner
behavior in the presence of dropped columns; furthermore, if we ever get
around to allowing rearrangement of logical column ordering, the original
definition would become nearly untenable from a usability standpoint.
Per recent discussion of dblink's handling of dropped columns.
Not back-patched for fear of breaking existing applications.
Tom Lane [Tue, 15 Jun 2010 19:04:15 +0000 (19:04 +0000)]
Fix dblink_build_sql_insert() and related functions to handle dropped
columns correctly. In passing, get rid of some dead logic in the
underlying get_sql_insert() etc functions --- there is no caller that
will pass null value-arrays to them.
Robert Haas [Tue, 15 Jun 2010 18:43:35 +0000 (18:43 +0000)]
Document new 9.0 behavior of ANALYZE on inheritance hierarchies.
In particular, note that autovacuum does not yet understand that it might
need to vacuum inheritance parents as a result of changes to the child
tables.
Tom Lane [Tue, 15 Jun 2010 16:22:19 +0000 (16:22 +0000)]
Consolidate and improve checking of key-column-attnum arguments for
dblink_build_sql_insert() and related functions. In particular, be sure to
reject references to dropped and out-of-range column numbers. The numbers
are still interpreted as physical column numbers, though, for backward
compatibility.
This patch replaces Joe's patch of 2010-02-03, which handled only some aspects
of the problem.
Itagaki Takahiro [Tue, 15 Jun 2010 07:52:11 +0000 (07:52 +0000)]
Add new GUC categories corresponding to sections in docs, and move
description for vacuum_defer_cleanup_age to the correct category.
Sections in postgresql.conf are also sorted in the same order with docs.
Per gripe by Fujii Masao, suggestion by Heikki Linnakangas, and patch by me.
Tom Lane [Mon, 14 Jun 2010 20:49:33 +0000 (20:49 +0000)]
Rearrange dblink's dblink_build_sql_insert() and related routines to open and
lock the target relation just once per SQL function call. The original coding
obtained and released lock several times per call. Aside from saving a
not-insignificant number of cycles, this eliminates possible race conditions
if someone tries to modify the relation's schema concurrently. Also
centralize locking and permission-checking logic.
Problem noted while investigating a trouble report from Robert Voinea --- his
problem is still to be fixed, though.
If a corrupt WAL record is received by streaming replication, disconnect
and retry. If the record is genuinely corrupt in the master database,
there's little hope of recovering, but it's better than simply retrying
to apply the corrupt WAL record in a tight loop without even trying to
retransmit it, which is what we used to do.
Itagaki Takahiro [Mon, 14 Jun 2010 00:49:24 +0000 (00:49 +0000)]
Remove max_standby_delay message from ps display of recovery process
in waiting status. The parameter is not so interesting in ps display
because it is referable in postgresql.conf.
Use "replication" as the database name when constructing a connection
string for a streaming replication connection. It's ignored by the
server, but allows libpq to pick up the password from .pgpass where
"replication" is specified as the database name.
Patch by Fujii Masao per Tom's suggestion, with some wording changes by me.
Return NULL instead of 0/0 in pg_last_xlog_receive_location() and
pg_last_xlog_replay_location(). Per Robert Haas's suggestion, after
Itagaki Takahiro pointed out an issue in the docs. Also, some wording
changes in the docs by me.
Robert Haas [Thu, 10 Jun 2010 01:26:30 +0000 (01:26 +0000)]
Quote all string values in EXPLAIN (FORMAT YAML) output.
While my previous attempt seems to always produce valid YAML, it
doesn't always produce YAML that means what it appears to mean,
because of tokens like "0xa" and "true", which without quotes will
be interpreted as integer or Boolean literals. So, instead, just
quote everything that's not known to be a number, as we do for
JSON.
Dean Rasheed, with some changes to the comments by me.
In standby mode, respect checkpoint_segments in addition to
checkpoint_timeout to trigger restartpoints. We used to deliberately only
do time-based restartpoints, because if checkpoint_segments is small we
would spend time doing restartpoints more often than really necessary.
But now that restartpoints are done in bgwriter, they're not as
disruptive as they used to be. Secondly, because streaming replication
stores the streamed WAL files in pg_xlog, we want to clean it up more
often to avoid running out of disk space when checkpoint_timeout is large
and checkpoint_segments small.
Patch by Fujii Masao, with some minor changes by me.
Magnus Hagander [Wed, 9 Jun 2010 10:54:45 +0000 (10:54 +0000)]
Make the walwriter close it's handle to an old xlog segment if it's no longer
the current one. Not doing this would leave the walwriter with a handle to a
deleted file if there was nothing for it to do for a long period of time,
preventing the file from being completely removed.
Reported by Tollef Fog Heen, and thanks to Heikki for some hand-holding with
the patch.
Avoid "identifier will be truncated" warning in dblink
when connection string is longer than NAMEDATALEN.
The previous fix for long connection name broke the behavior.
Robert Haas [Wed, 9 Jun 2010 02:39:34 +0000 (02:39 +0000)]
Attempt to fix EXPLAIN (FORMAT YAML) quoting to behave sanely.
The previous code failed to quote in many cases where quoting was necessary -
YAML has loads of special characters, including -:[]{},"'|*& - so quote much
more aggressively, and only refrain from quoting things where it seems fairly
clear that it isn't necessary.
Ensure default-only storage parameters for TOAST relations
to be initialized with proper values. Affected parameters are
fillfactor, analyze_threshold, and analyze_scale_factor.
Especially uninitialized fillfactor caused inefficient page usage
because we built a StdRdOptions struct in which fillfactor is zero
if any reloption is set for the toast table.
In addition, we disallow toast.autovacuum_analyze_threshold and
toast.autovacuum_analyze_scale_factor because we didn't actually
support them; they are always ignored.
Report by Rumko on pgsql-bugs on 12 May 2010.
Analysis by Tom Lane and Alvaro Herrera. Patch by me.
Replace "slave" to "standby" in documentation for consistent terminology.
Almost all of the terms in docs and messages were replaced, but still
remains in a few comments and README files in codes.
Tom Lane [Thu, 3 Jun 2010 23:00:14 +0000 (23:00 +0000)]
Adjust misleading comment in walsender.c. We try to send all WAL data that's
been written out from shared memory, but the previous phrasing might be read
to say that we send only what's been fsync'd.
Tom Lane [Thu, 3 Jun 2010 22:17:32 +0000 (22:17 +0000)]
Add current WAL end (as seen by walsender, ie, GetWriteRecPtr() result)
and current server clock time to SR data messages. These are not currently
used on the slave side but seem likely to be useful in future, and it'd be
better not to change the SR protocol after release. Per discussion.
Also do some minor code review and cleanup on walsender.c, and improve the
protocol documentation.
Fix dblink to treat connection names longer than NAMEDATALEN-2 (62 bytes).
Now long names are adjusted with truncate_identifier() and NOTICE messages
are raised if names are actually truncated.
Tom Lane [Sun, 30 May 2010 21:59:02 +0000 (21:59 +0000)]
Fix misuse of Lossy Counting (LC) algorithm in compute_tsvector_stats().
We must filter out hashtable entries with frequencies less than those
specified by the algorithm, else we risk emitting junk entries whose
actual frequency is much less than other lexemes that did not get
tabulated. This is bad enough by itself, but even worse is that
tsquerysel() believes that the minimum frequency seen in pg_statistic is a
hard upper bound for lexemes not included, and was thus underestimating
the frequency of non-MCEs.
Also, set the threshold frequency to something with a little bit of theory
behind it, to wit assume that the input distribution is approximately
Zipfian. This might need adjustment in future, but some preliminary
experiments suggest that it's not too unreasonable.
Back-patch to 8.4, where this code was introduced.