Tom Lane [Fri, 8 Aug 2008 17:01:34 +0000 (17:01 +0000)]
Install checks in executor startup to ensure that the tuples produced by an
INSERT or UPDATE will match the target table's current rowtype. In pre-8.3
releases inconsistency can arise with stale cached plans, as reported by
Merlin Moncure. (We patched the equivalent hazard on the SELECT side in Feb
2007; I'm not sure why we thought there was no risk on the insertion side.)
In 8.3 and HEAD this problem should be impossible due to plan cache
invalidation management, but it seems prudent to make the check anyway.
Back-patch as far as 8.0. 7.x versions lack ALTER COLUMN TYPE, so there
seems no way to abuse a stale plan comparably.
Tom Lane [Tue, 5 Aug 2008 21:28:48 +0000 (21:28 +0000)]
Do not allow Unique nodes to be scanned backwards. The code claimed that it
would work, but in fact it didn't return the same rows when moving backwards
as when moving forwards. This would have no visible effect in a DISTINCT
query (at least assuming the column datatypes use a strong definition of
equality), but it gave entirely wrong answers for DISTINCT ON queries.
Tom Lane [Mon, 28 Jul 2008 18:45:05 +0000 (18:45 +0000)]
Update 8.1 and 8.0 plpython to work with Python 2.5. This backports several
fixes made during the 8.2 development cycle, but not backported at the time
for lack of confidence in the new coding.
I didn't touch 7.4 because it has more problems than this: the configure
probe for Python fails.
Tom Lane [Thu, 10 Jul 2008 01:17:52 +0000 (01:17 +0000)]
Fix mis-calculation of extParam/allParam sets for plan nodes, as seen in
bug #4290. The fundamental bug is that masking extParam by outer_params,
as finalize_plan had been doing, caused us to lose the information that
an initPlan depended on the output of a sibling initPlan. On reflection
the best thing to do seemed to be not to try to adjust outer_params for
this case but get rid of it entirely. The only thing it was really doing
for us was to filter out param IDs associated with SubPlan nodes, and that
can be done (with greater accuracy) while processing individual SubPlan
nodes in finalize_primnode. This approach was vindicated by the discovery
that the masking method was hiding a second bug: SS_finalize_plan failed to
remove extParam bits for initPlan output params that were referenced in the
main plan tree (it only got rid of those referenced by other initPlans).
It's not clear that this caused any real problems, given the limited use
of extParam by the executor, but it's certainly not what was intended.
I originally thought that there was also a problem with needing to include
indirect dependencies on external params in initPlans' param sets, but it
turns out that the executor handles this correctly so long as the depended-on
initPlan is earlier in the initPlans list than the one using its output.
That seems a bit of a fragile assumption, but it is true at the moment,
so I just documented it in some code comments rather than making what would
be rather invasive changes to remove the assumption.
Back-patch to 8.1. Previous versions don't have the case of initPlans
referring to other initPlans' outputs, so while the existing logic is still
questionable for them, there are not any known bugs to be fixed. So I'll
refrain from changing them for now.
Tom Lane [Tue, 8 Jul 2008 22:18:02 +0000 (22:18 +0000)]
Fix performance bug in write_syslog(): the code to preferentially break the
log message at newlines cost O(N^2) for very long messages with few or no
newlines. For messages in the megabyte range this became the dominant cost.
Per gripe from Achilleas Mantzios.
Patch all the way back, since this is a safe change with no portability
risks. I am also thinking of increasing PG_SYSLOG_LIMIT, but that should
be done separately.
Tom Lane [Mon, 7 Jul 2008 20:25:22 +0000 (20:25 +0000)]
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean
results always contribute two groups, regardless of the expression contents.
This is very substantially more accurate than the regular heuristic for
certain boolean tests like "col IS NULL". Per gripe from Sam Mason.
Back-patch to all supported releases, since the behavior of
estimate_num_groups() hasn't changed all that much since 7.4.
Tom Lane [Mon, 7 Jul 2008 18:10:12 +0000 (18:10 +0000)]
Fix AT TIME ZONE (in all three variants) so that we first try to interpret
the timezone argument as a timezone abbreviation, and only try it as a full
timezone name if that fails. The zic database has four zones (CET, EET, MET,
WET) that are full daylight-savings zones and yet have names that are the
same as their abbreviations for standard time, resulting in ambiguity.
In the timestamp input functions we resolve the ambiguity by preferring the
abbreviation, and AT TIME ZONE should work the same way. (No functionality
is lost because the zic database also has other names for these zones, eg
Europe/Zurich.) Per gripe from Jaromir Talir.
Backpatch to 8.1. Older releases did not have the issue because AT TIME ZONE
only accepted abbreviations not zone names. (Thus, this patch also arguably
fixes a compatibility botch introduced at 8.1: in ambiguous cases we now
behave the same as 8.0 did.)
Tom Lane [Tue, 1 Jul 2008 03:41:18 +0000 (03:41 +0000)]
Fix identify_system_timezone() so that it tests the behavior of the system
timezone setting in the current year and for 100 years back, rather than
always examining years 1904-2004. The original coding would have problems
distinguishing zones whose behavior diverged only after 2004; which is a
situation we will surely face sometime, if it's not out there already.
In passing, also prevent selection of the dummy "Factory" timezone, even
if that's exactly what the system is using. Reporting time as GMT seems
better than that.
Tom Lane [Tue, 17 Jun 2008 00:53:04 +0000 (00:53 +0000)]
Clean up a number of bogosities around pltcl's handling of the Tcl "result":
1. Directly reading interp->result is deprecated in Tcl 8.0 and later;
you're supposed to use Tcl_GetStringResult. This code finally broke with
Tcl 8.5, because Tcl_GetVar can now have side-effects on interp->result even
though it preserves the logical state of the result. (There's arguably a
Tcl issue here, because Tcl_GetVar could invalidate the pointer result of a
just-preceding Tcl_GetStringResult, but I doubt the Tcl guys will see it as
a bug.)
2. We were being sloppy about the encoding of the result: some places would
push database-encoding data into the Tcl result, which should not happen,
and we were assuming that any error result coming back from Tcl was in the
database encoding, which is not a good assumption.
3. There were a lot of calls of Tcl_SetResult that uselessly specified
TCL_VOLATILE for constant strings. This is only a minor performance issue,
but I fixed it in passing since I had to look at all the calls anyway.
#2 is a live bug regardless of which Tcl version you are interested in,
so back-patch even to branches that are unlikely to be used with Tcl 8.5.
I went back as far as 8.0, which is as far as the patch applied easily;
7.4 was using a different error processing scheme that has got its own
problems :-(
Tom Lane [Mon, 9 Jun 2008 19:34:24 +0000 (19:34 +0000)]
Fix datetime input functions to correctly detect integer overflow when
running on a 64-bit platform ... strtol() will happily return 64-bit
output in that case. Per bug #4231 from Geoff Tolley.
Tom Lane [Sun, 8 Jun 2008 21:10:05 +0000 (21:10 +0000)]
ALTER AGGREGATE OWNER seems to have been missed by the last couple of
patches that dealt with object ownership. It wasn't updating pg_shdepend
nor adjusting the aggregate's ACL. In 8.2 and up, fix this permanently
by making it use AlterFunctionOwner_oid. In 8.1, the function code wasn't
factored that way, so just copy and paste.
Tom Lane [Fri, 6 Jun 2008 17:59:53 +0000 (17:59 +0000)]
Fix pg_get_ruledef() so that negative numeric constants are parenthesized.
This is needed because :: casting binds more tightly than minus, so for
example -1::integer is not the same as (-1)::integer, and there are cases
where the difference is important. In particular this caused a failure
in SELECT DISTINCT ... ORDER BY ... where expressions that should have
matched were seen as different by the parser; but I suspect that there
could be other cases where failure to parenthesize leads to subtler
semantic differences in reloaded rules. Per report from Alexandr Popov.
Tom Lane [Wed, 28 May 2008 00:45:59 +0000 (00:45 +0000)]
Improve GRANT documentation to point out that UPDATE and DELETE typically
require SELECT privilege as well, since you normally need to read existing
column values within such commands. This behavior is according to spec,
but we'd never documented it before. Per gripe from Volkan Yazici.
Tom Lane [Tue, 27 May 2008 21:13:39 +0000 (21:13 +0000)]
Back-patch the 8.3 fix that prohibits TRUNCATE, CLUSTER, and REINDEX when the
current transaction has any open references to the target relation or index
(implying it has an active query using the relation). Also back-patch the
8.2 fix that prohibits TRUNCATE and CLUSTER when there are pending
AFTER-trigger events. Per suggestion from Heikki.
Tom Lane [Mon, 26 May 2008 18:54:50 +0000 (18:54 +0000)]
Fix an old corner-case bug in set_config_option: push_old_value has to be
called before, not after, calling the assign_hook if any. This is because
push_old_value might fail (due to palloc out-of-memory), and in that case
there would be no stack entry to tell transaction abort to undo the GUC
assignment. Of course the actual assignment to the GUC variable hasn't
happened yet --- but the assign_hook might have altered subsidiary state.
Without a stack entry we won't call it again to make it undo such actions.
So this is necessary to make the world safe for assign_hooks with side
effects. Per a discussion a couple weeks ago with Magnus.
Back-patch to 8.0. 7.x did not have the problem because it did not have
allocatable stacks of GUC values.
Tom Lane [Sun, 25 May 2008 21:51:23 +0000 (21:51 +0000)]
Adjust timestamp regression tests to prevent two low-probability failure
cases. Recent buildfarm experience shows that it is sometimes possible
to execute several SQL commands in less time than the granularity of
Windows' not-very-high-resolution gettimeofday(), leading to a failure
because the tests expect the value of now() to change and it doesn't.
Also, it was recognized some time ago that the same area of the tests
could fail if local midnight passes between the insertion and the checking
of the values for 'yesterday', 'tomorrow', etc. Clean all this up per
ideas from myself and Greg Stark.
There remains a window for failure if the transaction block is entered
exactly at local midnight (so that 'now' and 'today' have the same value),
but that seems low-probability enough to live with.
Since the point of this change is mostly to eliminate buildfarm noise,
back-patch to all versions we are still actively testing.
Remove arbitrary 10MB limit on two-phase state file size. It's not that hard
to go beoynd 10MB, as demonstrated by Gavin Sharry's example of dropping a
schema with ~25000 objects. The really bogus thing about the limit was that
it was enforced when a state file file was read in, not when it was written,
so you would end up with a prepared transaction that you can't commit or
abort, and the only recourse was to shut down the server and remove the file
by hand.
Raise the limit to MaxAllocSize, and enforce it also when a state file is
written. We could've removed the limit altogether, but reading in a file
larger than MaxAllocSize would fail anyway because we read it into a
palloc'd buffer.
Backpatch down to 8.1, where 2PC and this issue was introduced.
Tom Lane [Fri, 9 May 2008 22:37:47 +0000 (22:37 +0000)]
Fix an ancient oversight in change_varattnos_of_a_node: it neglected to update
varoattno along with varattno. This resulted in having Vars that were not
seen as equal(), causing inheritance of the "same" constraint from different
parent relations to fail. An example is
Tom Lane [Thu, 24 Apr 2008 20:18:15 +0000 (20:18 +0000)]
Fix ALTER TABLE ADD COLUMN ... PRIMARY KEY so that the new column is correctly
checked to see if it's been initialized to all non-nulls. The implicit NOT
NULL constraint was not being checked during the ALTER (in fact, not even if
there was an explicit NOT NULL too), because ATExecAddColumn neglected to
set the flag needed to make the test happen. This has been broken since
the capability was first added, in 8.0.
Magnus Hagander [Mon, 21 Apr 2008 09:45:05 +0000 (09:45 +0000)]
Add link to major version release notes at the top of the minor
version ones, to make it clear to users just browsing the notes
that there are a lot more changes available from whatever version
they are at than what's in the minor version release notes.
Tom Lane [Fri, 11 Apr 2008 22:53:16 +0000 (22:53 +0000)]
Fix several datatype input functions that were allowing unused bytes in their
results to contain uninitialized, unpredictable values. While this was okay
as far as the datatypes themselves were concerned, it's a problem for the
parser because occurrences of the "same" literal might not be recognized as
equal by datumIsEqual (and hence not by equal()). It seems sufficient to fix
this in the input functions since the only critical use of equal() is in the
parser's comparisons of ORDER BY and DISTINCT expressions.
Per a trouble report from Marc Cousin.
Patch all the way back. Interestingly, array_in did not have the bug before
8.2, which may explain why the issue went unnoticed for so long.
Tom Lane [Sat, 5 Apr 2008 01:58:42 +0000 (01:58 +0000)]
Defend against JOINs having more than 32K columns altogether. We cannot
currently support this because we must be able to build Vars referencing
join columns, and varattno is only 16 bits wide. Perhaps this should be
improved in future, but considering that it never came up before, I'm not
sure the problem is worth much effort. Per bug #4070 from Marcello
Ceschia.
The problem seems largely academic in 8.0 and 7.4, because they have
(different) O(N^2) performance issues with such wide joins, but
back-patch all the way anyway.
Tom Lane [Mon, 31 Mar 2008 01:32:48 +0000 (01:32 +0000)]
Fix a number of places that were making file-type tests infelicitously.
The places that did, eg,
(statbuf.st_mode & S_IFMT) == S_IFDIR
were correct, but there is no good reason not to use S_ISDIR() instead,
especially when that's what the other 90% of our code does. The places
that did, eg,
(statbuf.st_mode & S_IFDIR)
were flat out *wrong* and would fail in various platform-specific ways,
eg a symlink could be mistaken for a regular file on most Unixen.
The actual impact of this is probably small, since the problem cases
seem to always involve symlinks or sockets, which are unlikely to be
found in the directories that PG code might be scanning. But it's
clearly trouble waiting to happen, so patch all the way back anyway.
(There seem to be no occurrences of the mistake in 7.4.)
Tom Lane [Tue, 25 Mar 2008 19:31:40 +0000 (19:31 +0000)]
Adjust DatumGetBool macro so that it isn't fooled by garbage in the Datum
to the left of the actual bool value. While in most cases there won't be
any, our support for old-style user-defined functions violates the C spec
to the extent of calling functions that might return char or short through
a function pointer declared to return "char *", which we then coerce to
Datum. It is not surprising that the result might contain garbage
high-order bits ... what is surprising is that we didn't see such cases
long ago. Per report from Magnus.
This is a back-patch of a change that was made in HEAD almost exactly
a year ago. I had refrained from back-patching at the time, but now
we find that this is *necessary* for contrib to work with gcc 4.3.
Add the missing cyrillic "Yo" characters ('e' and 'E' with two dots) to the
ISO_8859-5 <-> MULE_INTERNAL conversion tables.
This was discovered when trying to convert a string containing those characters
from ISO_8859-5 to Windows-1251, because we use MULE_INTERNAL/KOI8R as an
intermediate encoding between those two.
While the missing "Yo" was just an omission in the conversion tables, there are
a few other characters like the "Numero" sign ("No" as a single character) that
exists in all the other cyrillic encodings (win1251, ISO_8859-5 and cp866), but
not in KOI8R. Added comments about that.
Tom Lane [Wed, 19 Mar 2008 02:41:00 +0000 (02:41 +0000)]
Fix regexp substring matching (substring(string from pattern)) for the corner
case where there is a match to the pattern overall but the user has specified
a parenthesized subexpression and that subexpression hasn't got a match.
An example is substring('foo' from 'foo(bar)?'). This should return NULL,
since (bar) isn't matched, but it was mistakenly returning the whole-pattern
match instead (ie, 'foo'). Per bug #4044 from Rui Martins.
This has been broken since the beginning; patch in all supported versions.
The old behavior was sufficiently inconsistent that it's impossible to believe
anyone is depending on it.
Tom Lane [Wed, 12 Mar 2008 20:12:14 +0000 (20:12 +0000)]
Fix LISTEN/NOTIFY race condition reported by Laurent Birtz, by postponing
pg_listener modifications commanded by LISTEN and UNLISTEN until the end
of the current transaction. This allows us to hold the ExclusiveLock on
pg_listener until after commit, with no greater risk of deadlock than there
was before. Aside from fixing the race condition, this gets rid of a
truly ugly kludge that was there before, namely having to ignore
HeapTupleBeingUpdated failures during NOTIFY. There is a small potential
incompatibility, which is that if a transaction issues LISTEN or UNLISTEN
and then looks into pg_listener before committing, it won't see any resulting
row insertion or deletion, where before it would have. It seems unlikely
that anyone would be depending on that, though.
This patch also disallows LISTEN and UNLISTEN inside a prepared transaction.
That case had some pretty undesirable properties already, such as possibly
allowing pg_listener entries to be made for PIDs no longer present, so
disallowing it seems like a better idea than trying to maintain the behavior.
Tom Lane [Mon, 10 Mar 2008 21:50:40 +0000 (21:50 +0000)]
Use -fwrapv in CFLAGS if we are using a version of gcc that accepts this flag.
This prevents compiler optimizations that assume overflow won't occur, which
breaks numerous overflow tests that we need to have working. It is known
that gcc 4.3 causes problems and possible that 4.1 does. Per my proposal
of some time ago and a recent report from Kris Jurka.
Backpatch as far as 8.0, which is as far as the patch conveniently goes.
7.x was pretty short of overflow tests anyway, so it may not matter there,
even assuming that anyone cares whether 7.x builds on recent gcc.
Tom Lane [Fri, 7 Mar 2008 15:59:23 +0000 (15:59 +0000)]
Change hashscan.c to keep its list of active hash index scans in
TopMemoryContext, rather than scattered through executor per-query contexts.
This poses no danger of memory leak since the ResourceOwner mechanism
guarantees release of no-longer-needed items. It is needed because the
per-query context might already be released by the time we try to clean up
the hash scan list. Report by ykhuang, diagnosis by Heikki.
Back-patch to 8.0, where the ResourceOwner-based cleanup was introduced.
The given test case does not fail before 8.2, probably because we rearranged
transaction abort processing somehow; but this coding is undoubtedly risky
so I'll patch 8.0 and 8.1 anyway.
Tom Lane [Wed, 27 Feb 2008 17:44:41 +0000 (17:44 +0000)]
If RelationBuildDesc() fails to open a critical system index, PANIC with
a relevant error message instead of just dumping core. Odd that nobody
reported this before Darren Reed.
Tom Lane [Mon, 25 Feb 2008 23:21:22 +0000 (23:21 +0000)]
Fix datetime input to behave correctly for Feb 29 in years BC.
Formerly, DecodeDate attempted to verify the day-of-the-month exactly, but
it was under the misapprehension that it would know whether we were looking
at a BC year or not. In reality this check can't be made until the calling
function (eg DecodeDateTime) has processed all the fields. So, split the
BC adjustment and validity checks out into a new function ValidateDate that
is called only after processing all the fields. In passing, this patch
makes DecodeTimeOnly work for BC inputs, which it never did before.
(The historical veracity of all this is nonexistent, of course, but if
we're going to say we support proleptic Gregorian calendar then we should
do it correctly. In any case the unpatched code is broken because it could
emit dates that it would then reject on re-inputting.)
Per report from Bernd Helmle. Back-patch as far as 8.0; in 7.x we were
not using our own calendar support and so this seems a bit too risky
to put into 7.4.
Tom Lane [Wed, 20 Feb 2008 22:18:35 +0000 (22:18 +0000)]
Fix mistakes in pg_ctl's code for "start -w" that tries to cope with
non-default settings for the postmaster's port number. The code to parse
command line options and postgresql.conf entries wasn't quite right about
whitespace or quotes, and it was coded in a not-very-readable way too.
Per bug #3969 from Itagaki Takahiro, though this is more extensive than his
proposed patch (which fixed only the whitespace problem).
This code has been broken since it was put in in 8.0, so patch all the way
back.
Tom Lane [Wed, 20 Feb 2008 17:44:26 +0000 (17:44 +0000)]
Put a CHECK_FOR_INTERRUPTS call into the loops that try to find a unique new
OID or new relfilenode. If the existing OIDs are sufficiently densely
populated, this could take a long time (perhaps even be an infinite loop),
so it seems wise to allow the system to respond to a cancel interrupt here.
Per a gripe from Jacky Leng.
Backpatch as far as 8.1. Older versions just fail on OID collision,
instead of looping.
Tom Lane [Mon, 11 Feb 2008 19:14:45 +0000 (19:14 +0000)]
Repair VACUUM FULL bug introduced by HOT patch: the original way of
calculating a page's initial free space was fine, and should not have been
"improved" by letting PageGetHeapFreeSpace do it. VACUUM FULL is going to
reclaim LP_DEAD line pointers later, so there is no need for a guard
against the page being too full of line pointers, and having one risks
rejecting pages that are perfectly good move destinations.
This also exposed a second bug, which is that the empty_end_pages logic
assumed that any page with no live tuples would get entered into the
fraged_pages list automatically (by virtue of having more free space than
the threshold in the do_frag calculation). This assumption certainly
seems risky when a low fillfactor has been chosen, and even without
tunable fillfactor I think it could conceivably fail on a page with many
unused line pointers. So fix the code to force do_frag true when notup
is true, and patch this part of the fix all the way back.
Tom Lane [Thu, 7 Feb 2008 21:08:16 +0000 (21:08 +0000)]
Some variants of ALTER OWNER tried to make the "object" field of the
statement be a list of bare C strings, rather than String nodes, which is
what they need to be for copyfuncs/equalfuncs to work. Fortunately these
node types never go out to disk (if they did, we'd likely have noticed the
problem sooner), so we can just fix it without creating a need for initdb.
This bug has been there since 8.0, but 8.3 exposes it in a more common
code path (Parse messages) than prior releases did. Per bug #3940 from
Vladimir Kokovic.
Tom Lane [Mon, 14 Jan 2008 18:46:33 +0000 (18:46 +0000)]
Fix an ancient oversight in libpq's handling of V3-protocol COPY OUT mode:
we need to be able to swallow NOTICE messages, and potentially also
ParameterStatus messages (although the latter would be a bit weird),
without exiting COPY OUT state. Fix it, and adjust the protocol documentation
to emphasize the need for this. Per off-list report from Alexander Galler.
Tom Lane [Sun, 6 Jan 2008 01:03:31 +0000 (01:03 +0000)]
A long time ago, Peter pointed out that ruleutils.c didn't dump simple
constant ORDER/GROUP BY entries properly:
http://archives.postgresql.org/pgsql-hackers/2001-04/msg00457.php
The original solution to that was in fact no good, as demonstrated by
today's report from Martin Pitt:
http://archives.postgresql.org/pgsql-bugs/2008-01/msg00027.php
We can't use the column-number-reference format for a constant that is
a resjunk targetlist entry, a case that was unfortunately not thought of
in the original discussion. What we can do instead (which did not work
at the time, but does work in 7.3 and up) is to emit the constant with
explicit ::typename decoration, even if it otherwise wouldn't need it.
This is sufficient to keep the parser from thinking it's a column number
reference, and indeed is probably what the user must have done to get
such a thing into the querytree in the first place.
Tom Lane [Thu, 3 Jan 2008 21:28:30 +0000 (21:28 +0000)]
The original patch to disallow non-passworded connections to non-superusers
failed to cover all the ways in which a connection can be initiated in dblink.
Plug the remaining holes. Also, disallow transient connections in functions
for which that feature makes no sense (because they are only sensible as
part of a sequence of operations on the same connection). Joe Conway
Tom Lane [Thu, 3 Jan 2008 21:24:26 +0000 (21:24 +0000)]
Make standard maintenance operations (including VACUUM, ANALYZE, REINDEX,
and CLUSTER) execute as the table owner rather than the calling user, using
the same privilege-switching mechanism already used for SECURITY DEFINER
functions. The purpose of this change is to ensure that user-defined
functions used in index definitions cannot acquire the privileges of a
superuser account that is performing routine maintenance. While a function
used in an index is supposed to be IMMUTABLE and thus not able to do anything
very interesting, there are several easy ways around that restriction; and
even if we could plug them all, there would remain a risk of reading sensitive
information and broadcasting it through a covert channel such as CPU usage.
To prevent bypassing this security measure, execution of SET SESSION
AUTHORIZATION and SET ROLE is now forbidden within a SECURITY DEFINER context.
Thanks to Itagaki Takahiro for reporting this vulnerability.
Tom Lane [Thu, 3 Jan 2008 20:48:57 +0000 (20:48 +0000)]
Fix assorted security-grade bugs in the regex engine. All of these problems
are shared with Tcl, since it's their code to begin with, and the patches
have been copied from Tcl 8.5.0. Problems:
CVE-2007-4769: Inadequate check on the range of backref numbers allows
crash due to out-of-bounds read.
CVE-2007-4772: Infinite loop in regex optimizer for pattern '($|^)*'.
CVE-2007-6067: Very slow optimizer cleanup for regex with a large NFA
representation, as well as crash if we encounter an out-of-memory condition
during NFA construction.
Part of the response to CVE-2007-6067 is to put a limit on the number of
states in the NFA representation of a regex. This seems needed even though
the within-the-code problems have been corrected, since otherwise the code
could try to use very large amounts of memory for a suitably-crafted regex,
leading to potential DOS by driving the system into swap, activating a kernel
OOM killer, etc.
Although there are certainly plenty of ways to drive the system into effective
DOS with poorly-written SQL queries, these problems seem worth treating as
security issues because many applications might accept regex search patterns
from untrustworthy sources.
Thanks to Will Drewry of Google for reporting these problems. Patches by Will
Drewry and Tom Lane.
Tom Lane [Tue, 18 Dec 2007 00:04:22 +0000 (00:04 +0000)]
Make path_recv() and poly_recv() reject paths/polygons containing no points.
The zero-point case is sensible so far as the data structure is concerned,
so maybe we ought to allow it sometime; but right now the textual input
routines for these types don't allow it, and it seems that not all the
functions for the types are prepared to cope.
Report and patch by Merlin Moncure.
Andrew Dunstan [Sat, 1 Dec 2007 15:39:49 +0000 (15:39 +0000)]
Workaround for perl problem where evaluating UTF8 regexes can cause
implicit loading of modules, thereby breaking Safe rules.
We compile and call a tiny perl function on trusted interpreter init, after which
the problem does not occur.
Alvaro Herrera [Fri, 23 Nov 2007 01:47:12 +0000 (01:47 +0000)]
Fix buggy usage of vsnprintf in PL/Python by removing it altogether, instead
relying on stringinfo.c. This fixes a problem reported by Marko Kreen, but I
didn't use his patch, per subsequent discussion.
Tom Lane [Thu, 22 Nov 2007 17:47:40 +0000 (17:47 +0000)]
Prevent Perl from introducing a possibly-incompatible definition of type
"bool" into plperl.c. This has always been a hazard since Perl allows a
platform-specific choice to define bool as int rather than char, but
evidently this didn't happen on any platform we support ... until OS X 10.5.
Per report from Brandon Maust.
Back-patch as far as 8.0 --- a bit arbitrary, but it seems unlikely anyone
will be trying to port 7.x onto new platforms.
Joe Conway [Sat, 10 Nov 2007 05:01:50 +0000 (05:01 +0000)]
Have crosstab variants treat NULL rowid as a category in its own right,
per suggestion from Tom Lane. This fixes crash-bug reported by Stefan
Schwarzer.
Tom Lane [Fri, 9 Nov 2007 20:10:20 +0000 (20:10 +0000)]
Second pass at improving LIKE/regex estimation in non-C locales. It turns
out that it's actually quite likely that a string that is an extension of
the given prefix will sort as larger than the "greater" string our previous
code created. To provide some defense against that, do the comparisons
against a modified string instead of just the bare prefix. We tack on
"Z", "z", "y", or "9", whichever is seen as largest in the current locale.
Testing suggests that this is sufficient at least for cases involving
ASCII data.
Tom Lane [Thu, 8 Nov 2007 23:23:07 +0000 (23:23 +0000)]
If an index depends on no columns of its table, give it a dependency on the
whole table instead, to ensure that it goes away when the table is dropped.
Per bug #3723 from Sam Mason.
Backpatch as far as 7.4; AFAICT 7.3 does not have the issue, because it doesn't
have general-purpose expression indexes and so there must be at least one
column referenced by an index.
Tom Lane [Wed, 7 Nov 2007 22:37:43 +0000 (22:37 +0000)]
Improve the performance of LIKE/regex estimation in non-C locales, by making
make_greater_string() try harder to generate a string that's actually greater
than its input string. Before we just assumed that making a string that was
memcmp-greater was enough, but it is easy to generate examples where this is
not so when the locale is not C. Instead, loop until the relevant comparison
function agrees that the generated string is greater than the input.
Unfortunately this is probably not enough to guarantee that the generated
string is greater than all extensions of the input, so we cannot relax the
restriction to C locale for the LIKE/regex index optimization. But it should
at least improve the odds of getting a useful selectivity estimate in
prefix_selectivity(). Per example from Guillaume Smet.
Backpatch to 8.1, mainly because that's what the complainant is using...
Tom Lane [Sun, 28 Oct 2007 19:08:16 +0000 (19:08 +0000)]
Fix a couple of issues with pg_dump's handling of inheritance child tables
that have default expressions different from their parent. First, if the
parent table's default expression has to be split out as a separate
ALTER TABLE command, we need a dependency constraint to ensure that the
child's command is given second. This is because the ALTER TABLE on the
parent will propagate to the child. (We can't prevent that by using ONLY on
the parent's command, since it's possible that other children exist that
should receive the inherited default.) Second, if the child has a NULL
default where the parent does not, we have to explicitly say DEFAULT NULL on
the child in order for this state to be preserved after reload. (The latter
actually doesn't work right because of a backend bug, but that is a separate
issue.)
Backpatch as far as 8.0. 7.x pg_dump has enough issues with altered tables
(due to lack of dependency analysis) that trying to fix this one doesn't seem
very productive.
Tom Lane [Sat, 13 Oct 2007 15:55:58 +0000 (15:55 +0000)]
Fix ALTER COLUMN TYPE to preserve the tablespace and reloptions of indexes
it affects. The original coding neglected tablespace entirely (causing
the indexes to move to the database's default tablespace) and for an index
belonging to a UNIQUE or PRIMARY KEY constraint, it would actually try to
assign the parent table's reloptions to the index :-(. Per bug #3672 and
subsequent investigation.
8.0 and 8.1 did not have reloptions, but the tablespace bug is present.
Tom Lane [Thu, 11 Oct 2007 21:28:12 +0000 (21:28 +0000)]
Ensure that the result of evaluating a function during constant-expression
simplification gets detoasted before it is incorporated into a Const node.
Otherwise, if an immutable function were to return a TOAST pointer (an
unlikely case, but it can be made to happen), we would end up with a plan
that depends on the continued existence of the out-of-line toast datum.
Tom Lane [Thu, 4 Oct 2007 20:45:02 +0000 (20:45 +0000)]
Keep the planner from failing on "WHERE false AND something IN (SELECT ...)".
eval_const_expressions simplifies this to just "WHERE false", but we have
already done pull_up_IN_clauses so the IN join will be done, or at least
planned, anyway. The trouble case comes when the sub-SELECT is itself a join
and we decide to implement the IN by unique-ifying the sub-SELECT outputs:
with no remaining reference to the output Vars in WHERE, we won't have
propagated the Vars up to the upper join point, leading to "variable not found
in subplan target lists" error. Fix by adding an extra scan of in_info_list
and forcing all Vars mentioned therein to be propagated up to the IN join
point. Per bug report from Miroslav Sulc.
Tom Lane [Sat, 29 Sep 2007 01:36:29 +0000 (01:36 +0000)]
Make archive recovery always start a new timeline, rather than only when a
recovery stop time was used. This avoids a corner-case risk of trying to
overwrite an existing archived copy of the last WAL segment, and seems
simpler and cleaner all around than the original definition. Per example
from Jon Colverson and subsequent analysis by Simon.
Reduce the size of memory allocations by lazy vacuum when processing a small
table, by allocating just enough for a hardcoded number of dead tuples per
page. The current estimate is 200 dead tuples per page.
Per reports from Jeff Amiel, Erik Jones and Marko Kreen, and subsequent
discussion.
CVS: ----------------------------------------------------------------------
CVS: Enter Log. Lines beginning with `CVS:' are removed automatically
CVS:
CVS: Committing in .
CVS:
CVS: Modified Files:
CVS: commands/vacuumlazy.c
CVS: ----------------------------------------------------------------------
Tom Lane [Sun, 16 Sep 2007 15:56:39 +0000 (15:56 +0000)]
Fix overflow in extract(epoch from interval) for intervals exceeding 68 years.
Seems to have been introduced in 8.1 by careless SECS_PER_DAY
search-and-replace.
Tom Lane [Sun, 16 Sep 2007 02:38:02 +0000 (02:38 +0000)]
Fix aboriginal mistake in lazy VACUUM's code for truncating away
no-longer-needed pages at the end of a table. We thought we could throw away
pages containing HEAPTUPLE_DEAD tuples; but this is not so, because such
tuples very likely have index entries pointing at them, and we wouldn't have
removed the index entries. The problem only emerges in a somewhat unlikely
race condition: the dead tuples have to have been inserted by a transaction
that later aborted, and this has to have happened between VACUUM's initial
scan of the page and then rechecking it for empty in count_nondeletable_pages.
But that timespan will include an index-cleaning pass, so it's not all that
hard to hit. This seems to explain a couple of previously unsolved bug
reports.