Tom Lane [Fri, 31 Aug 2007 23:35:22 +0000 (23:35 +0000)]
Apply a band-aid fix for the problem that 8.2 and up completely misestimate
the number of rows likely to be produced by a query such as
SELECT * FROM t1 LEFT JOIN t2 USING (key) WHERE t2.key IS NULL;
What this is doing is selecting for t1 rows with no match in t2, and thus
it may produce a significant number of rows even if the t2.key table column
contains no nulls at all. 8.2 thinks the table column's null fraction is
relevant and thus may estimate no rows out, which results in terrible plans
if there are more joins above this one. A proper fix for this will involve
passing much more information about the context of a clause to the selectivity
estimator functions than we ever have. There's no time left to write such a
patch for 8.3, and it wouldn't be back-patchable into 8.2 anyway. Instead,
put in an ad-hoc test to defeat the normal table-stats-based estimation when
an IS NULL test is evaluated at an outer join, and just use a constant
estimate instead --- I went with 0.5 for lack of a better idea. This won't
catch every case but it will catch the typical ways of writing such queries,
and it seems unlikely to make things worse for other queries.
Tom Lane [Fri, 31 Aug 2007 18:33:40 +0000 (18:33 +0000)]
Extend whole-row Var evaluation to cope with the case that the sub-plan
generating the tuples has resjunk output columns. This is not possible for
simple table scans but can happen when evaluating a whole-row Var for a view.
Per example from Patryk Kordylewski. The problem exists back to 8.0 but
I'm not going to risk back-patching further than 8.2 because of the many
changes in this area.
Tom Lane [Fri, 31 Aug 2007 01:44:06 +0000 (01:44 +0000)]
Rewrite make_outerjoininfo's construction of min_lefthand and min_righthand
sets for outer joins, in the light of bug #3588 and additional thought and
experimentation. The original methodology was fatally flawed for nests of
more than two outer joins: it got the relationships between adjacent joins
right, but didn't always come to the right conclusions about whether a join
could be interchanged with one two or more levels below it. This was largely
caused by a mistaken idea that we should use the min_lefthand + min_righthand
sets of a sub-join as the minimum left or right input set of an upper join
when we conclude that the sub-join can't commute with the upper one. If
there's a still-lower join that the sub-join *can* commute with, this method
led us to think that that one could commute with the topmost join; which it
can't. Another problem (not directly connected to bug #3588) was that
make_outerjoininfo's processing-order-dependent method for enforcing outer
join identity #3 didn't work right: if we decided that join A could safely
commute with lower join B, we dropped all information about sub-joins under B
that join A could perhaps not safely commute with, because we removed B's
entire min_righthand from A's.
To fix, make an explicit computation of all inner join combinations that occur
below an outer join, and add to that the full syntactic relsets of any lower
outer joins that we determine it can't commute with. This method gives much
more direct enforcement of the outer join rearrangement identities, and it
turns out not to cost a lot of additional bookkeeping.
Thanks to Richard Harris for the bug report and test case.
Tom Lane [Thu, 30 Aug 2007 05:27:29 +0000 (05:27 +0000)]
Fix int8mul so that overflow check is applied correctly for INT64_IS_BUSTED
case, per Florian Pflug.
Not back-patched since it's unclear that anyone but me still cares ...
Tom Lane [Wed, 29 Aug 2007 17:24:29 +0000 (17:24 +0000)]
Relax permissions checks on dbsize functions, per discussion. Revert out all
checks for individual-table-size functions, since anyone in the database could
get approximate values from pg_class.relpages anyway. Allow database-size to
users with CONNECT privilege for the target database (note that this is
granted by default). Allow tablespace-size if the user has CREATE privilege
on the tablespace (which is *not* granted by default), or if the tablespace is
the default tablespace for the current database (since we treat that as
implicitly allowing use of the tablespace).
Tom Lane [Wed, 29 Aug 2007 16:31:36 +0000 (16:31 +0000)]
Fix aboriginal bug in _tarAddFile(): when complaining that the amount of data
read from the temp file didn't match the file length reported by ftello(),
the wrong variable's value was printed, and so the message made no sense.
Clean up a couple other coding infelicities while at it.
Tom Lane [Tue, 28 Aug 2007 03:23:44 +0000 (03:23 +0000)]
Improve behavior of log_lock_waits patch. Ensure that something gets logged
even if the "deadlock detected" ERROR message is suppressed by an exception
catcher. Be clearer about the event sequence when a soft deadlock is fixed:
the fixing process might or might not still have to wait, so log that
separately. Fix race condition when someone releases us from the lock partway
through printing all this junk --- we'd not get confused about our state, but
the log message sequence could have been misleading, ie, a "still waiting"
message with no subsequent "acquired" message. Greg Stark and Tom Lane.
Tom Lane [Mon, 27 Aug 2007 03:36:08 +0000 (03:36 +0000)]
Fix a couple of misbehaviors rooted in the fact that the default creation
namespace isn't necessarily first in the search path (there could be implicit
schemas ahead of it). Examples are
test=# set search_path TO s1;
test=# create view pg_timezone_names as select * from pg_timezone_names();
ERROR: "pg_timezone_names" is already a view
test=# create table pg_class (f1 int primary key);
ERROR: permission denied: "pg_class" is a system catalog
You'd expect these commands to create the requested objects in s1, since
names beginning with pg_ aren't supposed to be reserved anymore. What is
happening is that we create the requested base table and then execute
additional commands (here, CREATE RULE or CREATE INDEX), and that code is
passed the same RangeVar that was in the original command. Since that
RangeVar has schemaname = NULL, the secondary commands think they should do a
path search, and that means they find system catalogs that are implicitly in
front of s1 in the search path.
This is perilously close to being a security hole: if the secondary command
failed to apply a permission check then it'd be possible for unprivileged
users to make schema modifications to system catalogs. But as far as I can
find, there is no code path in which a check doesn't occur. Which makes it
just a weird corner-case bug for people who are silly enough to want to
name their tables the same as a system catalog.
The relevant code has changed quite a bit since 8.2, which means this patch
wouldn't work as-is in the back branches. Since it's a corner case no one
has reported from the field, I'm not going to bother trying to back-patch.
Tom Lane [Mon, 27 Aug 2007 01:39:25 +0000 (01:39 +0000)]
Remove the 'not in' operator (!!=). This was a hangover from Berkeley
days that was obsolete the moment we had IN (SELECT ...) capability.
It's arguably a security hole since it applied no permissions check to
the table it searched, and since it was never documented anywhere,
removing it seems more appropriate than fixing it.
Tom Lane [Mon, 27 Aug 2007 01:24:50 +0000 (01:24 +0000)]
Require SELECT privilege on a table to do dblink_get_pkey(). This is
not all that exciting when the system catalogs are readable by all,
but some people try to lock them down, and would not like this sort of
end run ...
Tom Lane [Mon, 27 Aug 2007 01:19:14 +0000 (01:19 +0000)]
Restrict pg_relation_size to relation owner, pg_database_size to DB owner,
and pg_tablespace_size to superusers. Perhaps we could weaken the first
case to just require SELECT privilege, but that doesn't work for the
other cases, so use ownership as the common concept.
Tom Lane [Mon, 27 Aug 2007 00:57:36 +0000 (00:57 +0000)]
Make currtid() functions require SELECT privileges on the target table.
While it's not clear that TID linkage info is of any great use to a
nefarious user, it's certainly unexpected that these functions wouldn't
insist on read privileges.
Tom Lane [Mon, 27 Aug 2007 00:13:51 +0000 (00:13 +0000)]
Restrict pgrowlocks function to superusers. (This might be too strict,
but no permissions check at all is certainly no good.) Clean up usage
of some deprecated APIs.
Tom Lane [Sun, 26 Aug 2007 23:59:50 +0000 (23:59 +0000)]
Restrict pgstattuple functions to superusers. (This might be too strict,
but no permissions check at all is certainly no good.) Clean up usage
of some deprecated APIs.
Tom Lane [Sun, 26 Aug 2007 21:44:25 +0000 (21:44 +0000)]
Make ARRAY(SELECT ...) return an empty array, rather than a NULL, when the
sub-select returns zero rows. Per complaint from Jens Schicke. Since this
is more in the nature of a definition change than a bug, not back-patched.
Tom Lane [Sat, 25 Aug 2007 20:29:25 +0000 (20:29 +0000)]
Adjust with-system-tzdata patch to not attempt to install a symlink,
but just hardwire the specified timezone database path into the executable.
Per discussion, this avoids some packaging disadvantages of using a
symlink.
Tom Lane [Sat, 25 Aug 2007 19:08:19 +0000 (19:08 +0000)]
Fix brain fade in DefineIndex(): it was continuing to access the table's
relcache entry after having heap_close'd it. This could lead to misbehavior
if a relcache flush wiped out the cache entry meanwhile. In 8.2 there is a
very real risk of CREATE INDEX CONCURRENTLY using the wrong relid for locking
and waiting purposes. I think the bug is only cosmetic in 8.0 and 8.1,
because their transgression is limited to using RelationGetRelationName(rel)
in an ereport message immediately after heap_close, and there's no way (except
with special debugging options) for a cache flush to occur in that interval.
Not quite sure that it's cosmetic in 7.4, but seems best to patch anyway.
Found by trying to run the regression tests with CLOBBER_CACHE_ALWAYS enabled.
Maybe we should try to do that on a regular basis --- it's awfully slow,
but perhaps some fast buildfarm machine could do it once in awhile.
Tom Lane [Sat, 25 Aug 2007 17:47:44 +0000 (17:47 +0000)]
Simplify implementation of ts_debug() function --- use a join instead
of redundant sub-selects. initdb not forced, since this is just a
cosmetic change, but the new code won't show up till you do one.
Tom Lane [Sat, 25 Aug 2007 00:03:59 +0000 (00:03 +0000)]
Cleanup for some problems in tsearch patch:
- ispell initialization crashed on empty dictionary file
- ispell initialization crashed on affix file with prefixes but no suffixes
- stop words file was run through pg_verify_mbstr, with database
encoding, but it's supposed to be UTF-8; similar bug for synonym files
- bunch of comments added, typos fixed, and other cleanup
Introduced consistent encoding checking/conversion of data read from tsearch
configuration files, by doing this in a single t_readline() subroutine
(replacing direct usages of fgets). Cleaned up API for readstopwords too.
Tom Lane [Thu, 23 Aug 2007 16:15:51 +0000 (16:15 +0000)]
Fix combo_decrypt() to throw an error for zero-length input when using a
padded encryption scheme. Formerly it would try to access res[(unsigned) -1],
which resulted in core dumps on 64-bit machines, and was certainly trouble
waiting to happen on 32-bit machines (though in at least the known case
it was harmless because that byte would be overwritten after return).
Per report from Ken Colson; fix by Marko Kreen.
Tom Lane [Wed, 22 Aug 2007 22:30:20 +0000 (22:30 +0000)]
Suppress testing the options of CREATE TEXT SEARCH DICTIONARY during
initdb. We should create all the standard dictionaries even though
some of them may not work in template1's encoding. Per Teodor.
Tom Lane [Wed, 22 Aug 2007 05:13:50 +0000 (05:13 +0000)]
Remove option to change parser of an existing text search configuration.
This prevents needing to do complex and poorly-defined updates of the
mapping table if the new parser has different token types than the old.
Per discussion.
Tom Lane [Wed, 22 Aug 2007 02:25:34 +0000 (02:25 +0000)]
Make \dFt and \dFt+ do something different --- omitting the function
names in the former case seems more consistent with the behavior of
other \dF commands.
Tom Lane [Wed, 22 Aug 2007 01:39:46 +0000 (01:39 +0000)]
Simplify the syntax of CREATE/ALTER TEXT SEARCH DICTIONARY by treating the
init options of the template as top-level options in the syntax. This also
makes ALTER a bit easier to use, since options can be replaced individually.
I also made these statements verify that the tmplinit method will accept
the new settings before they get stored; in the original coding you didn't
find out about mistakes until the dictionary got invoked.
Under the hood, init methods now get options as a List of DefElem instead
of a raw text string --- that lets tsearch use existing options-pushing code
instead of duplicating functionality.
Tom Lane [Tue, 21 Aug 2007 21:24:00 +0000 (21:24 +0000)]
Simplify CREATE TEXT SEARCH CONFIGURATION by eliminating the separate
'with map' parameter; as things now stand there's really not much point
in specifying a config-to-copy if you don't copy its map. Also, use
COPY instead of TEMPLATE as the key word for a config-to-copy, so as
to avoid confusion with text search templates. Per discussion; the
just-committed reference page for the command already describes it
this way.
Tom Lane [Tue, 21 Aug 2007 21:08:47 +0000 (21:08 +0000)]
First rough cut at text search documentation: bare bones reference
pages for the new SQL commands. I also committed Bruce's text search
introductory chapter, as-is except for fixing some markup errors,
so that there would be a place for the reference pages to link to.
Tom Lane [Tue, 21 Aug 2007 16:08:23 +0000 (16:08 +0000)]
Exclude snowball/libstemmer/ files from the set processed by pgindent.
There's not much point in prettifying machine-generated code, and it
seems best to keep these files exactly like upstream anyway. Also add
some notes about why various files are excluded.
Tom Lane [Tue, 21 Aug 2007 02:40:06 +0000 (02:40 +0000)]
Fix potential access-off-the-end-of-memory in varbit_out(): it fetched the
byte after the last full byte of the bit array, regardless of whether that
byte was part of the valid data or not. Found by buildfarm testing.
Thanks to Stefan Kaltenbrunner for nailing down the cause.
Tom Lane [Tue, 21 Aug 2007 01:12:28 +0000 (01:12 +0000)]
contrib/tsearch2 now fails its regression test because of conflicts with
the core additions. For the moment I diked it out of contrib/Makefile.
We should look at turning it into a backward-compatibility package.
Tom Lane [Tue, 21 Aug 2007 01:11:32 +0000 (01:11 +0000)]
Tsearch2 functionality migrates to core. The bulk of this work is by
Oleg Bartunov and Teodor Sigaev, but I did a lot of editorializing,
so anything that's broken is probably my fault.
Documentation is nonexistent as yet, but let's land the patch so we can
get some portability testing done.
Andrew Dunstan [Sun, 19 Aug 2007 01:41:25 +0000 (01:41 +0000)]
Provide for logfiles in machine readable CSV format. In consequence, rename
redirect_stderr to logging_collector.
Original patch from Arul Shaji, subsequently modified by Greg Smith, and then
heavily modified by me.
Tom Lane [Wed, 15 Aug 2007 21:39:50 +0000 (21:39 +0000)]
Arrange to cache a ResultRelInfo in the executor's EState for relations that
are not one of the query's defined result relations, but nonetheless have
triggers fired against them while the query is active. This was formerly
impossible but can now occur because of my recent patch to fix the firing
order for RI triggers. Caching a ResultRelInfo avoids duplicating work by
repeatedly opening and closing the same relation, and also allows EXPLAIN
ANALYZE to "see" and report on these extra triggers. Use the same mechanism
to cache open relations when firing deferred triggers at transaction shutdown;
this replaces the former one-element-cache strategy used in that case, and
should improve performance a bit when there are deferred triggers on a number
of relations.
Tom Lane [Wed, 15 Aug 2007 19:15:47 +0000 (19:15 +0000)]
Repair problems occurring when multiple RI updates have to be done to the same
row within one query: we were firing check triggers before all the updates
were done, leading to bogus failures. Fix by making the triggers queued by
an RI update go at the end of the outer query's trigger event list, thereby
effectively making the processing "breadth-first". This was indeed how it
worked pre-8.0, so the bug does not occur in the 7.x branches.
Per report from Pavel Stehule.
Bruce Momjian [Wed, 15 Aug 2007 15:53:30 +0000 (15:53 +0000)]
Add third idea about pulling data from indexes.
> A third idea would be for a heap scan to check if all rows are visible
> and if so set a per-table flag which can be checked by index scans.
> Any change to the table would have to clear the flag. To detect
> changes during the heap scan a counter could be set at the start and
> checked at the end --- if it is the same, the table has not been
> modified --- any table change would increment the counter.
Tom Lane [Tue, 14 Aug 2007 17:35:18 +0000 (17:35 +0000)]
Fix oversight in async-commit patch: there were some places in heapam.c
that still thought they could set HEAP_XMAX_COMMITTED immediately after
seeing the other transaction commit. Make them use the same logic as
tqual.c does to determine if the hint bit can be set yet.