Tom Lane [Sat, 22 Sep 2007 04:40:03 +0000 (04:40 +0000)]
Although I'd misdiagnosed the reason for the recent failures on
buildfarm member grebe, I see no reason to revert the 1-byte-header-friendly
changes I made in varlena.c. Instead, tweak the code a little bit to
get more advantage out of that.
Andrew Dunstan [Sat, 22 Sep 2007 03:58:34 +0000 (03:58 +0000)]
Go back to using a separate method for doing ILIKE for single byte
character encodings that doesn't involve calling lower(). This should
cure the performance regression in this case complained of by Guillaume
Smet. It still leaves the horrid performance for multi-byte encodings
introduced in 8.2, but there's no obvious solution for that in sight.
Tom Lane [Sat, 22 Sep 2007 00:36:38 +0000 (00:36 +0000)]
Fix varlena.c routines to allow 1-byte-header text values. This is now
demonstrably necessary for text_substring() since regexp_split functions
may pass it such a value; and we might as well convert the whole file
at once. Per buildfarm results (though I wonder why most machines aren't
showing a failure).
Tom Lane [Fri, 21 Sep 2007 22:52:52 +0000 (22:52 +0000)]
Fix regex, LIKE, and some other second-rank text-manipulation functions
to not cause needless copying of text datums that have 1-byte headers.
Greg Stark, in response to performance gripe from Guillaume Smet and
ITAGAKI Takahiro.
Tom Lane [Fri, 21 Sep 2007 21:25:42 +0000 (21:25 +0000)]
Improve handling of prune/no-prune decisions by storing a page's oldest
unpruned XMAX in its header. At the cost of 4 bytes per page, this keeps us
from performing heap_page_prune when there's no chance of pruning anything.
Seems to be necessary per Heikki's preliminary performance testing.
Tom Lane [Fri, 21 Sep 2007 18:24:28 +0000 (18:24 +0000)]
Change tqual.c tests to use !TransactionIdIsCurrentTransactionId, rather than
TransactionIdDidAbort, when handling the case that xmin is one of the current
transaction's XIDs and the tuple has been deleted. xmax must also be one of
the current transaction's XIDs, since no one else can see it yet, and it's
cheaper to look at local state than shared state to find out if xmax aborted.
Per an idea of Heikki's.
Tom Lane [Fri, 21 Sep 2007 17:36:53 +0000 (17:36 +0000)]
Make some simple performance improvements in TransactionIdIsInProgress().
For XIDs of our own transaction and subtransactions, it's cheaper to ask
TransactionIdIsCurrentTransactionId() than to look in shared memory.
Also, the xids[] work array is always the same size within any given
process, so malloc it just once instead of doing a palloc/pfree on every
call; aside from being faster this lets us get rid of some goto's, since
we no longer have any end-of-function pfree to do. Both ideas by Heikki.
Tom Lane [Fri, 21 Sep 2007 00:30:49 +0000 (00:30 +0000)]
Insert a hack in pl/tcl to disable Tcl's built-in Notifier subsystem, which
has a bad habit of launching multiple threads within the backend and thereby
causing all kinds of havoc. Fortunately, we don't need it, and recent Tcl
versions provide an easy way to disable it. Diagnosis and fix by
Steve Marshall, Paul Bayer, and Doug Knight of WSI Corporation.
Bruce Momjian [Thu, 20 Sep 2007 18:54:19 +0000 (18:54 +0000)]
Done:
> * -Consider shrinking expired tuples to just their headers
> * -Allow heap reuse of UPDATEd rows if no indexed columns are changed,
> and old and new versions are on the same heap page
Not needed anymore:
< * Reuse index tuples that point to heap tuples that are not visible to
< anyone?
Tom Lane [Thu, 20 Sep 2007 17:56:33 +0000 (17:56 +0000)]
HOT updates. When we update a tuple without changing any of its indexed
columns, and the new version can be stored on the same heap page, we no longer
generate extra index entries for the new version. Instead, index searches
follow the HOT-chain links to ensure they find the correct tuple version.
In addition, this patch introduces the ability to "prune" dead tuples on a
per-page basis, without having to do a complete VACUUM pass to recover space.
VACUUM is still needed to clean up dead index entries, however.
Pavan Deolasee, with help from a bunch of other people.
Neil Conway [Wed, 19 Sep 2007 22:31:48 +0000 (22:31 +0000)]
Prevent corr() from returning the wrong results for negative correlation
values. The previous coding essentially assumed that x = sqrt(x*x), which
does not hold for x < 0.
Thanks to Jie Zhang at Greenplum and Gavin Sherry for reporting this
issue.
Andrew Dunstan [Tue, 18 Sep 2007 17:41:17 +0000 (17:41 +0000)]
Close previously open holes for invalidly encoded data to enter the
database via builtin functions, as recently discussed on -hackers.
chr() now returns a character in the database encoding. For UTF8 encoded databases
the argument is treated as a Unicode code point. For other multi-byte encodings
the argument must designate a strict ascii character, or an error is raised,
as is also the case if the argument is 0.
ascii() is adjusted so that it remains the inverse of chr().
The two argument form of convert() is gone, and the three argument form now
takes a bytea first argument and returns a bytea. To cover this loss three new
functions are introduced:
. convert_from(bytea, name) returns text - converts the first argument from the
named encoding to the database encoding
. convert_to(text, name) returns bytea - converts the first argument from the
database encoding to the named encoding
. length(bytea, name) returns int - gives the length of the first argument in
characters in the named encoding
Tom Lane [Sun, 16 Sep 2007 16:33:04 +0000 (16:33 +0000)]
Remove Assert(BgWriterShmem != NULL), which is rather pointless since
we'd dump core anyway immediately afterward if it were null; and it
seems to confuse some versions of icc into generating bad code.
Per report from Sergey Koposov. Patched in HEAD only, for the moment,
since this is only likely to affect developers.
Tom Lane [Sun, 16 Sep 2007 15:56:20 +0000 (15:56 +0000)]
Fix overflow in extract(epoch from interval) for intervals exceeding 68 years.
Seems to have been introduced in 8.1 by careless SECS_PER_DAY
search-and-replace.
Tom Lane [Sun, 16 Sep 2007 02:37:46 +0000 (02:37 +0000)]
Fix aboriginal mistake in lazy VACUUM's code for truncating away
no-longer-needed pages at the end of a table. We thought we could throw away
pages containing HEAPTUPLE_DEAD tuples; but this is not so, because such
tuples very likely have index entries pointing at them, and we wouldn't have
removed the index entries. The problem only emerges in a somewhat unlikely
race condition: the dead tuples have to have been inserted by a transaction
that later aborted, and this has to have happened between VACUUM's initial
scan of the page and then rechecking it for empty in count_nondeletable_pages.
But that timespan will include an index-cleaning pass, so it's not all that
hard to hit. This seems to explain a couple of previously unsolved bug
reports.
Bruce Momjian [Fri, 14 Sep 2007 18:50:48 +0000 (18:50 +0000)]
Add:
>
> * Make it possible to use RETURNING together with conditional DO INSTEAD
> rules, such as for partitioning setups
>
> http://archives.postgresql.org/pgsql-hackers/2007-09/msg00577.php
Bruce Momjian [Fri, 14 Sep 2007 04:25:24 +0000 (04:25 +0000)]
Add tsearch tab completion support:
* adds a few missing words to some commands (like adding GIN as a valid
index type or OWNED BY for ALTER SEQUENCE,...)
* support for ALTER TABLE foo ENABLE/DISABLE REPLICA TRIGGER/RULE
* autocomplete CREATE DATABASE foo TEMPLATE (mostly done to prevent
conflicts with the TEMPLATE keyword for text search)
* support for ALTER/CREATE/DROP TEXT SEARCH as well as COMMENT ON TEXT
SEARCH and the corresponding psql backslash commands.
This proved a little more difficult than expected due to the fact that
words_after_create[] is used for two purposes - one is to provide a list
of words that follow immediatly after CREATE (or DROP) and the other
purpose is to use it for autocompleting anywhere in the statement if the
word in that struct is found with a query.
Since TEXT SEARCH CONFIGURATION|DICTIONARY|TEMPLATE|PARSER results in 3
words instead of one (as all the other words in that list are) I added a
flag to the struct to tell create_command_generator() to skip that entry
for autocompleting immediatly after CREATE which feels like a dirty
hack (but that holds true for a lot of code in tab-complete.c).
Tom Lane [Fri, 14 Sep 2007 03:25:31 +0000 (03:25 +0000)]
Remove ill-considered (not to mention undocumented) attempt to make
contrib/intarray's GIN opclass override the built-in default. Per bug #3048
and other complaints.
Tom Lane [Wed, 12 Sep 2007 22:10:26 +0000 (22:10 +0000)]
Redefine the lp_flags field of item pointers as having four states, rather
than two independent bits (one of which was never used in heap pages anyway,
or at least hadn't been in a very long time). This gives us flexibility to
add the HOT notions of redirected and dead item pointers without requiring
anything so klugy as magic values of lp_off and lp_len. The state values
are chosen so that for the states currently in use (pre-HOT) there is no
change in the physical representation.
Tom Lane [Tue, 11 Sep 2007 19:50:25 +0000 (19:50 +0000)]
Give ICC its preferred, documented form of the no-strict-aliasing
switch, viz '-fno-alias'. Since we have a separate code path here
anyway, it's just as easy to cooperate.
Tom Lane [Tue, 11 Sep 2007 17:15:33 +0000 (17:15 +0000)]
Make sure that open hash table scans are cleaned up when bgwriter tries to
recover from elog(ERROR). Problem was created by introduction of hash seq
search tracking awhile back, and affects all branches that have bgwriter;
in HEAD the disease has snuck into autovacuum and walwriter too. (Not sure
that the latter two use hash_seq_search at the moment, but surely they might
someday.) Per report from Sergey Koposov.
Tom Lane [Tue, 11 Sep 2007 16:17:46 +0000 (16:17 +0000)]
Include hash table name in all the internal-error elog messages in
dynahash.c. Sergey Koposov's current open problem shows the possible
usefulness of this, and it doesn't add much code.
Add regression tests for ispell, synonym and thesaurus dictionaries.
Rename synonym.syn.sample and thesaurs.ths.sample to
synonym_sample.syn and thesaurs_sample.ths accordingly to be able to use they
in regression test.
Fix ts_debug function to prevent unneeded calls of ts_lexize().
It will be mush better to reimplement ts_debug in C (instead of SQL as now),
but it's planned for the future.
Refactor from Heikki Linnakangas <heikki@enterprisedb.com>:
* Defined new struct WordEntryPosVector that holds a uint16 length and a
variable size array of WordEntries. This replaces the previous
convention of a variable size uint16 array, with the first element
implying the length. WordEntryPosVector has the same layout in memory,
but is more readable in source code. The POSDATAPTR and POSDATALEN
macros are still used, though it would now be more readable to access
the fields in WordEntryPosVector directly.
* Removed needfree field from DocRepresentation. It was always set to false.
Tom Lane [Tue, 11 Sep 2007 00:06:42 +0000 (00:06 +0000)]
Arrange for SET LOCAL's effects to persist until the end of the current top
transaction, unless rolled back or overridden by a SET clause for the same
variable attached to a surrounding function call. Per discussion, these
seem the best semantics. Note that this is an INCOMPATIBLE CHANGE: in 8.0
through 8.2, SET LOCAL's effects disappeared at subtransaction commit
(leading to behavior that made little sense at the SQL level).
I took advantage of the opportunity to rewrite and simplify the GUC variable
save/restore logic a little bit. The old idea of a "tentative" value is gone;
it was a hangover from before we had a stack. Also, we no longer need a stack
entry for every nesting level, but only for those in which a variable's value
actually changed.
Remove the vacuum_delay_point call in count_nondeletable_pages, because we hold
an exclusive lock on the table at this point, which we want to release as soon
as possible. This is called in the phase of lazy vacuum where we truncate the
empty pages at the end of the table.
An alternative solution would be to lower the vacuum delay settings before
starting the truncating phase, but this doesn't work very well in autovacuum
due to the autobalancing code (which can cause other processes to change our
cost delay settings). This case could be considered in the balancing code, but
it is simpler this way.
Fixes from Heikki Linnakangas <heikki@enterprisedb.com>:
Apparently it's a bug I introduced when I refactored spell.c to use the
readline function for reading and recoding the input file. I didn't
notice that some calls to STRNCMP used the non-lowercased version of the
input line.
Tom Lane [Mon, 10 Sep 2007 00:57:22 +0000 (00:57 +0000)]
Code review for GUC revert-values-if-removed-from-postgresql.conf patch;
and in passing, fix some bogosities dating from the custom_variable_classes
patch. Fix guc-file.l to correctly check changes in custom_variable_classes
that are attempted concurrently with additions/removals of custom variables,
and don't allow the new setting to be applied in advance of checking it.
Clean up messy and undocumented situation for string variables with NULL
boot_val. Fix DefineCustomVariable functions to initialize boot_val
correctly. Prevent find_option from inserting bogus placeholders for custom
variables that are simply inquired about rather than being set.
Andrew Dunstan [Sun, 9 Sep 2007 20:40:54 +0000 (20:40 +0000)]
Provide for a file specifying non-standard config options for temp install
for pg_regress, via --temp-config option. Pick this up in the make file
via TEMP_CONFIG setting.
Tom Lane [Sat, 8 Sep 2007 20:31:15 +0000 (20:31 +0000)]
Replace the former method of determining snapshot xmax --- to wit, calling
ReadNewTransactionId from GetSnapshotData --- with a "latestCompletedXid"
variable that is updated during transaction commit or abort. Since
latestCompletedXid is written only in places that had to lock ProcArrayLock
exclusively anyway, and is read only in places that had to lock ProcArrayLock
shared anyway, it adds no new locking requirements to the system despite being
cluster-wide. Moreover, removing ReadNewTransactionId from snapshot
acquisition eliminates the need to take both XidGenLock and ProcArrayLock at
the same time. Since XidGenLock is sometimes held across I/O this can be a
significant win. Some preliminary benchmarking suggested that this patch has
no effect on average throughput but can significantly improve the worst-case
transaction times seen in pgbench. Concept by Florian Pflug, implementation
by Tom Lane.
Tom Lane [Fri, 7 Sep 2007 20:59:26 +0000 (20:59 +0000)]
Don't take ProcArrayLock while exiting a transaction that has no XID; there is
no need for serialization against snapshot-taking because the xact doesn't
affect anyone else's snapshot anyway. Per discussion. Also, move various
info about the interlocking of transactions and snapshots out of code comments
and into a hopefully-more-cohesive discussion in access/transam/README.
Also, remove a couple of now-obsolete comments about having to force some WAL
to be written to persuade RecordTransactionCommit to do its thing.
Improve page split in rtree emulation. Now if splitted result has
big misalignement, then it tries to split page basing on distribution
of boxe's centers.
Per report from Dolafi, Tom <dolafit@janelia.hhmi.org>
Backpatch is needed, change doesn't affect on-disk storage.
Improvements from Heikki Linnakangas <heikki@enterprisedb.com>
- change the alignment requirement of lexemes in TSVector slightly.
Lexeme strings were always padded to 2-byte aligned length to make sure
that if there's position array (uint16[]) it has the right alignment.
The patch changes that so that the padding is not done when there's no
positions. That makes the storage of tsvectors without positions
slightly more compact.
- added some #include "miscadmin.h" lines I missed in the earlier when I
added calls to check_stack_depth().
- Reimplement the send/recv functions, and added a comment
above them describing the on-wire format. The CRC is now recalculated in
tsquery as well per previous discussion.
Improving various checks by Heikki Linnakangas <heikki@enterprisedb.com>
- add code to check that the query tree is well-formed. It was indeed
possible to send malformed queries in binary mode, which produced all
kinds of strange results.
- make the left-field a uint32. There's no reason to
arbitrarily limit it to 16-bits, and it won't increase the disk/memory
footprint either now that QueryOperator and QueryOperand are separate
structs.
- add check_stack_depth() call to all recursive functions I found.
Some of them might have a natural limit so that you can't force
arbitrarily deep recursions, but check_stack_depth() is cheap enough
that seems best to just stick it into anything that might be a problem.
Refactoring by Heikki Linnakangas <heikki@enterprisedb.com> with
small editorization by me
- Brake the QueryItem struct into QueryOperator and QueryOperand.
Type was really the only common field between them. QueryItem still
exists, and is used in the TSQuery struct as before, but it's now a
union of the two. Many other changes fell from that, like separation
of pushval_asis function into pushValue, pushOperator and pushStop.
- Moved some structs that were for internal use only from header files
to the right .c-files.
- Moved tsvector parser to a new tsvector_parser.c file. Parser code was
about half of the size of tsvector.c, it's also used from tsquery.c, and
it has some data structures of its own, so it seems better to separate
it. Cleaned up the API so that TSVectorParserState is not accessed from
outside tsvector_parser.c.
- Separated enumerations (#defines, really) used for QueryItem.type
field and as return codes from gettoken_query. It was just accidental
code sharing.
- Removed ParseQueryNode struct used internally by makepol and friends.
push*-functions now construct QueryItems directly.
- Changed int4 variables to just ints for variables like "i" or "array
size", where the storage-size was not significant.
Tom Lane [Thu, 6 Sep 2007 17:31:58 +0000 (17:31 +0000)]
Make eval_const_expressions() preserve typmod when simplifying something like
null::char(3) to a simple Const node. (It already worked for non-null values,
but not when we skipped evaluation of a strict coercion function.) This
prevents loss of typmod knowledge in situations such as exhibited in bug
#3598. Unfortunately there seems no good way to fix that bug in 8.1 and 8.2,
because they simply don't carry a typmod for a plain Const node.
In passing I made all the other callers of makeNullConst supply "real" typmod
values too, though I think it probably doesn't matter anywhere else.
Tom Lane [Wed, 5 Sep 2007 21:11:19 +0000 (21:11 +0000)]
Volatile-qualify the ProcArray PGPROC pointer in a bunch of routines
that examine fields that could change under them. This is just to make
really sure that when we are fetching a value 'only once', that's what
actually happens. Possibly this is a bug that should be back-patched,
but in the absence of solid evidence that it's needed, I won't bother.