Bruce Momjian [Thu, 12 Jun 2003 01:36:44 +0000 (01:36 +0000)]
This patch updates pg_autovacuum in several ways:
* A few bug fixes
* fixes solaris compile and crash issue
* decouple vacuum analyze and analyze thresholds
* detach from tty (dameonize)
* improved logging layout
* more conservative default configuration
* improved, expanded and updated README
please apply and 1st convenience, or before code freeze which ever comes
first :-)
At this point I think I have brought pg_autovacuum and its client side
design as far as I think it should go. It works, keeping file sizes in
check, helps performance and give the administrator a fair amount
flexibility in configuring it.
Next up is to do the FSM based design that is integrated into the back
end.
Bruce Momjian [Thu, 12 Jun 2003 01:17:19 +0000 (01:17 +0000)]
I found the libpq function PGunescapeBytea a little slow. It was taking a
minute and a half to decode a 500Kb on a fairly fast machine. I think the
culprit is sscanf.
I attach a patch that replaces the function with one used to perform the same
task in pyPgSQL (a Python interface to PostgreSQL). This code was written by
Billy Allie, author of pyPgSQL. I've changed a few variable names to match
those in the original code and removed a bit of Pythonness.
Billy has kindly looked at the code and points out that it is slightly
stricter than the original implementation and if it encounters an invalid
bytea such as '\12C' it drops the unescape '\' and outputs '12C'.
The code is licensed by the author under a BSD license.
I've performed limited testing of the function by putting JPEGs into
PostgreSQL, extracting them using them using the new function and diffing
against the original files.
The new function is significantly faster on my machine with the JPEGs being
decoded in less than a second. I attach a modified libpq example program that
I used for my testing.
Bruce Momjian [Wed, 11 Jun 2003 22:37:46 +0000 (22:37 +0000)]
This patch extracts page buffer pooling and the simple
least-recently-used strategy from clog.c into slru.c. It doesn't
change any visible behaviour and passes all regression tests plus a
TruncateCLOG test done manually.
Apart from refactoring I made a little change to SlruRecentlyUsed,
formerly ClogRecentlyUsed: It now skips incrementing lru_counts, if
slotno is already the LRU slot, thus saving a few CPU cycles. To make
this work, lru_counts are initialised to 1 in SimpleLruInit.
SimpleLru will be used by pg_subtrans (part of the nested transactions
project), so the main purpose of this patch is to avoid future code
duplication.
Bruce Momjian [Wed, 11 Jun 2003 18:44:15 +0000 (18:44 +0000)]
Changes:
1 intarray: bugfix for int[]-int[] operation
2 intarray: split _int.c to several files (_int.c now is unused)
3 ntarray (gist__intbig_ops opclass): use special type for index storage
4 ltree (gist__ltree_ops opclass), intarray (gist__intbig_ops): optimize
GiST's
penalty and picksplit interface functions, now use Hemming distance.
Tom Lane [Wed, 11 Jun 2003 16:29:42 +0000 (16:29 +0000)]
pg_dump and pg_restore were stripping quotes and downcasing some but
not all SQL identifiers taken from command line arguments. We decided
years ago that that was a bad idea: identifiers taken from the command
line should be treated as literally correct. Remove the inconsistent
code that has crept in recently. Also fix pg_dump so that the combination
of --schema and --table does what you'd expect, namely dump exactly one
table from exactly one schema. Per gripe from Deepak Bhole of Red Hat.
Bruce Momjian [Wed, 11 Jun 2003 15:05:50 +0000 (15:05 +0000)]
Well, the discussion about SSL a bit back perked my interest and I did
some reading on the subject.
1) PostgreSQL uses ephemeral keying, for its connections (good thing)
2) PostgreSQL doesn't set the cipher list that it allows (bad thing,
fixed)
3) PostgreSQL's renegotiation code wasn't text book correct (could be
bad, fixed)
4) The rate of renegotiating was insanely low (as Tom pointed out, set
to a more reasonable level)
I haven't checked around much to see if there are any other SSL bits
that need some review, but I'm doing some OpenSSL work right now
and'll send patches for improvements along the way (if I find them).
At the very least, the changes in this patch will make security folks
happier for sure. The constant renegotiation of sessions was likely a
boon to systems that had bad entropy gathering means (read: Slowaris
/dev/rand|/dev/urand != ANDIrand). The new limit for renegotiations
is 512MB which should be much more reasonable.
Bruce Momjian [Wed, 11 Jun 2003 15:02:25 +0000 (15:02 +0000)]
During looking stuff up for a discussion on -general, I realized that
I'd placed the check for newly created matching pk rows for on update no
action earlier than it needed to be so that it'd check even when the key
values hadn't changed. This patch moves it to after checking for NULLs
in the old row and comparing the values since the select's probably more
expensive.
Michael Meskes [Wed, 11 Jun 2003 06:39:13 +0000 (06:39 +0000)]
Make sure a variable is no longer referenced when it is removed.
Fixed counting bug in parsing "->" operator.
Removed that silly debugging function I accidently committed last night.
Tom Lane [Mon, 9 Jun 2003 17:59:19 +0000 (17:59 +0000)]
freeaddrinfo2() does need two parameters after all, per comment by
Kurt Roeckx. Add some documentation to try to prevent others from
repeating my mistake.
Tom Lane [Sun, 8 Jun 2003 17:43:00 +0000 (17:43 +0000)]
libpq can now talk to either 3.0 or 2.0 protocol servers. It first tries
protocol 3, then falls back to 2 if postmaster rejects the startup packet
with an old-format error message. A side benefit of the rewrite is that
SSL-encrypted connections can now be made without blocking. (I think,
anyway, but do not have a good way to test.)
Tom Lane [Fri, 6 Jun 2003 16:25:35 +0000 (16:25 +0000)]
Add defense in assign_session_authorization() against trying to do
catalog lookups when not in a transaction. This prevents bizarre
failures if someone tries to set a value for session_authorization in
postgresql.conf. Per report from Fernando Nasser.
Tom Lane [Fri, 6 Jun 2003 15:04:03 +0000 (15:04 +0000)]
Implement outer-level aggregates to conform to the SQL spec, with
extensions to support our historical behavior. An aggregate belongs
to the closest query level of any of the variables in its argument,
or the current query level if there are no variables (e.g., COUNT(*)).
The implementation involves adding an agglevelsup field to Aggref,
and treating outer aggregates like outer variables at planning time.
Bruce Momjian [Mon, 2 Jun 2003 18:42:31 +0000 (18:42 +0000)]
Add:
> * Allow a single index to index multiple tables (for inheritance and subtables) 408a410
> * Improve the planner to use CHECK constraints to prune the plan (for subtables) 418a421
> * Allow partitioning of table into multiple subtables 419a423
> T
Tom Lane [Fri, 30 May 2003 20:23:10 +0000 (20:23 +0000)]
Small performance improvement for hash joins and hash aggregation:
when the plan is ReScanned, we don't have to rebuild the hash table
if there is no parameter change for its child node. This idea has
been used for a long time in Sort and Material nodes, but was not in
the hash code till now.
Tom Lane [Thu, 29 May 2003 22:30:02 +0000 (22:30 +0000)]
Ensure that in all flex lexers that are part of the backend, a
yy_fatal_error() call results in elog(ERROR) not exit(). This was
already fixed in the main lexer and plpgsql, but extend same technique
to all the other dot-l files. Also, on review of the possible calls
to yy_fatal_error(), it seems safe to use elog(ERROR) not elog(FATAL).
Tom Lane [Thu, 29 May 2003 20:40:36 +0000 (20:40 +0000)]
Adopt latest bison's spelling of 'syntax error' rather than 'parse error'
for grammar-detected problems. Revert Makefile hack that kept it looking
like the pre-bison-1.875 output.
Tom Lane [Thu, 29 May 2003 19:15:34 +0000 (19:15 +0000)]
Don't explicitly close() the session socket during backend exit; instead
leave it for the kernel to do after the process dies. This allows clients
to wait for the backend to exit if they wish (after sending X message,
wait till EOF is detected on the socket).
Barry Lind [Thu, 29 May 2003 03:21:32 +0000 (03:21 +0000)]
Initial attempt to integrate in V3 protocol support. This is still a work in
progress, although all RTs pass using the V3 protocol on a 7.4 database and also pass using the V2 protocol on a 7.3 database.
SSL support is known not to work.
Tom Lane [Wed, 28 May 2003 22:32:50 +0000 (22:32 +0000)]
Fix some planner performance problems with large WHERE clauses, by
introducing new 'FastList' list-construction subroutines to use in
hot spots. This avoids the O(N^2) behavior of repeated lappend's
by keeping a tail pointer, while not changing behavior by reversing
list order as the lcons() method would do.
Tom Lane [Wed, 28 May 2003 18:19:09 +0000 (18:19 +0000)]
Fix things so that an error occuring during standalone-backend processing
in initdb will result in exit(1), allowing the initdb script to realize
that there's something wrong.
Tom Lane [Wed, 28 May 2003 17:25:02 +0000 (17:25 +0000)]
In bootstrap and standalone-backend modes, do not sort LOG elevel out
of order; the 'server log' output is actually client output in these
scenarios and we ought to treat elevels the same way as in the client
case. This allows initdb to not send backend stderr to /dev/null anymore,
which makes it much more likely that people will notice problems during
initdb.
Tom Lane [Wed, 28 May 2003 16:04:02 +0000 (16:04 +0000)]
Replace functional-index facility with expressional indexes. Any column
of an index can now be a computed expression instead of a simple variable.
Restrictions on expressions are the same as for predicates (only immutable
functions, no sub-selects). This fixes problems recently introduced with
inlining SQL functions, because the inlining transformation is applied to
both expression trees so the planner can still match them up. Along the
way, improve efficiency of handling index predicates (both predicates and
index expressions are now cached by the relcache) and fix 7.3 oversight
that didn't record dependencies of predicate expressions.
Bruce Momjian [Tue, 27 May 2003 17:49:47 +0000 (17:49 +0000)]
Make debug_ GUC varables output DEBUG1 rather than LOG, and mention in
docs that CLIENT/LOG_MIN_MESSAGES now controls debug_* output location.
Doc changes included.
Tom Lane [Mon, 26 May 2003 17:51:38 +0000 (17:51 +0000)]
Make sure printtup() always sends the number of columns previously
advertised in RowDescription message. Depending on the physical tuple's
column count is not really correct, since according to heap_getattr()
conventions the tuple may be short some columns, which will automatically
get read as nulls. Problem has been latent since forever, but was only
exposed by recent change to skip a projection step in SELECT * FROM...