Teodor Sigaev [Mon, 20 Jun 2005 10:29:37 +0000 (10:29 +0000)]
1. full functional WAL for GiST
2. improve vacuum for gist
- use FSM
- full vacuum:
- reforms parent tuple if it's needed
( tuples was deleted on child page or parent tuple remains invalid
after crash recovery )
- truncate index file if possible
3. fixes bugs and mistakes
Tom Lane [Sun, 19 Jun 2005 22:41:00 +0000 (22:41 +0000)]
Avoid unnecessary palloc overhead in _bt_first(). The temporary
scankeys arrays that it needs can never have more than INDEX_MAX_KEYS
entries, so it's reasonable to just allocate them as fixed-size local
arrays, and save the cost of palloc/pfree. Not a huge savings, but
a cycle saved is a cycle earned ...
Tom Lane [Sun, 19 Jun 2005 21:34:03 +0000 (21:34 +0000)]
Simplify uses of readdir() by creating a function ReadDir() that
includes error checking and an appropriate ereport(ERROR) message.
This gets rid of rather tedious and error-prone manipulation of errno,
as well as a Windows-specific bug workaround, at more than a dozen
call sites. After an idea in a recent patch by Heikki Linnakangas.
Tom Lane [Sun, 19 Jun 2005 20:00:39 +0000 (20:00 +0000)]
Arrange to fsync two-phase-commit state files only during checkpoints;
given reasonably short lifespans for prepared transactions, this should
mean that only a small minority of state files ever need to be fsynced
at all. Per discussion with Heikki Linnakangas.
Tom Lane [Sat, 18 Jun 2005 20:51:30 +0000 (20:51 +0000)]
When using C-string lookup keys in a dynahash.c hash table, use strncpy()
not memcpy() to copy the offered key into the hash table during HASH_ENTER.
This avoids possible core dump if the passed key is located very near the
end of memory. Per report from Stefan Kaltenbrunner.
Tom Lane [Sat, 18 Jun 2005 19:33:42 +0000 (19:33 +0000)]
Add a time-of-preparation column to the pg_prepared_xacts view, per an
old suggestion by Oliver Jowett. Also, add a transaction column to the
pg_locks view to show the xid of each transaction holding or awaiting
locks; this allows prepared transactions to be properly associated with
the locks they own. There was already a column named 'transaction',
and I chose to rename it to 'transactionid' --- since this column is
new in the current devel cycle there should be no backwards compatibility
issue to worry about.
Bruce Momjian [Fri, 17 Jun 2005 19:20:27 +0000 (19:20 +0000)]
Re-order item.
> * Auto-fill the free space map by scanning the buffer cache or by
> checking pages written by the background writer
< * Auto-fill the free space map by scanning the buffer cache or by
< checking pages written by the background writer
Bruce Momjian [Fri, 17 Jun 2005 02:20:43 +0000 (02:20 +0000)]
Add:
>
> * Create a bitmap of pages that need vacuuming
>
> Instead of sequentially scanning the entire table, have the background
> writer or some other process record pages that have expired rows, then
> VACUUM can look at just those pages rather than the entire table. In
> the event of a system crash, the bitmap would probably be invalidated.
Tom Lane [Wed, 15 Jun 2005 16:24:07 +0000 (16:24 +0000)]
Improve hash method for bitmapsets: some examination of actual outputs
shows that adding a circular shift between words greatly improves the
distribution of hash outputs.
Neil Conway [Wed, 15 Jun 2005 07:27:44 +0000 (07:27 +0000)]
Change the implementation of hash join to attempt to avoid unnecessary
work if either of the join relations are empty. The logic is:
(1) if the inner relation's startup cost is less than the outer
relation's startup cost and this is not an outer join, read
a single tuple from the inner relation via ExecHash()
- if NULL, we're done
(2) read a single tuple from the outer relation
- if NULL, we're done
(3) build the hash table on the inner relation
- if hash table is empty and this is not an outer join,
we're done
(4) otherwise, do hash join as usual
The implementation uses the new MultiExecProcNode API, per a
suggestion from Tom: invoking ExecHash() now produces the first
tuple from the Hash node's child node, whereas MultiExecHash()
builds the hash table.
I had to put in a bit of a kludge to get the row count returned
for EXPLAIN ANALYZE to be correct: since ExecHash() is invoked to
return a tuple, and then MultiExecHash() is invoked, we would
return one too many tuples to EXPLAIN ANALYZE. I hacked around
this by just manually detecting this situation and subtracting 1
from the EXPLAIN ANALYZE row count.
Bruce Momjian [Wed, 15 Jun 2005 00:35:16 +0000 (00:35 +0000)]
> Here's a patch I added against plperl, originally against beta5, now
> against rc1. It simply checks with GetDatabaseEncoding() if the current
> database is in UTF-8, and if so, sets the UTF-8 flag on the arguments
> that are passed to perl. This means that it isn't necessary to
> utf8::upgrade() every string, as perl has no way of knowing offhand
> that a string is UTF-8 -- but postgres does, because the database
> encoding is specified, so it makes sense to turn the flag on. You
> should also be able to properly manipulate UTF-8 strings now from
> plperl as opposed to plperlu, because otherwise you'd have to use
> encoding 'utf8' which was not allowed. It could also eliminate some
> unexpected bugs if you assume that perl knows the string is unicode.
It
> is enabled only for perl 5.6 and higher, so earlier versions will not
> be affected.
>
> I have been assured by crab that the patch is quite harmless and will
> not break anything. It would be great to see it in 8 final! :-)
Bruce Momjian [Wed, 15 Jun 2005 00:34:11 +0000 (00:34 +0000)]
This patch makes it possible to use the full set of timezones when doing
"AT TIME ZONE", and not just the shorlist previously available. For
example:
SELECT CURRENT_TIMESTAMP AT TIME ZONE 'Europe/London';
works fine now. It will also obey whatever DST rules were in effect at
just that date, which the previous implementation did not.
It also supports the AT TIME ZONE on the timetz datatype. The whole
handling of DST is a bit bogus there, so I chose to make it use whatever
DST rules are in effect at the time of executig the query. not sure if
anybody is actuallyi *using* timetz though, it seems pretty
unpredictable just because of this...
Bruce Momjian [Wed, 15 Jun 2005 00:09:26 +0000 (00:09 +0000)]
>> Do you agree that using a hashtable for it in general is a good idea
>> assuming this sideeffect is removed, though?
>
>I have no problem with the hashtable, only with preloading it with
>everything. What I'd like to see is that the table inherited at fork()
>contains just the data for the default timezone. (At least in the
>normal case where that setting hasn't been changed since postmaster
>start.)
Here's a patch doing this. Changes score_timezone not to use pg_tzset(),
and thus not loading all the zones in the cache. The actual timezone
being picked will be set using set_global_timezone() which in turn calls
pg_tzset() and loads it in the cache.
Tom Lane [Tue, 14 Jun 2005 22:15:33 +0000 (22:15 +0000)]
Simplify shared-memory lock data structures as per recent discussion:
it is sufficient to track whether a backend holds a lock or not, and
store information about transaction vs. session locks only in the
inside-the-backend LocalLockTable. Since there can now be but one
PROCLOCK per lock per backend, LockCountMyLocks() is no longer needed,
thus eliminating some O(N^2) behavior when a backend holds many locks.
Also simplify the LockAcquire/LockRelease API by passing just a
'sessionLock' boolean instead of a transaction ID. The previous API
was designed with the idea that per-transaction lock holding would be
important for subtransactions, but now that we have subtransactions we
know that this is unwanted. While at it, add an 'isTempObject' parameter
to LockAcquire to indicate whether the lock is being taken on a temp
table. This is not used just yet, but will be needed shortly for
two-phase commit.
Bruce Momjian [Tue, 14 Jun 2005 17:50:38 +0000 (17:50 +0000)]
Attached is a makefile I hacked up to build pg_config under MSVC - the
reason is that it's required (more or less) in order to build the latest
DBD::Pg code and I was testing that out under MSVC.
Bruce Momjian [Tue, 14 Jun 2005 17:43:14 +0000 (17:43 +0000)]
Add GUC krb_server_hostname so the server hostname can be specified as
part of service principal. If not set, any service principal matching
an entry in the keytab can be used.
Tom Lane [Tue, 14 Jun 2005 14:21:16 +0000 (14:21 +0000)]
The random selection in function linear() could deliver a value equal to max
if geqo_rand() returns exactly 1.0, resulting in failure due to indexing
off the end of the pool array. Also, since this is using inexact float math,
it seems wise to guard against roundoff error producing values slightly
outside the expected range. Per report from bug@zedware.org.
Teodor Sigaev [Tue, 14 Jun 2005 11:45:14 +0000 (11:45 +0000)]
WAL for GiST. It work for online backup and so on, but on
recovery after crash (power loss etc) it may say that it can't restore
index and index should be reindexed.
Neil Conway [Tue, 14 Jun 2005 06:43:15 +0000 (06:43 +0000)]
Allow the parameters to PL/PgSQL's RAISE statement to be expressions,
instead of just scalar variables. Add regression tests and update the
documentation. Along the way, remove some redundant error checking
code from exec_stmt_perform().
Original patch from Pavel Stehule, reworked by Neil Conway.
Tom Lane [Tue, 14 Jun 2005 04:04:30 +0000 (04:04 +0000)]
Teach planner to optionally ignore index columns that have an equality
constraint while determining whether the index sort order matches the
query's ORDER BY. This for example allows an index on (x,y) to match
... WHERE x = 42 ORDER BY y;
It only works for btree indexes, but since those are the only ones we
currently have that are ordered at all, that's good enough for now.
Per popular demand.
Neil Conway [Tue, 14 Jun 2005 00:10:02 +0000 (00:10 +0000)]
Cleanup for "#option dump" in PL/PgSQL: don't print empty ELSE blocks,
fix two grammatical errors, and print the INTO target of EXECUTE INTO
if one is specified.
Tom Lane [Mon, 13 Jun 2005 23:14:49 +0000 (23:14 +0000)]
Change the planner to allow indexscan qualification clauses to use
nonconsecutive columns of a multicolumn index, as per discussion around
mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This
turns out to require only minimal changes in btree, and so far as I can
see none at all in GiST. btcostestimate did need some work, but its
original assumption that index selectivity == heap selectivity was
quite bogus even before this.
Neil Conway [Mon, 13 Jun 2005 06:36:22 +0000 (06:36 +0000)]
Per discussion on -hackers, this patch changes psql's "expanded" output
mode to only affect the presentation of normal query results, not the
output of psql slash commands. Documentation updated. I also made
some unrelated minor psql cleanup. Per suggestion from Stuart Cooper.
Tom Lane [Mon, 13 Jun 2005 02:26:53 +0000 (02:26 +0000)]
Adjust lo_open() so that specifying INV_READ without INV_WRITE creates
a descriptor that uses the current transaction snapshot, rather than
SnapshotNow as it did before (and still does if INV_WRITE is set).
This means pg_dump will now dump a consistent snapshot of large object
contents, as it never could do before. Also, add a lo_create() function
that is similar to lo_creat() but allows the desired OID of the large
object to be specified. This will simplify pg_restore considerably
(but I'll fix that in a separate commit).
Neil Conway [Sun, 12 Jun 2005 00:07:07 +0000 (00:07 +0000)]
This patch removes some old code from libpq that implements a URI-like
syntax for database connection parameters. It has been inside an
#ifdef NOT_USED block since 2001 or so and is marked as "broken", so
I don't think it is likely to be rehabilitated any time soon.
Neil Conway [Sun, 12 Jun 2005 00:00:21 +0000 (00:00 +0000)]
libpq was not consistently checking for memory allocation failures. This
patch adds missing checks to the call sites of malloc(), strdup(),
PQmakeEmptyPGresult(), pqResultAlloc(), and pqResultStrdup(), and updates
the documentation. Per original report from Volkan Yazici about
PQmakeEmptyPGresult() not checking for malloc() failure.
Tom Lane [Fri, 10 Jun 2005 22:25:37 +0000 (22:25 +0000)]
Separate predicate-testing code out of indxpath.c, making it a module
in its own right. As proposed by Simon Riggs, but with some editorializing
of my own.
Neil Conway [Fri, 10 Jun 2005 16:23:11 +0000 (16:23 +0000)]
Implement two new special variables in PL/PgSQL: SQLSTATE and SQLERRM.
These contain the SQLSTATE and error message of the current exception,
respectively. They are scope-local variables that are only defined
in exception handlers (so attempting to reference them outside an
exception handler is an error). Update the regression tests and the
documentation.
Also, do some minor related cleanup: export an unpack_sql_state()
function from the backend and use it to unpack a SQLSTATE into a
string, and add a free_var() function to pl_exec.c
Original patch from Pavel Stehule, review by Neil Conway.
Bruce Momjian [Fri, 10 Jun 2005 15:34:26 +0000 (15:34 +0000)]
the following patch makes the filename used to store the readline
history customizable through a variable named HISTFILE, analogous to
psql's already implemented HISTCONTROL and HISTSIZE variables, and
bash's HISTFILE-Variable.
The motivation was to be able to get psql to maintain separate
histories for separate databases. This is now easily achievable
through a line like the following in ~/.psqlrc:
Tom Lane [Fri, 10 Jun 2005 03:32:25 +0000 (03:32 +0000)]
Quick hack to allow the outer query's tuple_fraction to be passed down
to a subquery if the outer query is simple enough that the LIMIT can
be reflected directly to the subquery. This didn't use to be very
interesting, because a subquery that couldn't have been flattened into
the upper query was usually not going to be very responsive to
tuple_fraction anyway. But with new code that allows UNION ALL subqueries
to pay attention to tuple_fraction, this is useful to do. In particular
this lets the optimization occur when the UNION ALL is directly inside
a view.
Tom Lane [Fri, 10 Jun 2005 02:21:05 +0000 (02:21 +0000)]
If a LIMIT is applied to a UNION ALL query, plan each UNION arm as
if the limit were directly applied to it. This does not actually
add a LIMIT plan node to the generated subqueries --- that would be
useless overhead --- but it does cause the planner to prefer fast-
start plans when the limit is small. After an idea from Phil Endecott.
Tom Lane [Fri, 10 Jun 2005 00:28:54 +0000 (00:28 +0000)]
Revise searching of subplan target lists to use something more efficient
than tlist_member calls. Building a large join tlist is still O(N^2),
but with a much smaller constant factor than before.
Bruce Momjian [Thu, 9 Jun 2005 22:29:52 +0000 (22:29 +0000)]
This patch against 8.0.0beta1 source adds log_line_prefix options for
millisecond timestamps (%m) and remote host (%h). The milliseconds are
useful for QPS measurements.
Tom Lane [Thu, 9 Jun 2005 21:52:07 +0000 (21:52 +0000)]
Fix assign_datestyle() so that it doesn't misleadingly complain about
'conflicting datestyle specifications' for input that's actually only
redundant, such as SET DATESTYLE = MDY, MDY. Per recent gripe.
Tom Lane [Thu, 9 Jun 2005 21:25:22 +0000 (21:25 +0000)]
Make SPI set SPI_processed for CREATE TABLE AS / SELECT INTO commands;
this in turn causes CREATE TABLE AS in plpgsql to set ROW_COUNT.
This is how it behaved before 7.4; I had unintentionally changed the
behavior in a bit of sloppy micro-optimization.