Neil Conway [Wed, 23 Feb 2005 22:46:17 +0000 (22:46 +0000)]
This patch optimizes the md5_text() function (which is used to
implement the md5() SQL-level function). The old code did the
following:
1. de-toast the datum
2. convert it to a cstring via textout()
3. get the length of the cstring via strlen()
Since we are treating the datum context as a blob of binary data,
the latter two steps are unnecessary. Once the data has been
detoasted, we can just use it as-is, and derive its length from
the varlena metadata.
This patch improves some run-of-the-mill md5() computations by
just under 10% in my limited tests, and passes the regression tests.
I also noticed that md5_text() wasn't checking the return value
of md5_hash(); encountering OOM at precisely the right moment
could result in returning a random md5 hash. This patch corrects
that. A better fix would be to make md5_hash() only return on
success (and/or allocate via palloc()), but since it's used in
the frontend as well I don't see an easy way to do that.
Neil Conway [Tue, 22 Feb 2005 07:18:27 +0000 (07:18 +0000)]
This patch changes makes some significant changes to how compilation
and parsing work in PL/PgSQL:
- memory management is now done via palloc(). The compiled representation
of each function now has its own memory context. Therefore, the storage
consumed by a function can be reclaimed via MemoryContextDelete().
During compilation, the CurrentMemoryContext is the function's memory
context. This means that a palloc() is sufficient to allocate memory
that will have the same lifetime as the function itself. As a result,
code invoked during compilation should be careful to pfree() temporary
allocations to avoid leaking memory. Since a lot of the code in the
backend is not careful about releasing palloc'ed memory, that means
we should switch into a temporary memory context before invoking
backend functions. A temporary context appropriate for such allocations
is `compile_tmp_cxt'.
- The ability to use palloc() allows us to simply a lot of the code in
the parser. Rather than representing lists of elements via ad hoc
linked lists or arrays, we can use the List type. Rather than doing
malloc followed by memset(0), we can just use palloc0().
- We now check that the user has supplied the right number of parameters
to a RAISE statement. Supplying either too few or too many results in
an error (at runtime).
- PL/PgSQL's parser needs to accept arbitrary SQL statements. Since we
do not want to duplicate the SQL grammar in the PL/PgSQL grammar, this
means we need to be quite lax in what the PL/PgSQL grammar considers
a "SQL statement". This can lead to misleading behavior if there is a
syntax error in the function definition, since we assume a malformed
PL/PgSQL construct is a SQL statement. Furthermore, these errors were
only detected at runtime (when we tried to execute the alleged "SQL
statement" via SPI).
To rectify this, the patch changes the parser to invoke the main SQL
parser when it sees a string it believes to be a SQL expression. This
means that synctically-invalid SQL will be rejected during the
compilation of the PL/PgSQL function. This is only done when compiling
for "validation" purposes (i.e. at CREATE FUNCTION time), so it should
not impose a runtime overhead.
- Fixes for the various buffer overruns I've patched in stable branches
in the past few weeks. I've rewritten code where I thought it was
warranted (unlike the patches applied to older branches, which were
minimally invasive).
Bruce Momjian [Tue, 22 Feb 2005 03:56:22 +0000 (03:56 +0000)]
Add support to port/snprintf.c for position parameter specification:
+ # Determine if printf supports %1$ argument selection, e.g. %5$ selects
+ # the fifth argument after the printf print string.
+ # This is not in the C99 standard, but in the Single Unix Specification (SUS).
+ # It is used in our langauge translation strings.
Bruce Momjian [Mon, 21 Feb 2005 17:30:33 +0000 (17:30 +0000)]
Clarify item:
< * Allow server configuration parameters to be remotely modified
> * Allow pg_hba.conf settings to be controlled via SQL
>
> This would require a new global table that is dumped to flat file for
> use by the postmaster. We do a similar thing for pg_shadow currently.
>
Bruce Momjian [Mon, 21 Feb 2005 04:58:52 +0000 (04:58 +0000)]
Clarify item:
< * Consider use of open/fcntl(O_DIRECT) to minimize OS caching
> * Consider use of open/fcntl(O_DIRECT) to minimize OS caching,
> especially for WAL writes
Tom Lane [Sun, 20 Feb 2005 22:02:19 +0000 (22:02 +0000)]
Use SnapshotNow instead of SnapshotSelf for reading the catalogs
during flat-file writing. The only difference is that SnapshotSelf
would consider tuples of the 'current command' within the current
transaction as valid, where SnapshotNow wouldn't. We can eliminate
the need for this with one extra CommandCounterIncrement call before
we start reading the catalogs.
Tom Lane [Sun, 20 Feb 2005 21:46:50 +0000 (21:46 +0000)]
Remove some no-longer-needed kluges for bootstrapping, in particular
the AMI_OVERRIDE flag. The fact that TransactionLogFetch treats
BootstrapTransactionId as always committed is sufficient to make
bootstrap work, and getting rid of extra tests in heavily used code
paths seems like a win. The files produced by initdb are demonstrably
the same after this change.
Tom Lane [Sun, 20 Feb 2005 04:45:59 +0000 (04:45 +0000)]
Flat file cleanup phase 2: make it work for pg_group. The flat group
file now identifies group members by usesysid not name; this avoids
needing to depend on SearchSysCache which we can't use during startup.
(The old representation was entirely broken anyway, since we did not
regenerate the file following RENAME USER.) It's only a 95% solution
because if the group membership list is big enough to be toasted out
of line, we cannot read it during startup. I think this will do for
the moment, until we have time to implement the planned pg_role
replacement for pg_group.
Tom Lane [Sun, 20 Feb 2005 02:22:07 +0000 (02:22 +0000)]
Add code to prevent transaction ID wraparound by enforcing a safe limit
in GetNewTransactionId(). Since the limit value has to be computed
before we run any real transactions, this requires adding code to database
startup to scan pg_database and determine the oldest datfrozenxid.
This can conveniently be combined with the first stage of an attack on
the problem that the 'flat file' copies of pg_shadow and pg_group are
not properly updated during WAL recovery. The code I've added to
startup resides in a new file src/backend/utils/init/flatfiles.c, and
it is responsible for rewriting the flat files as well as initializing
the XID wraparound limit value. This will eventually allow us to get
rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add
a trigger to pg_database.
Tom Lane [Sat, 19 Feb 2005 23:16:15 +0000 (23:16 +0000)]
New arrangement to always let the bgwriter do checkpoints broke
CHECKPOINT and some other commands in the context of a standalone
backend. Allow a standalone backend to do its own checkpoints.
Tom Lane [Sat, 19 Feb 2005 19:33:08 +0000 (19:33 +0000)]
Ensure that the resolved datatype of any unknown Param is propagated
into the sub-SELECT targetlist when it appears in the context
INSERT INTO foo SELECT $1 ... Per report from Abhijit Menon-Sen.
Tom Lane [Fri, 11 Feb 2005 22:15:12 +0000 (22:15 +0000)]
Add a regression test to verify that the stack depth checker actually
works (and max_stack_depth is not set too high for the platform).
Inspired by trouble report from Brian Betts.
Neil Conway [Fri, 11 Feb 2005 04:09:05 +0000 (04:09 +0000)]
Adjust input routines for float4, float8 and oid to reject the empty string
as valid input (it was previously treated as 0). This input was deprecated
in 8.0 (and a warning was emitted). Regression tests updated.
Tom Lane [Thu, 10 Feb 2005 20:36:28 +0000 (20:36 +0000)]
Fix SPI cursor support to allow scanning the results of utility commands
that return tuples (such as EXPLAIN). Per gripe from Michael Fuhr.
Side effect: fix an old bug that unintentionally disabled backward scans
for all SPI-created cursors.
Neil Conway [Wed, 9 Feb 2005 23:17:26 +0000 (23:17 +0000)]
ALTER TABLE ADD COLUMN exhibits a significant memory leak when adding a
column with a default expression. In that situation, we need to rewrite
the heap relation. To evaluate the new default expression, we use
ExecEvalExpr(); however, this can allocate memory in the current memory
context, and ATRewriteTable() does not switch out of the active portal's
heap memory context. The end result is a rather large memory leak (on
the order of gigabytes for a reasonably sized table).
This patch changes ATRewriteTable() to switch to the per-tuple memory
context before beginning the per-tuple loop. It also removes an explicit
heap_freetuple() in the loop, since that is no longer needed.
In an unrelated change, I noticed the code was scanning through the
attributes of the new tuple descriptor for each tuple of the old table.
I changed this to use precomputation, which should slightly speed up
the loop.
Bruce Momjian [Tue, 8 Feb 2005 03:21:02 +0000 (03:21 +0000)]
It seems like most people don't want automatic failover so I am removing
the item:
< o Automatic failover
<
< The proper solution to this will probably the use of a master/slave
< replication solution like Sloney and a connection pooling tool like
< pgpool.
<
Tom Lane [Sun, 6 Feb 2005 20:19:08 +0000 (20:19 +0000)]
Repair CLUSTER failure after ALTER TABLE SET WITHOUT OIDS. Turns out
there are corner cases involving dropping toasted columns in which the
previous coding would fail, too: the new version of the table might not
have any TOAST table, but we'd still propagate possibly-wide values of
dropped columns forward.
Tom Lane [Sat, 5 Feb 2005 19:38:58 +0000 (19:38 +0000)]
Marginal hack to merge adjacent ReleaseBuffer/ReadBuffer calls into
ReleaseAndReadBuffer during GIST index searches. We already did this
in btree and rtree, might as well do it here too.
Tom Lane [Thu, 3 Feb 2005 23:38:58 +0000 (23:38 +0000)]
Fix minor thinko in logic to set dump order when dumping from a pre-7.3
database: aggregates should be dumped in the same pass as operators,
not in the same pass as functions.
Tom Lane [Thu, 3 Feb 2005 23:29:19 +0000 (23:29 +0000)]
Ensure that all details of the ARC algorithm are hidden within freelist.c.
This refactoring does not change any algorithms or data structures, just
remove visibility of the ARC datastructures from other source files.
Tom Lane [Wed, 2 Feb 2005 21:49:09 +0000 (21:49 +0000)]
Adjust constant-folding of CASE expressions so that the simple comparison
form of CASE (eg, CASE 0 WHEN 1 THEN ...) can be constant-folded as it
was in 7.4. Also, avoid constant-folding result expressions that are
certainly unreachable --- the former coding was a bit cavalier about this
and could generate unexpected results for all-constant CASE expressions.
Add regression test cases. Per report from Vlad Marchenko.
Bruce Momjian [Wed, 2 Feb 2005 17:26:49 +0000 (17:26 +0000)]
Update RESET ALL items:
< all temporary tables, removal of any NOTIFYs, etc. This could be used
< for connection pooling. We could also change RESET ALL to have this
< functionality.
> all temporary tables, removal of any NOTIFYs, cursors, prepared
> queries(?), currval()s, etc. This could be used for connection pooling.
> We could also change RESET ALL to have this functionality.
Neil Conway [Wed, 2 Feb 2005 06:36:02 +0000 (06:36 +0000)]
Add support for temporary views, including documentation and regression
tests. Contributed by Koju Iijima, review from Neil Conway, Gavin Sherry
and Tom Lane.
Also, fix error in description of WITH CHECK OPTION clause in the CREATE
VIEW reference page: it should be "CASCADED", not "CASCADE".
Tom Lane [Tue, 1 Feb 2005 23:08:13 +0000 (23:08 +0000)]
Adjust estimate_num_groups() to not clamp per-relation group count
estimate to less than the number of values estimated for any one grouping
Var, as suggested by Manfred. This is intuitively right, and what's
more it puts the plan choices in the subselect regression test back the
way they were before ...
Tom Lane [Tue, 1 Feb 2005 19:35:14 +0000 (19:35 +0000)]
Adjust plpgsql to allow assignment to an element of an array that is
initially NULL. For 8.0 we changed the main executor to have this
behavior in an UPDATE of an array column, but plpgsql's equivalent case
was overlooked. Per report from Sven Willenberger.