Bruce Momjian [Sat, 26 Feb 2005 23:19:05 +0000 (23:19 +0000)]
Adjust OS-specific kernel settings to mention old and new BSD methods of
adjusting values:
> But to be on the safe side, it would make sense to do something similar
> to the BSD section, and comment about older distributions maybe needing
> to manipulate /proc/kernel/* directly.
Tom Lane [Sat, 26 Feb 2005 18:43:34 +0000 (18:43 +0000)]
Finish up the flat-files project: get rid of GetRawDatabaseInfo() hack
in favor of looking at the flat file copy of pg_database during backend
startup. This should finally eliminate the various corner cases in which
backend startup fails unexpectedly because it isn't able to distinguish
live and dead tuples in pg_database. Simplify locking on pg_database
to be similar to the rules used with pg_shadow and pg_group, and eliminate
FlushRelationBuffers operations that were used only to reduce the odds
of failure of GetRawDatabaseInfo.
initdb forced due to addition of a trigger to pg_database.
Bruce Momjian [Sat, 26 Feb 2005 18:39:04 +0000 (18:39 +0000)]
Clarify PGPASSWORD usage:
! authentication. Use of this environment variable is not
! recommended for security reasons (some operating systems
! allow non-root users to see process environment variables via
! <application>ps</>); instead consider using the
! <filename>~/.pgpass</> file (see <xref linkend="libpq-pgpass">).
Neil Conway [Wed, 23 Feb 2005 22:46:17 +0000 (22:46 +0000)]
This patch optimizes the md5_text() function (which is used to
implement the md5() SQL-level function). The old code did the
following:
1. de-toast the datum
2. convert it to a cstring via textout()
3. get the length of the cstring via strlen()
Since we are treating the datum context as a blob of binary data,
the latter two steps are unnecessary. Once the data has been
detoasted, we can just use it as-is, and derive its length from
the varlena metadata.
This patch improves some run-of-the-mill md5() computations by
just under 10% in my limited tests, and passes the regression tests.
I also noticed that md5_text() wasn't checking the return value
of md5_hash(); encountering OOM at precisely the right moment
could result in returning a random md5 hash. This patch corrects
that. A better fix would be to make md5_hash() only return on
success (and/or allocate via palloc()), but since it's used in
the frontend as well I don't see an easy way to do that.
Neil Conway [Tue, 22 Feb 2005 07:18:27 +0000 (07:18 +0000)]
This patch changes makes some significant changes to how compilation
and parsing work in PL/PgSQL:
- memory management is now done via palloc(). The compiled representation
of each function now has its own memory context. Therefore, the storage
consumed by a function can be reclaimed via MemoryContextDelete().
During compilation, the CurrentMemoryContext is the function's memory
context. This means that a palloc() is sufficient to allocate memory
that will have the same lifetime as the function itself. As a result,
code invoked during compilation should be careful to pfree() temporary
allocations to avoid leaking memory. Since a lot of the code in the
backend is not careful about releasing palloc'ed memory, that means
we should switch into a temporary memory context before invoking
backend functions. A temporary context appropriate for such allocations
is `compile_tmp_cxt'.
- The ability to use palloc() allows us to simply a lot of the code in
the parser. Rather than representing lists of elements via ad hoc
linked lists or arrays, we can use the List type. Rather than doing
malloc followed by memset(0), we can just use palloc0().
- We now check that the user has supplied the right number of parameters
to a RAISE statement. Supplying either too few or too many results in
an error (at runtime).
- PL/PgSQL's parser needs to accept arbitrary SQL statements. Since we
do not want to duplicate the SQL grammar in the PL/PgSQL grammar, this
means we need to be quite lax in what the PL/PgSQL grammar considers
a "SQL statement". This can lead to misleading behavior if there is a
syntax error in the function definition, since we assume a malformed
PL/PgSQL construct is a SQL statement. Furthermore, these errors were
only detected at runtime (when we tried to execute the alleged "SQL
statement" via SPI).
To rectify this, the patch changes the parser to invoke the main SQL
parser when it sees a string it believes to be a SQL expression. This
means that synctically-invalid SQL will be rejected during the
compilation of the PL/PgSQL function. This is only done when compiling
for "validation" purposes (i.e. at CREATE FUNCTION time), so it should
not impose a runtime overhead.
- Fixes for the various buffer overruns I've patched in stable branches
in the past few weeks. I've rewritten code where I thought it was
warranted (unlike the patches applied to older branches, which were
minimally invasive).
Bruce Momjian [Tue, 22 Feb 2005 03:56:22 +0000 (03:56 +0000)]
Add support to port/snprintf.c for position parameter specification:
+ # Determine if printf supports %1$ argument selection, e.g. %5$ selects
+ # the fifth argument after the printf print string.
+ # This is not in the C99 standard, but in the Single Unix Specification (SUS).
+ # It is used in our langauge translation strings.
Bruce Momjian [Mon, 21 Feb 2005 17:30:33 +0000 (17:30 +0000)]
Clarify item:
< * Allow server configuration parameters to be remotely modified
> * Allow pg_hba.conf settings to be controlled via SQL
>
> This would require a new global table that is dumped to flat file for
> use by the postmaster. We do a similar thing for pg_shadow currently.
>
Bruce Momjian [Mon, 21 Feb 2005 04:58:52 +0000 (04:58 +0000)]
Clarify item:
< * Consider use of open/fcntl(O_DIRECT) to minimize OS caching
> * Consider use of open/fcntl(O_DIRECT) to minimize OS caching,
> especially for WAL writes
Tom Lane [Sun, 20 Feb 2005 22:02:19 +0000 (22:02 +0000)]
Use SnapshotNow instead of SnapshotSelf for reading the catalogs
during flat-file writing. The only difference is that SnapshotSelf
would consider tuples of the 'current command' within the current
transaction as valid, where SnapshotNow wouldn't. We can eliminate
the need for this with one extra CommandCounterIncrement call before
we start reading the catalogs.
Tom Lane [Sun, 20 Feb 2005 21:46:50 +0000 (21:46 +0000)]
Remove some no-longer-needed kluges for bootstrapping, in particular
the AMI_OVERRIDE flag. The fact that TransactionLogFetch treats
BootstrapTransactionId as always committed is sufficient to make
bootstrap work, and getting rid of extra tests in heavily used code
paths seems like a win. The files produced by initdb are demonstrably
the same after this change.
Tom Lane [Sun, 20 Feb 2005 04:45:59 +0000 (04:45 +0000)]
Flat file cleanup phase 2: make it work for pg_group. The flat group
file now identifies group members by usesysid not name; this avoids
needing to depend on SearchSysCache which we can't use during startup.
(The old representation was entirely broken anyway, since we did not
regenerate the file following RENAME USER.) It's only a 95% solution
because if the group membership list is big enough to be toasted out
of line, we cannot read it during startup. I think this will do for
the moment, until we have time to implement the planned pg_role
replacement for pg_group.
Tom Lane [Sun, 20 Feb 2005 02:22:07 +0000 (02:22 +0000)]
Add code to prevent transaction ID wraparound by enforcing a safe limit
in GetNewTransactionId(). Since the limit value has to be computed
before we run any real transactions, this requires adding code to database
startup to scan pg_database and determine the oldest datfrozenxid.
This can conveniently be combined with the first stage of an attack on
the problem that the 'flat file' copies of pg_shadow and pg_group are
not properly updated during WAL recovery. The code I've added to
startup resides in a new file src/backend/utils/init/flatfiles.c, and
it is responsible for rewriting the flat files as well as initializing
the XID wraparound limit value. This will eventually allow us to get
rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add
a trigger to pg_database.
Tom Lane [Sat, 19 Feb 2005 23:16:15 +0000 (23:16 +0000)]
New arrangement to always let the bgwriter do checkpoints broke
CHECKPOINT and some other commands in the context of a standalone
backend. Allow a standalone backend to do its own checkpoints.
Tom Lane [Sat, 19 Feb 2005 19:33:08 +0000 (19:33 +0000)]
Ensure that the resolved datatype of any unknown Param is propagated
into the sub-SELECT targetlist when it appears in the context
INSERT INTO foo SELECT $1 ... Per report from Abhijit Menon-Sen.
Tom Lane [Fri, 11 Feb 2005 22:15:12 +0000 (22:15 +0000)]
Add a regression test to verify that the stack depth checker actually
works (and max_stack_depth is not set too high for the platform).
Inspired by trouble report from Brian Betts.
Neil Conway [Fri, 11 Feb 2005 04:09:05 +0000 (04:09 +0000)]
Adjust input routines for float4, float8 and oid to reject the empty string
as valid input (it was previously treated as 0). This input was deprecated
in 8.0 (and a warning was emitted). Regression tests updated.
Tom Lane [Thu, 10 Feb 2005 20:36:28 +0000 (20:36 +0000)]
Fix SPI cursor support to allow scanning the results of utility commands
that return tuples (such as EXPLAIN). Per gripe from Michael Fuhr.
Side effect: fix an old bug that unintentionally disabled backward scans
for all SPI-created cursors.
Neil Conway [Wed, 9 Feb 2005 23:17:26 +0000 (23:17 +0000)]
ALTER TABLE ADD COLUMN exhibits a significant memory leak when adding a
column with a default expression. In that situation, we need to rewrite
the heap relation. To evaluate the new default expression, we use
ExecEvalExpr(); however, this can allocate memory in the current memory
context, and ATRewriteTable() does not switch out of the active portal's
heap memory context. The end result is a rather large memory leak (on
the order of gigabytes for a reasonably sized table).
This patch changes ATRewriteTable() to switch to the per-tuple memory
context before beginning the per-tuple loop. It also removes an explicit
heap_freetuple() in the loop, since that is no longer needed.
In an unrelated change, I noticed the code was scanning through the
attributes of the new tuple descriptor for each tuple of the old table.
I changed this to use precomputation, which should slightly speed up
the loop.