Bruce Momjian [Tue, 30 May 2006 14:09:32 +0000 (14:09 +0000)]
Add pgmagic header block to store compile-time constants:
It now only checks four things:
Major version number (7.4 or 8.1 for example)
NAMEDATALEN
FUNC_MAX_ARGS
INDEX_MAX_KEYS
The three constants were chosen because:
1. We document them in the config page in the docs
2. We mark them as changable in pg_config_manual.h
3. Changing any of these will break some of the more popular modules:
FUNC_MAX_ARGS changes fmgr interface, every module uses this NAMEDATALEN
changes syscache interface, every PL as well as tsearch uses this
INDEX_MAX_KEYS breaks tsearch and anything using GiST.
Bruce Momjian [Tue, 30 May 2006 12:56:45 +0000 (12:56 +0000)]
Re-defines SHA2 symbols so that they would not conflict with certain
versions of OpenSSL. If your OpenSSL does not contain SHA2, then there
should be no conflict. But ofcourse, if someone upgrades OpenSSL,
server starts crashing.
Bruce Momjian [Tue, 30 May 2006 11:40:21 +0000 (11:40 +0000)]
Update PL documentation:
An article at WebProNews quoted from the PG docs as to the merits of
stored procedures. I have added a bit more material on their merits,
as well as making a few changes to improve the introductions to
PL/Perl and PL/Tcl.
Delay write of pg_stats file to once every five minutes, during
shutdown, or when requested by a backend:
It changes so the file is only written once every 5 minutes (changeable
of course, I just picked something) instead of once every half second.
It's still written when the stats collector shuts down, just as before.
And it is now also written on backend request. A backend requests a
rewrite by simply sending a special stats message. It operates on the
assumption that the backends aren't actually going to read the
statistics file very often, compared to how frequent it's written today.
Andrew Dunstan [Mon, 29 May 2006 13:51:23 +0000 (13:51 +0000)]
Make plperl's $_TD trigger data a global rather than a lexical variable,
with a fresh local value for each invocation, to avoid unexpected sharing
violations. Per recent -hackers discussion.
Teodor Sigaev [Mon, 29 May 2006 12:50:06 +0000 (12:50 +0000)]
Som improve page split in multicolumn GiST index.
If user picksplit on n-th column generate equals
left and right unions then it calls picksplit on n+1-th
column.
Tom Lane [Sun, 28 May 2006 21:13:54 +0000 (21:13 +0000)]
Fix up pg_dump to do string escaping fully correctly for client encoding
and standard_conforming_strings; likewise for the other client programs
that need it. As per previous discussion, a pg_dump dump now conforms
to the standard_conforming_strings setting of the source database.
We don't use E'' syntax in the dump, thereby improving portability of
the SQL. I added a SET escape_strings_warning = off command to keep
the dumps from getting a lot of back-chatter from that.
Tom Lane [Sat, 27 May 2006 19:45:52 +0000 (19:45 +0000)]
Revert ill-considered change to plpgsql: it should not rely on the
current setting of standard_conforming_strings to decide how to quote
strings that will be used later. There is much more to do here but
this particular change breaks the build on Windows, so fix it now.
Tom Lane [Sat, 27 May 2006 18:07:06 +0000 (18:07 +0000)]
Fix initdb to properly escape quotes and backslashes in the supplied
superuser password, and also in the paths of the various files it issues
SQL COPY commands for. Per bug #2424.
Tom Lane [Sat, 27 May 2006 17:38:46 +0000 (17:38 +0000)]
Re-introduce the yylex filter function formerly used to support UNION
JOIN, which I removed in a recent fit of over-optimism that we wouldn't
have any future use for it. Now it's needed to support disambiguating
WITH CHECK OPTION from WITH TIME ZONE. As proof of concept, add stub
grammar productions for WITH CHECK OPTION.
Bruce Momjian [Fri, 26 May 2006 23:48:54 +0000 (23:48 +0000)]
Use E'' strings internally only when standard_conforming_strings =
'off'. This allows pg_dump output with standard_conforming_strings =
'on' to generate proper strings that can be loaded into other databases
without the backslash doubling we typically do. I have added the
dumping of the standard_conforming_strings value to pg_dump.
I also added standard backslash handling for plpgsql.
Tom Lane [Fri, 26 May 2006 22:50:02 +0000 (22:50 +0000)]
Further hacking on performance of COPY OUT. It seems that fwrite()'s
per-call overhead is quite significant, at least on Linux: whatever
it's doing is more than just shoving the bytes into a buffer. Buffering
the data so we can call fwrite() just once per row seems to be a win.
Andrew Dunstan [Fri, 26 May 2006 17:34:16 +0000 (17:34 +0000)]
Add table_name and table_schema to plperl trigger data. relname is
kept but now deprecated. Patch from Adam Sjøgren. Add regression test to
show plperl trigger data (Andrew).
TBD: apply similar changes to plpgsql, plpython and pltcl.
Tom Lane [Wed, 24 May 2006 21:20:11 +0000 (21:20 +0000)]
Fix pg_restore to process BLOB COMMENT entries correctly; they aren't
really tables and shouldn't get DISABLE TRIGGER processing. Per bug
#2452 from Robert Treat.
Bruce Momjian [Wed, 24 May 2006 18:12:38 +0000 (18:12 +0000)]
Remove pg_attribute detail, add URL
< pg_attribute.attislocal has to be set to 'false' for ADD, and
< pg_attribute.attinhcount adjusted appropriately
> See http://archives.postgresql.org/pgsql-hackers/2006-05/msg00988.php.
Teodor Sigaev [Wed, 24 May 2006 11:01:39 +0000 (11:01 +0000)]
* Add support NULL to GiST.
* some refactoring and simplify code int gistutil.c and gist.c
* now in some cases it can be called used-defined
picksplit method for non-first column in index, but here
is a place to do more.
* small fix of docs related to support NULL.
Tom Lane [Tue, 23 May 2006 21:37:59 +0000 (21:37 +0000)]
Tweak writetup_heap/readtup_heap to avoid storing the tuple identity
and transaction visibility fields of tuples being sorted. These are
always uninteresting in a tuple being sorted (if the fields were actually
selected, they'd have been pulled out into user columns beforehand).
This saves about 24 bytes per row being sorted, which is a useful savings
for any but the widest of sort rows. Per recent discussion.
Bruce Momjian [Tue, 23 May 2006 20:57:53 +0000 (20:57 +0000)]
Add:
> o Add ALTER TABLE tab ADD/DROP INHERITS parent
>
> pg_attribute.attislocal has to be set to 'false' for ADD, and
> pg_attribute.attinhcount adjusted appropriately
>
Tom Lane [Tue, 23 May 2006 15:21:52 +0000 (15:21 +0000)]
Remove CXT_printf/CXT1_printf macros. If anyone had found them to be of
any use in the past many years, we'd have made some effort to include
them in all executor node types; but in fact they were only in
nodeAppend.c and nodeIndexscan.c, up until I copied nodeIndexscan.c's
occurrence into the new bitmap node types. Remove some other unused
macros in execdebug.h, too. Some day the whole header probably ought to
go away in favor of better-designed facilities.
Bruce Momjian [Mon, 22 May 2006 20:39:23 +0000 (20:39 +0000)]
Make "trigger" section:
> * Referential Integrity
>
> o Add MATCH PARTIAL referential integrity
> o Change foreign key constraint for array -> element to mean element
> in array?
> o Enforce referential integrity for system tables
>
>
< Referential Integrity
< =====================
<
< * Add MATCH PARTIAL referential integrity
> Triggers
> ========
< * Change foreign key constraint for array -> element to mean element
< in array? 801d804
< * Enforce referential integrity for system tables
Tom Lane [Sun, 21 May 2006 20:22:16 +0000 (20:22 +0000)]
Fix errors in fortuna PRNG reseeding logic that could cause a predictable
session key to be selected by pgp_sym_encrypt() in some cases. This only
affects non-OpenSSL-using builds. Marko Kreen
Tom Lane [Sun, 21 May 2006 20:19:23 +0000 (20:19 +0000)]
Modify libpq's string-escaping routines to be aware of encoding considerations
and standard_conforming_strings. The encoding changes are needed for proper
escaping in multibyte encodings, as per the SQL-injection vulnerabilities
noted in CVE-2006-2313 and CVE-2006-2314. Concurrent fixes are being applied
to the server to ensure that it rejects queries that may have been corrupted
by attempted SQL injection, but this merely guarantees that unpatched clients
will fail rather than allow injection. An actual fix requires changing the
client-side code. While at it we have also fixed these routines to understand
about standard_conforming_strings, so that the upcoming changeover to SQL-spec
string syntax can be somewhat transparent to client code.
Since the existing API of PQescapeString and PQescapeBytea provides no way to
inform them which settings are in use, these functions are now deprecated in
favor of new functions PQescapeStringConn and PQescapeByteaConn. The new
functions take the PGconn to which the string will be sent as an additional
parameter, and look inside the connection structure to determine what to do.
So as to provide some functionality for clients using the old functions,
libpq stores the latest encoding and standard_conforming_strings values
received from the backend in static variables, and the old functions consult
these variables. This will work reliably in clients using only one Postgres
connection at a time, or even multiple connections if they all use the same
encoding and string syntax settings; which should cover many practical
scenarios.
Clients that use homebrew escaping methods, such as PHP's addslashes()
function or even hardwired regexp substitution, will require extra effort
to fix :-(. It is strongly recommended that such code be replaced by use of
PQescapeStringConn/PQescapeByteaConn if at all feasible.
Tom Lane [Sun, 21 May 2006 20:10:42 +0000 (20:10 +0000)]
Add a new GUC parameter backslash_quote, which determines whether the SQL
parser will allow "\'" to be used to represent a literal quote mark. The
"\'" representation has been deprecated for some time in favor of the
SQL-standard representation "''" (two single quote marks), but it has been
used often enough that just disallowing it immediately won't do. Hence
backslash_quote allows the settings "on", "off", and "safe_encoding",
the last meaning to allow "\'" only if client_encoding is a valid server
encoding. That is now the default, and the reason is that in encodings
such as SJIS that allow 0x5c (ASCII backslash) to be the last byte of a
multibyte character, accepting "\'" allows SQL-injection attacks as per
CVE-2006-2314 (further details will be published after release). The
"on" setting is available for backward compatibility, but it must not be
used with clients that are exposed to untrusted input.
Thanks to Akio Ishida and Yasuo Ohgaki for identifying this security issue.
Tom Lane [Sun, 21 May 2006 20:05:21 +0000 (20:05 +0000)]
Change the backend to reject strings containing invalidly-encoded multibyte
characters in all cases. Formerly we mostly just threw warnings for invalid
input, and failed to detect it at all if no encoding conversion was required.
The tighter check is needed to defend against SQL-injection attacks as per
CVE-2006-2313 (further details will be published after release). Embedded
zero (null) bytes will be rejected as well. The checks are applied during
input to the backend (receipt from client or COPY IN), so it no longer seems
necessary to check in textin() and related routines; any string arriving at
those functions will already have been validated. Conversion failure
reporting (for characters with no equivalent in the destination encoding)
has been cleaned up and made consistent while at it.
Also, fix a few longstanding errors in little-used encoding conversion
routines: win1251_to_iso, win866_to_iso, euc_tw_to_big5, euc_tw_to_mic,
mic_to_euc_tw were all broken to varying extents.
Patches by Tatsuo Ishii and Tom Lane. Thanks to Akio Ishida and Yasuo Ohgaki
for identifying the security issues.
Alvaro Herrera [Fri, 19 May 2006 19:08:27 +0000 (19:08 +0000)]
Add last-vacuum/analyze-time columns to the stats collector, both manual and
issued by autovacuum. Add accessor functions to them, and use those in the
pg_stat_*_tables system views.
Catalog version bumped due to changes in the pgstat views and the pgstat file.
Patch from Larry Rosenman, minor improvements by me.
Tom Lane [Thu, 18 May 2006 19:56:46 +0000 (19:56 +0000)]
Fix choose_bitmap_and() so that partial index predicates are considered when
deciding whether a potential additional indexscan is redundant or not. As now
coded, any use of a partial index that was already used in a previous AND arm
will be rejected as redundant. This might be overly restrictive, but not
considering the point at all is definitely bad, as per example in bug #2441
from Arjen van der Meijden. In particular, a clauseless scan of a partial
index was *never* considered redundant by the previous coding, and that's
surely wrong. Being more flexible would also require some consideration
of how not to double-count the index predicate's selectivity.
Tom Lane [Thu, 18 May 2006 18:57:31 +0000 (18:57 +0000)]
When a bitmap indexscan is using a partial index, it is necessary to include
the partial index predicate in the scan's "recheck condition". Otherwise,
if the scan becomes lossy for lack of bitmap memory, we would fail to enforce
that returned rows satisfy the predicate. Noted while studying bug #2441
from Arjen van der Meijden.
Tom Lane [Thu, 18 May 2006 17:12:10 +0000 (17:12 +0000)]
Fix thinko in recent changes to handle ScalarArrayOpExpr as an indexable
condition: when there are multiple possible index paths involving
ScalarArrayOpExprs, they are logically to be ANDed together not ORed.
This thinko was a direct consequence of trying to put the processing
inside generate_bitmap_or_paths(), which I now see was a bit too cute.
So pull it out and make the callers do it separately (there are only two
that need it anyway). Partially responds to bug #2441 from Arjen van der Meijden.
There are some additional infelicities exposed by his example, but they
are also in 8.1.x, while this mistake is not.