Tom Lane [Sun, 5 Aug 2001 02:06:50 +0000 (02:06 +0000)]
Endeavor to make pgstats buffer process (a) safe and (b) useful.
Make sure it exits immediately when collector process dies --- in old code,
buffer process would hang around and compete with the new buffer process
for packets. Make sure it doesn't block on writing the pipe when the
collector falls more than a pipeload behind. Avoid leaking pgstats FDs
into every backend.
Bruce Momjian [Sat, 4 Aug 2001 19:42:34 +0000 (19:42 +0000)]
This patch is because Hurd does not support NOFILE. It is against current
cvs.
The Debian bug report says, "The upstream source makes use of NOFILE
unconditionalized. As the Hurd doesn't have an arbitrary limit on the
number of open files, this is not defined. But _SC_OPEN_MAX works fine
and returns 1024 (applications can increase this as they want), so I
suggest the below diff. Please forward this upstream, too."
Bruce Momjian [Sat, 4 Aug 2001 19:41:00 +0000 (19:41 +0000)]
This patch adds the following to the FTI module:
* The ability to index more than one column in a table with a single
trigger.
* All uses of sprintf changed to snprintf to prevent users from crashing
Postgres.
* Error messages made more consistent
* Some changes made to bring it into line with coding requirements for
triggers specified in the docs. (ie. check you're a trigger before casting
your context)
* The perl script that generate indices has been updated to support indexing
multiple columns in a table.
* Fairly well tested in our development environment indexing a food
database's brand and description fields. The size of the fti index is
around 300,000 rows.
* All docs and examples upgraded. This includes specifying more efficient
index usage that was specified before, better examples that don't produce
duplicates, etc.
Bruce Momjian [Sat, 4 Aug 2001 19:33:49 +0000 (19:33 +0000)]
> 1) When a row is retrieved, and then a SQL_FETCH_FIRST is issued, the
check
> in convert.c
> does not consider the fact that the value in the field has been altered to
> be a '1' if the
> backend handed it a 't'. The net result being that the first row on any
> subsequent queries
> has all it's boolean set to 0.
Bruce Momjian [Sat, 4 Aug 2001 19:32:04 +0000 (19:32 +0000)]
Attached is a patch that does the following:
1) improves performance of commit/rollback by reducing number of round
trips to the server
2) uses 7.1 functionality for setting the transaction isolation level
3) backs out a patch from 11 days ago because that code failed to
compile under jdk1.1
Details:
1) The old code was doing the following for each commit:
commit
begin
set transaction isolation level xxx
thus a call to commit was performing three round trips to the database.
The new code does this in one round trip as:
commit; begin; set transaction isolation level xxx
In a simple test program that performs 1000 transactions (where each
transaction does one simple select inside that transaction) has the
following before and after timings:
Client and Server on same machine
old new
--- ---
1.877sec 1.405sec 25.1% improvement
Client and Server on different machines
old new
--- ---
4.184sec 2.927sec 34.3% improvement
(all timings are an average of four different runs)
2) The driver was using 'set transaction isolation level xxx' at the
begining of each transaction, instead of using the new 7.1 syntax of
'set session characteristics as transaction isolation level xxx' which
only needs to be done once instead of for each transaction. This is
done conditionally (i.e. if server is 7.0 or older do the old behaviour,
else do the new behaviour) to not break backward compatibility. This
also required the movement of some code to check/test database version
numbers from the DatabaseMetaData object to the Connection object.
3) Finally while testing, I discovered that the code that was checked in
11 days ago actually didn't compile. The code in the patch for
Connection.setCatalog() used Properties.setProperty() which only exists
in JDK1.2 or higher. Thus compiling the JDBC1 driver failed as this
method doesn't exist. Thus I backed out that patch.
Tom Lane [Sat, 4 Aug 2001 00:14:43 +0000 (00:14 +0000)]
Fix handling of SIGCHLD, per recent pghackers discussion: on some
platforms system(2) gets confused unless the signal handler is set to
SIG_DFL, not SIG_IGN. pgstats.c now uses pqsignal() as it should,
not signal(). Also, arrange for the stats collector process to show
a reasonable ID in 'ps', rather than looking like a postmaster.
Tom Lane [Fri, 3 Aug 2001 22:09:55 +0000 (22:09 +0000)]
Fix win32.mak to support MULTIBYTE build --- it was pulling in several
backend files that it shouldn't anymore, causing compile failures.
Per report from Darko Prenosil.
Tom Lane [Fri, 3 Aug 2001 20:47:40 +0000 (20:47 +0000)]
For some reason, CREATE TYPE has only accepted alignment specifications
of 'int4' and 'double'. Add 'char' and 'int2' to allow user-defined types
to access the full set of supported alignments.
Tom Lane [Fri, 3 Aug 2001 19:43:05 +0000 (19:43 +0000)]
Fix pg_dump so that comments on views are dumped in the proper sequence.
Dump the alignment and storage information for user-defined types (how'd
that manage to slip through the cracks?), and don't dump 'shell' types
that don't have typisdefined set. Fix badly broken logic for dependencies
of type definitions (did not work for more than one user-defined type...).
Avoid memory leakage within pg_dump by being more careful to release
storage used by PQExpBuffer objects.
Tom Lane [Thu, 2 Aug 2001 21:31:23 +0000 (21:31 +0000)]
Clean up various memory leaks within plpgsql, and re-enable the
exec_eval_simple_expr shortcut, which was diked out in 7.1 because it
leaked too much space. CVS tip now leaks no memory in Chris Ruprecht's
example, which formerly leaked to the tune of 500 MB. (Much of this
is work that Jan already did; this commit just cleans up around the
edges.)
Tom Lane [Thu, 2 Aug 2001 18:08:43 +0000 (18:08 +0000)]
Add a SPI_copytupledesc function that parallels SPI_copytuple --- ie,
it copies the tupdesc into upper-executor memory. This is necessary
for returning tuple descriptors without leaking all of lower exec memory.
Bruce Momjian [Thu, 2 Aug 2001 15:45:55 +0000 (15:45 +0000)]
I noticed that pltcl didn't have any way to get to SPI_lastoid like plpgsql does.. I started using pltcl a lot because I like to decide when and how my queries get planned.. so I put one together really quick
Sorry I don't have the original around to make a quick diff, but its a very small change... I think this should be in the next release, there's no reason not to have it.
its a function with no expected arguments, so you can use it like:
spi_exec "INSERT INTO mytable(columns...) VALUES(values..)"
set oid [spi_lastoid]
spi_exec "SELECT mytable_id from mytable WHERE oid=$oid"
It just didn't make sense for me to use plpgsql and pltcl, or just screw
them both and use SPI from C.
Tom Lane [Thu, 2 Aug 2001 14:39:35 +0000 (14:39 +0000)]
Even though SO_PEERCRED is probably totally unportable, might as well
declare the getsockopt parameter as ACCEPT_TYPE_ARG3 to be consistent
with our other uses of getsockopt.
Tom Lane [Thu, 2 Aug 2001 14:27:40 +0000 (14:27 +0000)]
Remove SO_PASSCRED step in ident_unix --- according to Helge Bahmann,
that call is not needed to prepare for SO_PEERCRED. Also, simplify code
so that #ifdef SO_PEERCRED appears in only one place, to make it easier
to support other platforms with variants of this capability.
Tom Lane [Wed, 1 Aug 2001 23:52:50 +0000 (23:52 +0000)]
Digging through previous discussion of this patch, I note where Peter E.
points out how silly it is to use Autoconf to test for a preprocessor
symbol, when one can equally easily #ifdef on the symbol itself.
Accordingly, revert configure to prior state and do it that way.
Tom Lane [Wed, 1 Aug 2001 23:25:39 +0000 (23:25 +0000)]
Support ident authentication on local (Unix) socket connections, if the
system supports SO_PEERCRED requests for Unix sockets. This is an
amalgamation of patches submitted by Helge Bahmann and Oliver Elphick,
with some editorializing by yours truly.
Bruce Momjian [Wed, 1 Aug 2001 18:40:12 +0000 (18:40 +0000)]
The fti.pl supplied with the fulltextindex module generate ALL possible
substrings of two characters or greater, and is case-sensitive.
This patch makes it work correctly. It generates only the suffixes of each
word, plus lowercases them - as specified by the README file.
This brings it into line with the fti.c function, makes it case-insensitive
properly, removes the problem with duplicate rows being returned from an fti
search and greatly reduces the size of the generated index table.
Tatsuo Ishii [Wed, 1 Aug 2001 01:08:17 +0000 (01:08 +0000)]
When invoked with -i (initialize mode), split the copy command every
10000 tuples, rather than one big copy. This will prevent generating
too much WAL logs.
Tom Lane [Tue, 31 Jul 2001 22:55:45 +0000 (22:55 +0000)]
Cleanup code for preparsing pg_hba.conf and pg_ident.conf. Store line
number in the data structure so that we can give at least a minimally
useful idea of where the mistake is when we issue syntax error messages.
Move the ClientAuthentication() call to where it should have been in
the first place, so that postmaster memory releasing can happen in a
reasonable place also. Update obsolete comments, correct one real bug
(auth_argument was not picked up correctly).
Tom Lane [Tue, 31 Jul 2001 20:16:33 +0000 (20:16 +0000)]
Further thought shows that has_distinct_on_clause() needs to take much
more care with resjunk tlist entries than it was doing. The original
coding ignored resjunk entries entirely, but a resjunk entry that is
in either the distinctClause or sortClause lists indicates that DISTINCT
ON was used. It's important for ruleutils.c to get this right, else we
may dump views using DISTINCT ON incorrectly.
Tom Lane [Tue, 31 Jul 2001 17:56:31 +0000 (17:56 +0000)]
Fix optimizer to not try to push WHERE clauses down into a sub-SELECT that
has a DISTINCT ON clause, per bug report from Anthony Wood. While at it,
improve the DISTINCT-ON-clause recognizer routine to not be fooled by out-
of-order DISTINCT lists.
Tom Lane [Tue, 31 Jul 2001 01:16:09 +0000 (01:16 +0000)]
Change SQL commands embedded in the initdb script from the style
echo "command" | postgres
to the style
postgres <<EOF
command
EOF
This makes the script more legible (IMHO anyway) by reducing the need
to escape quotes, and allows us to execute successive SQL commands in
a single standalone-backend run, rather than needing to start a new
standalone backend for each command. With all the CREATE VIEWs that
are getting done now, this makes for a rather substantial reduction
in the runtime of initdb. (Some of us do initdb often enough to care
how long it runs ;-).)
Bruce Momjian [Mon, 30 Jul 2001 14:51:19 +0000 (14:51 +0000)]
This patch merges the identical methods from the JDBC1 and JDBC2
connection implementations (org.postgresql.jdbc[1|2].Connection) into
their superclass (org.postgresql.Connection).
It also changes the close() methods of Connection and PG_Stream, so that
PG_Stream no longer is responsible for sending the termination packet 'X'
to the backend. I figured that protocol-level stuff like that belonged in
Connection more than in PG_Stream.
Tom Lane [Sun, 29 Jul 2001 22:12:23 +0000 (22:12 +0000)]
Arrange for GRANT/REVOKE on a view to be dumped at the right time,
namely after the view definition rather than before it. Bug introduced
in 7.1 by changes to dump stuff in OID ordering.
Bruce Momjian [Sat, 21 Jul 2001 18:56:17 +0000 (18:56 +0000)]
Great, here is a context diff of CVS for implementing the get/setCatalog methods
in Connection - note: I've updated setCatalog(String catalog) from my previous
diff so it checks whether it is already connected to the specified catalog.
Bruce Momjian [Sat, 21 Jul 2001 04:32:42 +0000 (04:32 +0000)]
I downloaded new source for lib (only few hours old !!!), and made
changes on this new source to make non-blocking connection work. I
tested it, and PQSendQuery and PQGetResult are working fine.
In win32.h I added one line:
#define snprintf _snprintf
Bruce Momjian [Fri, 20 Jul 2001 17:45:06 +0000 (17:45 +0000)]
i've spotted a following problem using DBD::Pg under win32. winsock
functions do not set errno, so some normal conditions are treated as
fatal errors. e.g. fetching large tuples fails, as at some point recv()
returns EWOULDBLOCK. here's a patch, which replaces errno with
WSAGetLastError(). i've tried to to affect non-win32 code.
Tom Lane [Thu, 19 Jul 2001 02:12:35 +0000 (02:12 +0000)]
Arrange to recycle old XLOG log segment files as new segment files,
rather than deleting them only to have to create more. Steady state
is 2*CHECKPOINT_SEGMENTS + WAL_FILES + 1 segment files, which will
simply be renamed rather than constantly deleted and recreated.
To make this safe, added current XLOG file/offset number to page
header of XLOG pages, so that an un-overwritten page from an old
incarnation of a logfile can be reliably told from a valid page.
This change means that if you try to restart postmaster in a CVS-tip
database after installing the change, you'll get a complaint about
bad XLOG page magic number. If you don't want to initdb, run
contrib/pg_resetxlog (and be sure you shut down the old postmaster
cleanly).
Tom Lane [Tue, 17 Jul 2001 00:30:35 +0000 (00:30 +0000)]
Avoid assuming that pg_index table entries have unique OIDs, or even
that they have OIDs at all (the primary key for this table is indexrelid,
not OID). Simplify overly complex query to get name of primary key.
Tom Lane [Mon, 16 Jul 2001 22:43:34 +0000 (22:43 +0000)]
Improve documentation about reasoning behind the order of operations
in GetSnapshotData, GetNewTransactionId, CommitTransaction, AbortTransaction,
etc. Correct race condition in transaction status testing in
HeapTupleSatisfiesVacuum --- this wasn't important for old VACUUM with
exclusive lock on its table, but it sure is important now. All per
pghackers discussion 7/11/01 and 7/12/01.
Tom Lane [Mon, 16 Jul 2001 17:57:02 +0000 (17:57 +0000)]
Do not push down quals into subqueries that have LIMIT/OFFSET clauses,
since the added qual could change the set of rows that get past the
LIMIT. Per discussion on pgsql-sql 7/15/01.
Tom Lane [Mon, 16 Jul 2001 05:07:00 +0000 (05:07 +0000)]
Partial indexes work again, courtesy of Martijn van Oosterhout.
Note: I didn't force an initdb, figuring that one today was enough.
However, there is a new function in pg_proc.h, and pg_dump won't be
able to dump partial indexes until you add that function.
Tom Lane [Sun, 15 Jul 2001 22:48:19 +0000 (22:48 +0000)]
Restructure index AM interface for index building and index tuple deletion,
per previous discussion on pghackers. Most of the duplicate code in
different AMs' ambuild routines has been moved out to a common routine
in index.c; this means that all index types now do the right things about
inserting recently-dead tuples, etc. (I also removed support for EXTEND
INDEX in the ambuild routines, since that's about to go away anyway, and
it cluttered the code a lot.) The retail indextuple deletion routines have
been replaced by a "bulk delete" routine in which the indexscan is inside
the access method. I haven't pushed this change as far as it should go yet,
but it should allow considerable simplification of the internal bookkeeping
for deletions. Also, add flag columns to pg_am to eliminate various
hardcoded tests on AM OIDs, and remove unused pg_am columns.
Fix rtree and gist index types to not attempt to store NULLs; before this,
gist usually crashed, while rtree managed not to crash but computed wacko
bounding boxes for NULL entries (which might have had something to do with
the performance problems we've heard about occasionally).
Add AtEOXact routines to hash, rtree, and gist, all of which have static
state that needs to be reset after an error. We discovered this need long
ago for btree, but missed the other guys.
Oh, one more thing: concurrent VACUUM is now the default.
Peter Eisentraut [Sun, 15 Jul 2001 11:43:55 +0000 (11:43 +0000)]
Change xgettext rule to run in the source tree, so we don't have the
absolute paths of the source tree in the po files. Also, run msgfmt with
-c option in maintainer-check.
Bruce Momjian [Sun, 15 Jul 2001 04:21:26 +0000 (04:21 +0000)]
The attached patch fixes problems with the JDBC driver handling long
null terminated strings. The FE/BE protocol sends in some cases null
terminated strings to the client. The docs for the FE/BE protocol state
that there is no limit on the size of a null terminated string sent to
the client and a client should be coded using an expanding buffer to
deal with large strings. The old code did not do this and gave an error
if a null terminated string was greater than either 4 or 8K. It appears
that with the advent of TOAST very long SQL statements are becoming more
common, and apparently some error messages from the backend include the
SQL statement thus easily exceeding the 8K limit in the old code.
In fixing I also cleaned up some calls in the JDBC fastpath code that
were not doing character set conversion under multibyte, and removed
some methods that were no longer needed. I also removed a potential
threading problem with a shared variable that was being used in
Connection.java.
Thanks to Steve Wampler for discovering the problem and sending the
initial diffs that were the basis of this patch.
Tom Lane [Fri, 13 Jul 2001 22:55:59 +0000 (22:55 +0000)]
Initial implementation of concurrent VACUUM. Ifdef'd out for the moment,
because index locking issues are not handled correctly yet. Need to go
work on the index AMs next.