Tom Lane [Sat, 4 Mar 2006 19:30:12 +0000 (19:30 +0000)]
Incorporate a couple of recent tuplesort.c improvements into tuplestore.c.
In particular, ensure that enlargement of the memtuples[] array doesn't
fall foul of MaxAllocSize when work_mem is very large, and don't bother
enlarging it if that would force an immediate switch into 'tape' mode anyway.
Tom Lane [Sat, 4 Mar 2006 19:05:06 +0000 (19:05 +0000)]
Prevent sorting from requesting a SortTuple array that exceeds MaxAllocSize;
we'll go over to disk-based sort if we reach that limit.
This fixes Stefan Kaltenbrunner's observation that sorting can suffer an
'invalid memory alloc request size' failure when sort_mem is set large
enough. It's unfortunately not so easy to fix in 8.1 ...
Tatsuo Ishii [Sat, 4 Mar 2006 10:57:35 +0000 (10:57 +0000)]
Tighten up SJIS byte sequence check. Now we reject invalid SJIS byte
sequence such as "0x95 0x27". Patches from Akio Ishida.
Also update copyright notice.
Bruce Momjian [Sat, 4 Mar 2006 04:44:07 +0000 (04:44 +0000)]
> gettimeofday.c:35: warning: integer constant is too large for "long"
> type
Wouldn't it be better to use the UINT64CONST macro? I realize this
file is Windows-only, but we do worry about more than one compiler
on that platform.
Bruce Momjian [Sat, 4 Mar 2006 03:47:29 +0000 (03:47 +0000)]
That was a typo in my comment before the code (the nutshell
descriptions after the code are correct). Only shmmax needs to be
multiples of the page size (at least, that's how I interpret the
Darwin code).
Tom Lane [Fri, 3 Mar 2006 23:38:30 +0000 (23:38 +0000)]
Improve pg_dump and psql to use libpq's newer COPY support routines,
instead of the old deprecated ones.
Volkan Yazici, with some editorializing by moi.
Tom Lane [Fri, 3 Mar 2006 20:57:32 +0000 (20:57 +0000)]
Teach PQcmdTuples() that a COPY command tag might contain a row count,
and tighten up its sanity checking of the tag as a safety measure.
Volkan Yazici.
Tom Lane [Fri, 3 Mar 2006 19:54:10 +0000 (19:54 +0000)]
Make the COPY command return a command tag that includes the number of
rows copied. Backend side of Volkan Yazici's recent patch, with
corrections and documentation.
Tom Lane [Fri, 3 Mar 2006 18:25:14 +0000 (18:25 +0000)]
Dept. of second thoughts: rejigger the TRUNCATE ... CASCADE patch so that
relations are still checked for permissions etc as soon as they are
opened. The original form of the patch could hold exclusive lock for a
long time on relations that the user doesn't even have permissions to
access, let alone truncate.
Tom Lane [Fri, 3 Mar 2006 00:02:02 +0000 (00:02 +0000)]
Arrange to call AbsorbFsyncRequests every so often while performing a
checkpoint in the bgwriter. This forestalls overflow of the fsync request
queue, which is not fatal but causes considerable performance degradation
when it occurs (because backends then have to do their own fsyncs). Per
patch from Itagaki Takahiro, modified a little bit by me.
Tom Lane [Thu, 2 Mar 2006 21:56:14 +0000 (21:56 +0000)]
Remove unnecessary lo_lseek call in lo_open. Apparently there was once
a need for it back in the neolithic era, but it's certainly dead code in
any PG release we would recognize as such. Since it forces an additional
network round trip to the backend, getting rid of it should provide some
small performance improvement for large-object-using clients.
Tom Lane [Thu, 2 Mar 2006 21:49:09 +0000 (21:49 +0000)]
Fix ancient error in large objects usage example: overwrite() subroutine
was opening with INV_READ flag and then writing. Prior to 8.1 the backend
did not reject this, but now it does.
Teodor Sigaev [Thu, 2 Mar 2006 19:07:19 +0000 (19:07 +0000)]
Significantly improve ranking:
1) rank_cd now use weight of lexemes
2) rank_cd and rank can use any combination of normalization methods:
no normalization
normalization by log(length of document)
-----/------- by length of document
-----/------- by number of unique word in document
-----/------- by log(number of unique word in document)
-----/------- by number of covers (only rank_cd)
Bruce Momjian [Thu, 2 Mar 2006 18:18:00 +0000 (18:18 +0000)]
Add:
> * Improve port/qsort() to handle sorts with 50% unique and 50% duplicate
> value [qsort]
>
> This involves choosing better pivot points for the quicksort.
Tom Lane [Thu, 2 Mar 2006 05:34:12 +0000 (05:34 +0000)]
Fix possible crash at transaction end when a plpgsql function is used and
then modified within the same transaction. The code was using a linked list
of active PLpgSQL_expr structs, which was OK when it was written because
plpgsql never released any parse data structures for the life of the backend.
But since Neil fixed plpgsql's memory management, elements of the linked list
could be freed, leading to crash when the list is chased. Per report and test
case from Kris Jurka.
Tom Lane [Thu, 2 Mar 2006 01:18:26 +0000 (01:18 +0000)]
Fix up pg_dump to emit shell-type definitions at the proper time, to
make use of the recently added ability to create a shell type explicitly.
I also put in place some infrastructure to allow dump/no dump decisions
to be made separately for each database object, rather than the former
hardwired 'dump if in a dumpable schema' policy. This was needed anyway
for shell types so now seemed a convenient time to do it. The flexibility
isn't exposed to the user yet, but is ready for future extensions.
Neil Conway [Wed, 1 Mar 2006 23:00:56 +0000 (23:00 +0000)]
Woops: also update the alternative "expected" files for contrib/cube's
regression tests to account for the new error message wording. It seems
today is not my day...
Neil Conway [Wed, 1 Mar 2006 21:09:32 +0000 (21:09 +0000)]
Update the expected regression test results to account for the changes to
error messages I made yesterday -- thanks to Andrew Dunstan for reporting
this, and my apologies for missing it the first time.
Neil Conway [Wed, 1 Mar 2006 06:51:01 +0000 (06:51 +0000)]
Attached is a patch that replaces a bunch of places where StringInfos
are unnecessarily allocated on the heap rather than the stack. If the
StringInfo doesn't outlive the stack frame in which it is created,
there is no need to allocate it on the heap via makeStringInfo() --
stack allocation is faster. While it's not a big deal unless the
code is in a critical path, I don't see a reason not to save a few
cycles -- using stack allocation is not less readable.
I also cleaned up a bit of code along the way: moved variable
declarations into a more tightly-enclosing scope where possible,
fixed some pointless copying of strings in dblink, etc.
Neil Conway [Wed, 1 Mar 2006 06:30:32 +0000 (06:30 +0000)]
This patch makes the error message strings throughout the backend
more compliant with the error message style guide. In particular,
errdetail should begin with a capital letter and end with a period,
whereas errmsg should not. I also fixed a few related issues in
passing, such as fixing the repeated misspelling of "lexeme" in
contrib/tsearch2 (per Tom's suggestion).
Tom Lane [Tue, 28 Feb 2006 22:37:27 +0000 (22:37 +0000)]
Allow the syntax CREATE TYPE foo, with no parameters, to permit explicit
creation of a shell type. This allows a less hacky way of dealing with
the mutual dependency between a datatype and its I/O functions: make a
shell type, then make the functions, then define the datatype fully.
We should fix pg_dump to handle things this way, but this commit just deals
with the backend.
Martijn van Oosterhout, with some corrections by Tom Lane.
Neil Conway [Tue, 28 Feb 2006 20:03:52 +0000 (20:03 +0000)]
Allow PL/Python functions to return void, per gripe from James Robinson
(I didn't use his patch, however). A void-returning PL/Python function
must return None (from Python), which is translated into a void datum
(and *not* NULL) for Postgres. I also added some regression tests for
this functionality.
Tom Lane [Tue, 28 Feb 2006 04:10:28 +0000 (04:10 +0000)]
Extend the ExecInitNode API so that plan nodes receive a set of flag
bits indicating which optional capabilities can actually be exercised
at runtime. This will allow Sort and Material nodes, and perhaps later
other nodes, to avoid unnecessary overhead in common cases.
This commit just adds the infrastructure and arranges to pass the correct
flag values down to plan nodes; none of the actual optimizations are here
yet. I'm committing this separately in case anyone wants to measure the
added overhead. (It should be negligible.)
Peter Eisentraut [Mon, 27 Feb 2006 16:09:50 +0000 (16:09 +0000)]
Clean up CREATE FUNCTION syntax usage in contrib and elsewhere, in
particular get rid of single quotes around language names and old WITH ()
construct.
Tom Lane [Sun, 26 Feb 2006 22:58:12 +0000 (22:58 +0000)]
Improve sorting speed by pre-extracting the first sort-key column of
each tuple, as per my proposal of several days ago. Also, clean up
sort memory management by keeping all working data in a separate memory
context, and refine the handling of low-memory conditions.
Neil Conway [Sun, 26 Feb 2006 18:36:23 +0000 (18:36 +0000)]
Implement the <> operator for the tid type. Original patch from Mark
Kirkwood, minor improvements by Neil Conway. The regression tests have
been updated and the catversion has been bumped.
Peter Eisentraut [Fri, 24 Feb 2006 13:25:44 +0000 (13:25 +0000)]
The Makefile was invoking perl scripts as ./script.pl. This fails when
the script is not executable as UCS_to_most.pl is in CVS. It also won't
pick up any custom setting of the perl version/location to use. This
patch calls perl scripts like $(PERL) $(srcdir)/script.pl.
Andrew Dunstan [Fri, 24 Feb 2006 02:02:41 +0000 (02:02 +0000)]
Make restricted_exec feature for Windows more robust by using the environment
to pass the flag instead of the command line - some implementations of
getopt fail if getopt arguments are present after non-getopt arguments.
Neil Conway [Tue, 21 Feb 2006 23:01:54 +0000 (23:01 +0000)]
Cleanup the usage of ScanDirection: use the symbolic names for the
possible ScanDirection alternatives rather than magic numbers
(-1, 0, 1). Also, use the ScanDirection macros in a few places
rather than directly checking whether `dir == ForwardScanDirection'
and the like. Per patch from James William Pye. His patch also
changed ScanDirection to be a "char" rather than an enum, which
I haven't applied.
Tom Lane [Tue, 21 Feb 2006 18:01:32 +0000 (18:01 +0000)]
Fix old pg_dump oversight: default values for domains really need to be dumped
by decompiling the typdefaultbin expression, not just printing the typdefault
text which may be out-of-date or assume the wrong schema search path. (It's
the same hazard as for adbin vs adsrc in column defaults.) The catalogs.sgml
spec for pg_type implies that the correct procedure is to look to
typdefaultbin first and consider typdefault only if typdefaultbin is NULL.
I made dumping of both domains and base types do that, even though in the
current backend code typdefaultbin is always correct for domains and
typdefault for base types --- might as well try to future-proof it a little.
Per bug report from Alexander Galler.
Neil Conway [Mon, 20 Feb 2006 20:10:37 +0000 (20:10 +0000)]
Fix three Python reference leaks in PLy_traceback(). This would result
in leaking memory when invoking a PL/Python procedure that raises an
exception. Unfortunately this still leaks memory, but at least the
largest leak has been plugged.
This patch also fixes a reference counting mistake in PLy_modify_tuple()
for 8.0, 8.1 and HEAD: we don't actually own a reference to `platt', so
we shouldn't Py_DECREF() it.
Tom Lane [Sun, 19 Feb 2006 05:58:36 +0000 (05:58 +0000)]
Modify logtape.c so that the initial LogicalTapeSetCreate call only
allocates the control data. The per-tape buffers are allocated only
on first use. This saves memory in situations where tuplesort.c
overestimates the number of tapes needed (ie, there are fewer runs
than tapes). Also, this makes legitimate the coding in inittapes()
that includes tape buffer space in the maximum-memory calculation:
when inittapes runs, we've already expended the whole allowed memory
on tuple storage, and so we'd better not allocate all the tape buffers
until we've flushed some tuples out of memory.
Tom Lane [Sun, 19 Feb 2006 05:54:06 +0000 (05:54 +0000)]
Improve tuplesort.c to support variable merge order. The original coding
with fixed merge order (fixed number of "tapes") was based on obsolete
assumptions, namely that tape drives are expensive. Since our "tapes"
are really just a couple of buffers, we can have a lot of them given
adequate workspace. This allows reduction of the number of merge passes
with consequent savings of I/O during large sorts.
Neil Conway [Sun, 19 Feb 2006 00:04:28 +0000 (00:04 +0000)]
Add TABLESPACE and ON COMMIT clauses to CREATE TABLE AS. ON COMMIT is
required by the SQL standard, and TABLESPACE is useful functionality.
Patch from Kris Jurka, minor editorialization by Neil Conway.
Neil Conway [Sat, 18 Feb 2006 20:48:51 +0000 (20:48 +0000)]
Patch from Marko Kreen:
pgcrypto crypt()/md5 and hmac() leak memory when compiled against
OpenSSL as openssl.c digest ->reset will do two DigestInit calls
against a context. This happened to work with OpenSSL 0.9.6
but not with 0.9.7+.
Reason for the messy code was that I tried to avoid creating
wrapper structure to transport algorithm info and tried to use
OpenSSL context for it. The fix is to create wrapper structure.
It also uses newer digest API to avoid memory allocations
on reset with newer OpenSSLs.
Peter Eisentraut [Sat, 18 Feb 2006 16:15:23 +0000 (16:15 +0000)]
Add support for Windows codepages 1253, 1254, 1255, and 1257 and clean
up a bunch of the support utilities.
In src/backend/utils/mb/Unicode remove nearly duplicate copies of the
UCS_to_XXX perl script and replace with one version to handle all generic
files. Update the Makefile so that it knows about all the map files.
This produces a slight difference in some of the map files, using a
uniform naming convention and not mapping the null character.
In src/backend/utils/mb/conversion_procs create a master utf8<->win
codepage function like the ISO 8859 versions instead of having a separate
handler for each conversion.
There is an externally visible change in the name of the win1258 to utf8
conversion. According to the documentation notes, it was named
incorrectly and this changes it to a standard name.
Running the Unicode mapping perl scripts has shown some additional mapping
changes in koi8r and iso8859-7.