Robert Haas [Fri, 20 Aug 2010 01:07:50 +0000 (01:07 +0000)]
Remove the isLocalBuf argument from ReadBuffer_common.
Since an SMgrRelation now knows whether or not the underlying relation is
temporary, there's no point in also passing that information via an
additional argument.
Tom Lane [Thu, 19 Aug 2010 22:55:01 +0000 (22:55 +0000)]
Bring some sanity to the trace_recovery_messages code and docs.
Per gripe from Fujii Masao, though this is not exactly his proposed patch.
Categorize as DEVELOPER_OPTIONS and set context PGC_SIGHUP, as per Fujii,
but set the default to LOG because higher values aren't really sensible
(see the code for trace_recovery()). Fix the documentation to agree with
the code and to try to explain what the variable actually does. Get rid
of no-op calls trace_recovery(LOG), which accomplish nothing except to
demonstrate that this option confuses even its author.
Tom Lane [Thu, 19 Aug 2010 18:57:57 +0000 (18:57 +0000)]
Allow USING and INTO clauses of plpgsql's EXECUTE to appear in either order.
Aside from being more forgiving, this prevents a rather surprising misbehavior
when the "wrong" order was used: the old code didn't throw a syntax error,
but absorbed the INTO clause into the last USING expression, which then did
strange things downstream.
Intentionally not changing the documentation; we'll continue to advertise
only the "standard" clause order.
Backpatch to 8.4, where the USING clause was added to EXECUTE.
Tom Lane [Thu, 19 Aug 2010 18:10:48 +0000 (18:10 +0000)]
Keep exec_simple_check_plan() from thinking "SELECT foo INTO bar" is simple.
It's not clear if this situation can occur in plpgsql other than via the
EXECUTE USING case Heikki illustrated, which I will shortly close off.
However, ignoring the intoClause if it's there is surely wrong, so let's
patch it for safety.
Backpatch to 8.3, which is as far back as this code has a PlannedStmt
to deal with. There might be another way to make an equivalent test
before that, but since this is just preventing hypothetical bugs,
I'm not going to obsess about it.
Revert patch to coerce 'unknown' type parameters in the backend. As Tom
pointed out, it would need a 2nd pass after the whole query is processed to
correctly check that an unknown Param is coerced to the same target type
everywhere. Adding the 2nd pass would add a lot more code, which doesn't
seem worth the risk given that there isn't much of a use case for passing
unknown Params in the first place. The code would work without that check,
but it might be confusing and the behavior would be different from the
varparams case.
Instead, just coerce all unknown params in a PL/pgSQL USING clause to text.
That's simple, and is usually what users expect.
Revert the patch in CVS HEAD and master, and backpatch the new solution to
8.4. Unlike the previous solution, this applies easily to 8.4 too.
Tom Lane [Thu, 19 Aug 2010 16:16:20 +0000 (16:16 +0000)]
Allocate local buffers in a context of their own, rather than dumping them
into TopMemoryContext. This makes no functional difference, but makes it
easier to see what the space is being used for in MemoryContextStats dumps.
Per a recent example in which I was surprised by the size of TopMemoryContext.
Tom Lane [Thu, 19 Aug 2010 15:46:18 +0000 (15:46 +0000)]
Fix possible corruption of AfterTriggerEventLists in subtransaction rollback.
afterTriggerInvokeEvents failed to adjust events->tailfree when truncating
the last chunk of an event list. This could result in the data being
"de-truncated" by afterTriggerRestoreEventList during a subsequent
subtransaction abort. Even that wouldn't kill us, because the re-added data
would just be events marked DONE --- unless the data had been partially
overwritten by new events. Then we might crash, or in any case misbehave
(perhaps fire triggers twice, or fire triggers with the wrong event data).
Per bug #5622 from Thue Janus Kristensen.
Back-patch to 8.4 where the current trigger list representation was introduced.
Robert Haas [Thu, 19 Aug 2010 02:58:37 +0000 (02:58 +0000)]
Tidy up a few calls to smrgextend().
In the new API introduced by my patch to include the backend ID in
temprel filenames, the last argument to smrgextend() became skipFsync
rather than isTemp, but these calls didn't get the memo. It's not
really a problem to pass rel->rd_istemp rather than just plain false,
because smgrextend() now automatically skips the fsync for temprels
anyway, but this seems cleaner and saves some minute number of cycles.
Tom Lane [Wed, 18 Aug 2010 21:52:24 +0000 (21:52 +0000)]
Reset the per-output-tuple exprcontext each time through the main loop in
ExecModifyTable(). This avoids memory leakage when trigger functions leave
junk behind in that context (as they more or less must). Problem and solution
identified by Dean Rasheed.
I'm a bit concerned about the longevity of this solution --- once a plan can
have multiple ModifyTable nodes, we are very possibly going to have to do
something different. But it should hold up for 9.0.
Tom Lane [Wed, 18 Aug 2010 19:54:01 +0000 (19:54 +0000)]
Rename utf2ucs() to utf8_to_unicode(), and export it so it can be used
elsewhere.
Similarly rename the version in mbprint.c, not because this affects anything
but just to keep the two copies in exact sync. There was some discussion of
having only one copy in src/port/ instead, but this function is so small
and unlikely to change that that seems like overkill.
Slightly editorialized version of a patch by Joseph Adams. (The bug-fix
aspect of his patch was applied separately, and back-patched.)
Tom Lane [Wed, 18 Aug 2010 18:35:21 +0000 (18:35 +0000)]
Fix failure of "ALTER TABLE t ADD COLUMN c serial" when done by non-owner.
The implicitly created sequence was created as owned by the current user,
who could be different from the table owner, eg if current user is a
superuser or some member of the table's owning role. This caused sanity
checks in the SEQUENCE OWNED BY code to spit up. Although possibly we
don't need those sanity checks, the safest fix seems to be to make sure
the implicit sequence is assigned the same owner role as the table has.
(We still do all permissions checks as the current user, however.)
Per report from Josh Berkus.
Back-patch to 9.0. The bug goes back to the invention of SEQUENCE OWNED BY
in 8.2, but the fix requires an API change for DefineRelation(), which seems
to have potential for breaking third-party code if done in a minor release.
Given the lack of prior complaints, it's probably not worth fixing in the
stable branches.
Tom Lane [Wed, 18 Aug 2010 15:21:54 +0000 (15:21 +0000)]
Add missing handling of PlannedStmt.transientPlan in copyfuncs/outfuncs.
_outPlannedStmt is only debug support, so the omission there was not very
serious, but the omission in _copyPlannedStmt is a real bug. The consequence
would be that a copied plan tree would never be marked as a transient plan,
so that we would forget we ought to replan it after some not-yet-ready index
becomes ready for use. This might explain some past complaints about indexes
created with CREATE INDEX CONCURRENTLY not being used right away. Problem
spotted by Yeb Havinga.
Coerce 'unknown' type parameters to the right type in the fixed-params
parse_analyze() function. That case occurs e.g with PL/pgSQL
EXECUTE ... USING 'stringconstant'.
The coercion with a CoerceViaIO node. The result is similar to the coercion
via input function performed for unknown constants in coerce_type(),
except that this happens at runtime.
Backpatch to 9.0. The issue is present in 8.4 as well, but the coerce param
hook infrastructure this patch relies on was introduced in 9.0. Given the
lack of user reports and harmlessness of the bug, it's not worth attempting
a different fix just for 8.4.
Tom Lane [Mon, 16 Aug 2010 17:32:46 +0000 (17:32 +0000)]
Arrange to fsync the contents of lockfiles (both postmaster.pid and the
socket lockfile) when writing them. The lack of an fsync here may well
explain two different reports we've seen of corrupted lockfile contents,
which doesn't particularly bother the running server but can prevent a
new server from starting if the old one crashes. Per suggestion from
Alvaro.
Robert Haas [Mon, 16 Aug 2010 02:02:28 +0000 (02:02 +0000)]
Make LockDatabaseObject() AcceptInvalidationMessages().
This is appropriate for the same reasons we already do it in
LockSharedObject(): things might have changed while we were waiting
for the lock. There doesn't seem to be a live bug here at the moment,
but that's mostly because it isn't currently used for very much.
Tom Lane [Mon, 16 Aug 2010 00:06:18 +0000 (00:06 +0000)]
Fix psql's copy of utf2ucs() to match the backend's copy exactly;
in particular, propagate a fix in the test to see whether a UTF8 character has
length 4 bytes. This is likely of little real-world consequence because
5-or-more-byte UTF8 sequences are not supported by Postgres nor seen anywhere
in the wild, but still we may as well get it right. Problem found by Joseph
Adams.
Tom Lane [Sat, 14 Aug 2010 15:47:13 +0000 (15:47 +0000)]
Fix planner to make a reasonable assumption about the amount of memory space
used by array_agg(), string_agg(), and similar aggregate functions that use
"internal" as their transition datatype. The previous coding thought this
took *no* extra space, since "internal" is pass-by-value; but actually these
aggregates typically consume a great deal of space. Per bug #5608 from
Itagaki Takahiro, and fix suggestion from Hitoshi Harada.
Back-patch to 8.4, where array_agg was introduced.
Robert Haas [Fri, 13 Aug 2010 20:10:54 +0000 (20:10 +0000)]
Include the backend ID in the relpath of temporary relations.
This allows us to reliably remove all leftover temporary relation
files on cluster startup without reference to system catalogs or WAL;
therefore, we no longer include temporary relations in XLOG_XACT_COMMIT
and XLOG_XACT_ABORT WAL records.
Since these changes require including a backend ID in each
SharedInvalSmgrMsg, the size of the SharedInvalidationMessage.id
field has been reduced from two bytes to one, and the maximum number
of connections has been reduced from INT_MAX / 4 to 2^23-1. It would
be possible to remove these restrictions by increasing the size of
SharedInvalidationMessage by 4 bytes, but right now that doesn't seem
like a good trade-off.
Tom Lane [Fri, 13 Aug 2010 20:04:33 +0000 (20:04 +0000)]
Minor #include cleanup.
I just noticed that libpq's pqsignal.h was violating our general inclusion
style guidelines by explicitly including postgres_fe.h. Remove that, and
put it in pqsignal.c where it belongs.
Tom Lane [Fri, 13 Aug 2010 18:36:26 +0000 (18:36 +0000)]
Add xml_is_well_formed, xml_is_well_formed_document, xml_is_well_formed_content
functions to the core XML code. Per discussion, the former depends on
XMLOPTION while the others do not. These supersede a version previously
offered by contrib/xml2.
Tom Lane [Fri, 13 Aug 2010 16:27:11 +0000 (16:27 +0000)]
Fix Assert failure in PushOverrideSearchPath when trying to restore a search
path that specifies useTemp, but there is no active temp schema in the
current session. (This can happen if the path was saved during a transaction
that created a temp schema and was later rolled back.) For existing callers
it's sufficient to ignore the useTemp flag in this case, though we might
later want to offer an option to create a fresh temp schema. So far as I can
tell this is just an Assert failure: in a non-assert build, the code would
push a zero onto the new search path, which is useless but not very harmful.
Per bug report from Heikki.
Back-patch to 8.3; prior versions don't have this code.
Robert Haas [Fri, 13 Aug 2010 15:42:21 +0000 (15:42 +0000)]
Make RecordTransactionCommit() respect wal_level.
Since the only purpose of WAL-loggin SharedInvalidationMessages is to support
Hot Standby operation, they needn't be included when wal_level < hot_standby.
Tom Lane [Fri, 13 Aug 2010 14:38:04 +0000 (14:38 +0000)]
Fix pg_restore to complain if any arguments remain after parsing the switches
and input file name, per bug #5617 from Leo Shklovskii. Rearrange the
corresponding code in pg_dump and pg_dumpall so that all three programs
handle this in a consistent, straightforward fashion.
Back-patch to 9.0, but no further. Although this is certainly a bug, it's
possible that people have scripts that will be broken by the added error
check, so it seems better not to change the behavior in stable branches.
Tom Lane [Thu, 12 Aug 2010 21:10:59 +0000 (21:10 +0000)]
Get rid of a bunch of dubious error handling code in pgbench by just erroring
out immediately on any out-of-memory condition. It's rather pointless to
imagine that pgbench will be able to continue usefully after a malloc
failure, and in any case there were a number of unchecked mallocs.
Tom Lane [Thu, 12 Aug 2010 00:40:59 +0000 (00:40 +0000)]
Extend psql's \e and \ef commands so that a line number can be specified,
and the editor's cursor will be initially placed on that line. In \e the
lines are counted with respect to the query buffer, while in \ef they are
counted with line 1 = first line of function body. These choices are useful
for positioning the cursor on the line of a previously-reported error.
To avoid assumptions about what switch the user's editor takes for this
purpose, invent a new psql variable EDITOR_LINENUMBER_SWITCH with (at
present) no default value.
One incompatibility from previous behavior is that "\e 1234" will now
take "1234" as a line number not a file name. There are at least two
ways to select a numerically-named file if you really want to.
Pavel Stehule, reviewed by Jan Urbanski, with further editing by Robert Haas
and Tom Lane
The sanity check added to array_recv() wa a bit too tight; we must
continue to accept an empty array with dimension information. array_send()
can output such arrays.
Tom Lane [Tue, 10 Aug 2010 21:51:00 +0000 (21:51 +0000)]
Add three-parameter forms of array_to_string and string_to_array, to allow
better handling of NULL elements within the arrays. The third parameter
is a string that should be used to represent a NULL element, or should
be translated into a NULL element, respectively. If the third parameter
is NULL it behaves the same as the two-parameter form.
There are two incompatible changes in the behavior of the two-parameter form
of string_to_array. First, it will return an empty (zero-element) array
rather than NULL when the input string is of zero length. Second, if the
field separator is NULL, the function splits the string into individual
characters, rather than returning NULL as before. These two changes make
this form fully compatible with the behavior of the new three-parameter form.
Tom Lane [Mon, 9 Aug 2010 18:50:11 +0000 (18:50 +0000)]
Fix incorrect logic in plpgsql for cleanup after evaluation of non-simple
expressions. We need to deal with this when handling subscripts in an array
assignment, and also when catching an exception. In an Assert-enabled build
these omissions led to Assert failures, but I think in a normal build the
only consequence would be short-term memory leakage; which may explain why
this wasn't reported from the field long ago.
Back-patch to all supported versions. 7.4 doesn't have exceptions, but
otherwise these bugs go all the way back.
Tom Lane [Mon, 9 Aug 2010 02:25:07 +0000 (02:25 +0000)]
Modify the handling of RAISE without parameters so that the error it throws
can be caught in the same places that could catch an ordinary RAISE ERROR
in the same location. The previous coding insisted on throwing the error
from the block containing the active exception handler; which is arguably
more surprising, and definitely unlike Oracle's behavior.
Not back-patching, since this is a pretty obscure corner case. The risk
of breaking somebody's code in a minor version update seems to outweigh
any possible benefit.
Tom Lane [Sun, 8 Aug 2010 16:27:06 +0000 (16:27 +0000)]
Add stats functions and views to provide access to a transaction's own
statistics counts. These numbers are being accumulated but haven't yet been
transmitted to the collector (and won't be, until the transaction ends).
For some purposes, though, it's handy to be able to look at them.
Tom Lane [Sat, 7 Aug 2010 02:44:09 +0000 (02:44 +0000)]
Recognize functional dependency on primary keys. This allows a table's
other columns to be referenced without listing them in GROUP BY, so long as
the primary key column(s) are listed in GROUP BY.
Eventually we should also allow functional dependency on a UNIQUE constraint
when the columns are marked NOT NULL, but that has to wait until NOT NULL
constraints are represented in pg_constraint, because we need to have
pg_constraint OIDs for all the conditions needed to ensure functional
dependency.
Peter Eisentraut, reviewed by Alex Hunsaker and Tom Lane
Tom Lane [Thu, 5 Aug 2010 21:45:35 +0000 (21:45 +0000)]
Add a very specific hint for the case that we're unable to locate a function
matching a call like f(x, ORDER BY y,z). It could be that what the user
really wants is f(x,z ORDER BY y). We now have pretty conclusive evidence
that many people won't understand this problem without concrete guidance,
so give it to them. Per further discussion of the string_agg() problem.
Tom Lane [Thu, 5 Aug 2010 18:21:19 +0000 (18:21 +0000)]
Remove the single-argument form of string_agg(). It added nothing much in
functionality, while creating an ambiguity in usage with ORDER BY that at
least two people have already gotten seriously confused by. Also, add an
opr_sanity test to check that we don't in future violate the newly minted
policy of not having built-in aggregates with the same name and different
numbers of parameters. Per discussion of a complaint from Thom Brown.
Robert Haas [Thu, 5 Aug 2010 14:45:09 +0000 (14:45 +0000)]
Standardize get_whatever_oid functions for object types with
unqualified names.
- Add a missing_ok parameter to get_tablespace_oid.
- Avoid duplicating get_tablespace_od guts in objectNamesToOids.
- Add a missing_ok parameter to get_database_oid.
- Replace get_roleid and get_role_checked with get_role_oid.
- Add get_namespace_oid, get_language_oid, get_am_oid.
- Refactor existing code to use new interfaces.
Robert Haas [Wed, 4 Aug 2010 17:33:09 +0000 (17:33 +0000)]
Fix numeric_maximum_size() calculation.
The old computation can sometimes underestimate the necessary space
by 2 bytes; however we're not back-patching this, because this result
isn't used for anything critical. Per discussion with Tom Lane,
make the typmod test in this function match the ones in numeric()
and apply_typmod() exactly.
Tom Lane [Tue, 3 Aug 2010 21:21:03 +0000 (21:21 +0000)]
Replace the naive HYPOT() macro with a standards-conformant hypotenuse
function. This avoids unnecessary overflows and probably gives a more
accurate result as well.
Tom Lane [Tue, 3 Aug 2010 19:24:05 +0000 (19:24 +0000)]
Code review for --quote-all-identifiers patch: add missing --help documentation
for new pg_dump/pg_dumpall parameters, make a couple of trivial stylistic
adjustments to make the code follow usual project style.
Tom Lane [Tue, 3 Aug 2010 18:33:09 +0000 (18:33 +0000)]
Kibitzing on \conninfo patch: adjust the order of field output to match
the parameters of \connect, and fix oversight of not enabling translation
of the messages. Also, adjust \connect's similar messages to match, and
deal with 8.2-era violation of basic translatability guidelines there.
Tom Lane [Tue, 3 Aug 2010 16:31:02 +0000 (16:31 +0000)]
Add some comments to tinterval_cmp_internal pointing out its severe
implementation deficiencies. Per discussion of bug #5592, we're not
going to change it, but these things should be documented so that if
anyone ever reimplements type tinterval, they will be more careful.
Robert Haas [Tue, 3 Aug 2010 15:47:02 +0000 (15:47 +0000)]
Fix inheritance count tracking in ALTER TABLE .. ADD CONSTRAINT.
Without this patch, constraints inherited by children of a parent
table which itself has multiple inheritance parents can end up with
the wrong coninhcount. After dropping the constraint, the children
end up with a leftover copy of the constraint that is not dumped
and cannot be dropped. There is a similar problem with ALTER TABLE
.. ADD COLUMN, but that looks significantly more difficult to
resolve, so I'm committing this fix separately.
Back-patch to 8.4, which is the first release that has coninhcount.
Tom Lane [Tue, 3 Aug 2010 01:50:27 +0000 (01:50 +0000)]
Be a little more careful with the shift computations in QT2QTN and
makeTSQuerySign. The first of these is a live bug, on some platforms,
as per bug #5590 from John Regehr. However the consequences seem limited
because of the relatively narrow scope of use of QTNode.sign. The shift in
makeTSQuerySign is actually safe because TSQS_SIGLEN is unsigned, but it
seems like a good idea to insert an explicit cast rather than depend on that.
Tom Lane [Tue, 3 Aug 2010 00:10:39 +0000 (00:10 +0000)]
Fix core dump in QTNodeCompare when tsquery_cmp() is applied to two empty
tsqueries. CompareTSQ has to have a guard for the case rather than blindly
applying QTNodeCompare to random data past the end of the datums. Also,
change QTNodeCompare to be a little less trusting: use an actual test rather
than just Assert'ing that the input is sane. Problem encountered while
investigating another issue (I saw a core dump in autoanalyze on a table
containing multiple empty tsquery values).
Back-patch to all branches with tsquery support.
In HEAD, also fix some bizarre (though not outright wrong) coding in
tsq_mcontains().
Tom Lane [Mon, 2 Aug 2010 04:51:17 +0000 (04:51 +0000)]
Don't try to force use of -no-cpp-precomp on OS X. It's been five years
since Apple shipped a compiler that needed this switch, and there's
increasing interest in using other compilers that won't accept the switch
at all. Better to let anybody who still needs the switch inject it via
CPPFLAGS. Per gripe from Neil Conway.
Robert Haas [Mon, 2 Aug 2010 03:46:54 +0000 (03:46 +0000)]
Remove ancient PL/pgsql line numbering hack.
While this hack arguably has some benefit in terms of making PL/pgsql's
line numbering match the programmer's expectations, it also makes
PL/pgsql inconsistent with the remaining PLs, making it difficult for
clients to reliably determine where the error actually is. On balance,
it seems better to be consistent.
Tom Lane [Mon, 2 Aug 2010 02:29:39 +0000 (02:29 +0000)]
Tweak a couple of macros in the regex code to suppress compiler warnings
from "clang". The VERR changes make an assignment unconditional, which is
probably easier to read/understand anyway, and one can hardly argue that
it's worth shaving cycles off the case of reporting another error when
one has already been detected. The INSIST change limits where that macro
can be used, but not in a way that creates a problem for any existing call.
Tom Lane [Mon, 2 Aug 2010 01:24:54 +0000 (01:24 +0000)]
Fix an ancient typo that prevented the detection of conflicting fields when
interval input "invalid" was specified together with other fields. Spotted
by Neil Conway with the help of a clang warning. Although this has been
wrong since the interval code was written more than 10 years ago, it doesn't
affect anything beyond which error message you get for a wrong input, so not
worth back-patching very far.
Tom Lane [Sun, 1 Aug 2010 22:38:11 +0000 (22:38 +0000)]
Fix ANALYZE's ancient deficiency of not trying to collect stats for expression
indexes when the index column type (the opclass opckeytype) is different from
the expression's datatype. When coded, this limitation wasn't worth worrying
about because we had no intelligence to speak of in stats collection for the
datatypes used by such opclasses. However, now that there's non-toy
estimation capability for tsvector queries, it amounts to a bug that ANALYZE
fails to do this.
The fix changes struct VacAttrStats, and therefore constitutes an API break
for custom typanalyze functions. Therefore we can't back-patch it into
released branches, but it was agreed that 9.0 isn't yet frozen hard enough
to make such a change unacceptable. Ergo, back-patch to 9.0 but no further.
The API break had better be mentioned in 9.0 release notes.
Tom Lane [Sun, 1 Aug 2010 21:31:08 +0000 (21:31 +0000)]
Add some knowledge about prefix matches to tsmatchsel(). It's not terribly
bright, but it beats assuming that a prefix match behaves identically to an
exact match, which is what the code was doing before :-(. Noted while
experimenting with Artur Dobrowski's example.
Tom Lane [Sun, 1 Aug 2010 19:16:39 +0000 (19:16 +0000)]
Fix an additional set of problems in GIN's handling of lossy page pointers.
Although the key-combining code claimed to work correctly if its input
contained both lossy and exact pointers for a single page in a single TID
stream, in fact this did not work, and could not work without pretty
fundamental redesign. Modify keyGetItem so that it will not return such a
stream, by handling lossy-pointer cases a bit more explicitly than we did
before.
Per followup investigation of a gripe from Artur Dabrowski.
An example of a query that failed given his data set is
select count(*) from search_tab where
(to_tsvector('german', keywords ) @@ to_tsquery('german', 'ee:* | dd:*')) and
(to_tsvector('german', keywords ) @@ to_tsquery('german', 'aa:*'));
Back-patch to 8.4 where the lossy pointer code was introduced.