Tom Lane [Thu, 24 May 2018 16:38:55 +0000 (12:38 -0400)]
Fix objectaddress.c code for publication relations.
getObjectDescription and getObjectIdentity failed to schema-qualify
the name of the published table, which is bad in getObjectDescription and
unforgivable in getObjectIdentity. Actually, getObjectIdentity failed to
emit the table's name at all unless "objname" output is requested, which
accidentally works for some (all?) extant callers but is clearly not the
intended API. Somebody had also not gotten the memo that the output of
getObjectIdentity is not to be translated.
To fix getObjectDescription, I made it call getRelationDescription, which
required refactoring the translatable string for the case, but is more
future-proof in case we ever publish relations that aren't plain tables.
While at it, I made the English output look like "publication of table X
in publication Y"; the added "of" seems to me to make it read much better.
Back-patch to v10 where publications were introduced.
Tom Lane [Thu, 24 May 2018 16:07:41 +0000 (12:07 -0400)]
Properly schema-qualify additional object types in getObjectDescription().
Collations, conversions, extended statistics objects (in >= v10),
and all four types of text search objects have schema-qualified names.
getObjectDescription() ignored that and would emit just the base name of
the object, potentially producing wrong or at least highly misleading
output. Fix it to add the schema name whenever the object is not "visible"
in the current search path, as is the rule for other schema-qualifiable
object types.
Although in common situations the output won't change, this seems to me
(tgl) to be a bug worthy of back-patching, hence do so.
Andrew Dunstan [Thu, 24 May 2018 14:15:31 +0000 (23:45 +0930)]
Preserve information on use of git-external-diff
Now that the Working with git wiki page no longer suggests producing
context diffs, we should preserve the information on how to use
git-external-diff for those people who want to view context format
diffs. The most obvious place is in the script itself, so that's what's
done here.
Tom Lane [Wed, 23 May 2018 23:04:34 +0000 (19:04 -0400)]
Fix simple_prompt() to disable echo on Windows when stdin != terminal.
If echo = false, simple_prompt() is supposed to prevent echoing the
input (for password input). However, the Windows implementation applied
the mode change to STD_INPUT_HANDLE. That would not have the desired
effect if stdin isn't actually the terminal, for instance if the user
is piping something into psql. Fix it to apply the mode change to
the correct input file, so that passwords do not echo in such cases.
In passing, shorten and de-uglify this code by using #elif rather than
an #if nest and removing some duplicated code.
Back-patch to all supported versions. To simplify that, also back-patch
the portions of commit 9daec77e1 that got rid of an unnecessary
malloc/free in the same area.
Tom Lane [Wed, 23 May 2018 18:19:04 +0000 (14:19 -0400)]
Remove configure's check for nonstandard "long long" printf modifiers.
We used to claim to support platforms using 'q' or 'I64' as the printf
length modifier for long long int, by dint of replacing snprintf with
our own code which uses the C99 standard 'll' modifier. But that is
only adequate if we use INT64_MODIFIER only in snprintf-based calls,
not directly with the platform's native printf or fprintf. Which
hasn't been the case for years. We had not noticed, partially because
of inadequate test coverage, and partially because the buildfarm is
almost completely bare of machines that won't take 'll'. The last
one seems to have been frogmouth, which was adjusted recently so that
it will take 'll'. We might as well just give up on the pretense
that anything else works, and save ourselves some configure cycles.
Tom Lane [Wed, 23 May 2018 14:59:55 +0000 (10:59 -0400)]
Fix incorrect ordering of operations in pg_resetwal and pg_rewind.
Commit c37b3d08c dropped its added GetDataDirectoryCreatePerm call into
the wrong place in pg_resetwal.c, namely after the chdir to DataDir.
That broke invocations using a relative path, as reported by Tushar Ahuja.
We could have left it where it was and changed the argument to be ".",
but that'd result in a rather confusing error message in event of a
failure, so re-ordering seems like a better solution.
Similarly reorder operations in pg_rewind.c. The issue there is that
it doesn't seem like a good idea to do any actual operations before the
not-root check (on Unix) or the restricted token acquisition (on Windows).
I don't know that this is an actual bug, but I'm definitely not convinced
that it isn't, either.
Assorted other code review for c37b3d08c and da9b580d8: fix some
misspelled or otherwise badly worded comments, put the #include for
<sys/stat.h> where it actually belongs, etc.
Accept "B" in all memory-unit GUCs, and improve error messages.
Commit 6e7baa3227 added support for "B" unit, for specifying config options
in bytes. However, it was only accepted in GUC_UNIT_BYTE settings,
wal_segment_size and track_activity_query_size, and not e.g. in work_mem.
This patch makes it consistent, so that "B" accepted in all the same
contexts where "kB", "MB", and so forth are accepted.
Add "B" to the list of accepted units in the error hint, along with "kB",
"MB", etc.
Add an entry in the conversion table for "TB" to "B" conversion. A terabyte
is out of range for any GUC_UNIT_BYTE option, so you always get an "out of
range" error with that, but without it, you get a confusing error message
that claims that "TB" is not an accepted unit, with a hint that nevertheless
lists "TB" as an accepted unit.
Reviewed-by: Alexander Korotkov, Andres Freund
Discussion: https://www.postgresql.org/message-id/1bfe7f4a-7e22-aa6e-7b37-f4d222ed2d67@iki.fi
Tom Lane [Tue, 22 May 2018 17:32:52 +0000 (13:32 -0400)]
Widen COPY FROM's current-line-number counter from 32 to 64 bits.
Because the code for the HEADER option skips a line when this counter
is zero, a very long COPY FROM WITH HEADER operation would drop a line
every 2^32 lines. A lesser but still unfortunate problem is that errors
would show a wrong input line number for errors occurring beyond the
2^31'st input line. While such large input streams seemed impractical
when this code was first written, they're not any more. Widening the
counter (and some associated variables) to uint64 should be enough to
prevent problems for the foreseeable future.
The README lists all the files available in the directory, along with short
descriptions of each, but a few newly added ones were missing. While we're
at it, reorder the list into alphabetical order.
Andrew Gierth [Mon, 21 May 2018 16:02:17 +0000 (17:02 +0100)]
Fix SQL:2008 FETCH FIRST syntax to allow parameters.
OFFSET <x> ROWS FETCH FIRST <y> ROWS ONLY syntax is supposed to accept
<simple value specification>, which includes parameters as well as
literals. When this syntax was added all those years ago, it was done
inconsistently, with <x> and <y> being different subsets of the
standard syntax.
Rectify that by making <x> and <y> accept the same thing, and allowing
either a (signed) numeric literal or a c_expr there, which allows for
parameters, variables, and parenthesized arbitrary expressions.
Per bug #15200 from Lukas Eder.
Backpatch all the way, since this has been broken from the start.
Tom Lane [Mon, 21 May 2018 15:41:42 +0000 (11:41 -0400)]
Improve spelling of new FINALFUNC_MODIFY aggregate attribute.
I'd used SHARABLE as a value originally, but Peter Eisentraut points out
that dictionaries agree that SHAREABLE is the preferred spelling.
Run around and change that before it's too late.
Tom Lane [Mon, 21 May 2018 15:21:08 +0000 (11:21 -0400)]
Doc: fix bogus cross-reference link.
An xref to a <para>'s ID isn't very helpful because paras don't have
names. This causes a warning while building PDFs, though for some
reason not while building HTML. The link arguably went to the wrong
place, too.
Peter Eisentraut [Mon, 21 May 2018 14:01:49 +0000 (10:01 -0400)]
pg_basebackup: Remove short option -k
-k meant --no-verify-checksums, which is the opposite of what initdb
uses -k for. After discussion, a short option does not seem necessary,
so just keep the long option.
Tom Lane [Mon, 21 May 2018 04:32:28 +0000 (00:32 -0400)]
Fix unsafe usage of strerror(errno) within ereport().
This is the converse of the unsafe-usage-of-%m problem: the reason
ereport/elog provide that format code is mainly to dodge the hazard
of errno getting changed before control reaches functions within the
arguments of the macro. I only found one instance of this hazard,
but it's been there since 9.4 :-(.
Tom Lane [Sun, 20 May 2018 22:06:00 +0000 (18:06 -0400)]
Fix unportable usage of printf("%m").
While glibc's version of printf accepts %m, most others do not;
to be portable, we have to do it the hard way with strerror(errno).
pg_verify_checksums evidently did not get that memo.
Noted while fooling around with NetBSD-current, which generates
a compiler warning for this mistake.
Tom Lane [Sun, 20 May 2018 15:40:54 +0000 (11:40 -0400)]
printf("%lf") is not portable, so omit the "l".
The "l" (ell) width spec means something in the corresponding scanf usage,
but not here. While modern POSIX says that applying "l" to "f" and other
floating format specs is a no-op, SUSv2 says it's undefined. Buildfarm
experience says that some old compilers emit warnings about it, and at
least one old stdio implementation (mingw's "ANSI" option) actually
produces wrong answers and/or crashes.
Tom Lane [Sat, 19 May 2018 20:04:47 +0000 (16:04 -0400)]
Assorted minor cleanups for bootstrap-data Perl scripts.
FindDefinedSymbol was intended to take an array of possible include
paths, but it never actually worked correctly for any but the first
array element. Since there's no use-case for more than one path
anyway, let's just simplify this code and its callers by redefining
it as taking only one include path.
Minor other code-beautification without functional effects, except
that in one place we format the output as pgindent would do.
Tom Lane [Sat, 19 May 2018 18:22:18 +0000 (14:22 -0400)]
Support platforms where strtoll/strtoull are spelled __strtoll/__strtoull.
Ancient HPUX, for one, does this. We hadn't noticed due to the lack
of regression tests that required a working strtoll.
(I was slightly tempted to remove the other historical spelling,
strto[u]q, since it seems we have no buildfarm members testing that case.
But I refrained.)
Tom Lane [Sat, 19 May 2018 02:42:10 +0000 (22:42 -0400)]
Arrange to supply declarations for strtoll/strtoull if needed.
Buildfarm member dromedary is still unhappy about the recently-added
ecpg "long long" tests. The reason turns out to be that it includes
"-ansi" in its CFLAGS, and in their infinite wisdom Apple have decided
to hide the declarations of strtoll/strtoull in C89-compliant builds.
(I find it pretty curious that they hide those function declarations
when you can nonetheless declare a "long long" variable, but anyway
that is their behavior, both on dromedary's obsolete macOS version and
the newest and shiniest.) As a result, gcc assumes these functions
return "int", leading naturally to wrong results.
(Looking at dromedary's past build results, it's evident that this
problem also breaks pg_strtouint64() on 32-bit platforms; but we
evidently have no regression tests that exercise that function with
values above 32 bits.)
To fix, supply declarations for these functions when the platform
provides the functions but not the declarations, using the same type
of mechanism as we use for some other similar cases.
Stephen Frost [Sat, 19 May 2018 01:20:27 +0000 (21:20 -0400)]
Fix for globals.c- c.h must come first
Commit da9b580 mistakenly put a system header before postgres.h (which
includes c.h). That can cause portability issues and broke (at least)
builds with older Windows compilers.
Tom Lane [Fri, 18 May 2018 23:03:32 +0000 (19:03 -0400)]
Hot-fix ecpg regression test for missing ecpg_config.h inclusion.
I don't think this is really the best long-term answer, and in
particular it doesn't fix the pre-existing hazard in sqltypes.h.
But for the moment let's just try to make the buildfarm green again.
Tom Lane [Fri, 18 May 2018 17:04:59 +0000 (13:04 -0400)]
Add some test coverage for ecpg's "long long" support.
This will only actually exercise the "long long" code paths on platforms
where "long" is 32 bits --- otherwise, the SQL bigint type maps to
plain "long", and we will test that code path instead. But that's
probably sufficient coverage, and anyway we weren't testing either
code path before.
Tom Lane [Fri, 18 May 2018 16:52:28 +0000 (12:52 -0400)]
Recognize that MSVC can support strtoll() and strtoull().
This is needed for full support of "long long" variables in ecpg, but
the previous patch for bug #15080 (commits 51057feaa et al) missed it.
In MSVC versions where the functions don't exist under those names,
we can nonetheless use _strtoi64() and _strtoui64().
Tom Lane [Fri, 18 May 2018 16:10:19 +0000 (12:10 -0400)]
Small improvement for plpgsql regression test.
Use DISCARD PLANS instead of a reconnect to force reconstruction of
a cached plan; this corresponds more nearly to what people might
actually do in practice.
Tom Lane [Fri, 18 May 2018 15:53:18 +0000 (11:53 -0400)]
MSVC builds must use a separate stamp file for copying generated headers.
Commit bad51a49a tried to use a shortcut with just one stamp file
recording the actions of generating the pg_*_d.h headers and copying
them to the src/include/catalog/ directory. That doesn't work in all
scenarios though, so we must use two stamp files like the Makefiles do.
Tom Lane [Thu, 17 May 2018 15:10:50 +0000 (11:10 -0400)]
Make numeric power() handle NaNs according to the modern POSIX spec.
In commit 6bdf1303b, we ensured that power()/^ for float8 would honor
the NaN behaviors specified by POSIX standards released in this century,
ie NaN ^ 0 = 1 and 1 ^ NaN = 1. However, numeric_power() was not
touched and continued to follow the once-common behavior that every
case involving NaN input produces NaN. For consistency, let's switch
the numeric behavior to the modern spec in the same release that ensures
that behavior for float8.
(Note that while 6bdf1303b was initially back-patched, we later undid
that, concluding that any behavioral change should appear only in v11.)
Tom Lane [Wed, 16 May 2018 18:56:52 +0000 (14:56 -0400)]
Detoast plpgsql variables if they might live across a transaction boundary.
Up to now, it's been safe for plpgsql to store TOAST pointers in its
variables because the ActiveSnapshot for whatever query called the plpgsql
function will surely protect such TOAST values from being vacuumed away,
even if the owning table rows are committed dead. With the introduction of
procedures, that assumption is no longer good in "non atomic" executions
of plpgsql code. We adopt the slightly brute-force solution of detoasting
all TOAST pointers at the time they are stored into variables, if we're in
a non-atomic context, just in case the owning row goes away.
Some care is needed to avoid long-term memory leaks, since plpgsql tends
to run with CurrentMemoryContext pointing to its call-lifespan context,
but we shouldn't assume that no memory is leaked by heap_tuple_fetch_attr.
In plpgsql proper, we can do the detoasting work in the "eval_mcontext".
Most of the code thrashing here is due to the need to add this capability
to expandedrecord.c as well as plpgsql proper. In expandedrecord.c,
we can't assume that the caller's context is short-lived, so make use of
the short-term sub-context that was already invented for checking domain
constraints. In view of this repurposing, it seems good to rename that
variable and associated code from "domain_check_cxt" to "short_term_cxt".
Tom Lane [Wed, 16 May 2018 17:46:09 +0000 (13:46 -0400)]
Fix misprocessing of equivalence classes involving record_eq().
canonicalize_ec_expression() is supposed to agree with coerce_type() as to
whether a RelabelType should be inserted to make a subexpression be valid
input for the operators of a given opclass. However, it did the wrong
thing with named-composite-type inputs to record_eq(): it put in a
RelabelType to RECORDOID, which the parser doesn't. In some cases this was
harmless because all code paths involving a particular equivalence class
did the same thing, but in other cases this would result in failing to
recognize a composite-type expression as being a member of an equivalence
class that it actually is a member of. The most obvious bad effect was to
fail to recognize that an index on a composite column could provide the
sort order needed for a mergejoin on that column, as reported by Teodor
Sigaev. I think there might be other, subtler, cases that result in
misoptimization. It also seems possible that an unwanted RelabelType
would sometimes get into an emitted plan --- but because record_eq and
friends don't examine the declared type of their input expressions, that
would not create any visible problems.
To fix, just treat RECORDOID as if it were a polymorphic type, which in
some sense it is. We might want to consider formalizing that a bit more
someday, but for the moment this seems to be the only place where an
IsPolymorphicType() test ought to include RECORDOID as well.
This has been broken for a long time, so back-patch to all supported
branches.
Robert Haas [Wed, 16 May 2018 15:32:38 +0000 (11:32 -0400)]
Pass the correct PlannerInfo to PlanForeignModify/PlanDirectModify.
Previously, we passed the toplevel PlannerInfo, but we actually want
to pass the relevant subroot. One problem with passing the toplevel
PlannerInfo is that the FDW which wants to push down an UPDATE or
DELETE against a join won't find the relevant joinrel there.
As of commit 1bc0100d270e5bcc980a0629b8726a32a497e788, postgres_fdw
tries to do exactly this and can be made to fail an assertion as a
result.
It's possible that this should be regarded as a bug fix and
back-patched to earlier releases, but for lack of a test case that
fails in earlier releases, no back-patch for now.
Tom Lane [Tue, 15 May 2018 19:06:53 +0000 (15:06 -0400)]
Fix type checking for support functions of parallel VARIADIC aggregates.
The impact of VARIADIC on the combine/serialize/deserialize support
functions of an aggregate wasn't thought through carefully. There is
actually no impact, because variadicity isn't passed through to these
functions (and it doesn't seem like it would need to be). However,
lookup_agg_function was mistakenly told to check things as though it were
passed through. The net result was that it was impossible to declare an
aggregate that had both VARIADIC input and parallelism support functions.
In passing, fix a runtime check in nodeAgg.c for the combine function's
strictness to make its error message agree with the creation-time check.
The previous message was actually backwards, and it doesn't seem like
there's a good reason to have two versions of this message text anyway.
Back-patch to 9.6 where parallel aggregation was introduced.
Alvaro Herrera [Mon, 14 May 2018 17:09:32 +0000 (13:09 -0400)]
Don't allow partitioned index on foreign-table partitions
Creating indexes on foreign tables is already forbidden, but local
partitioned indexes (commit 8b08f7d4820f) forgot to check for them. Add
a preliminary check to prevent wasting time.
Another school of thought says to allow the index to be created if it's
not a unique index; but it's possible to do better in the future (enable
indexing of foreign tables, somehow), so we avoid painting ourselves in
a corner by rejecting all cases, to avoid future grief (a.k.a. backward
incompatible changes).
Reported-by: Arseny Sher
Author: Amit Langote, Álvaro Herrera
Discussion: https://postgr.es/m/87sh71cakz.fsf@ars-thinkpad
Alvaro Herrera [Sat, 12 May 2018 15:08:17 +0000 (12:08 -0300)]
docs: Rework sections on partition pruning/exclusion
Explain partition pruning more thoroughly, in a section above the one
that explains constraint exclusion, since the new feature is the one
that will be used more extensively from now on. Move some of the
material from the constraint exclusion subsection to the one on
partition pruning, so that we can explain the legacy method by
explaining the differences with the new one instead of repeating it.
Author: David Rowley, Álvaro Herrera Reviewed-by: Amit Langote, David G. Johnston, Justin Pryzby
Discussion: https://postgr.es/m/CAKJS1f8PECxEi1YQ9nhVtshtfOMHUzAMm_Zp4gGCOCnMPjEKJA@mail.gmail.com
Teodor Sigaev [Thu, 10 May 2018 10:31:47 +0000 (13:31 +0300)]
Various improvements of skipping index scan during vacuum technics
- Change vacuum_cleanup_index_scale_factor GUC to PGC_USERSET.
vacuum_cleanup_index_scale_factor GUC was defined as PGC_SIGHUP. But this
GUC affects not only autovacuum. So it might be useful to change it from user
session in order to influence manually runned VACUUM.
- Add missing tab-complete support for vacuum_cleanup_index_scale_factor
reloption.
- Fix condition for B-tree index cleanup.
Zero value of vacuum_cleanup_index_scale_factor means that user wants B-tree
index cleanup to be never skipped.
- Documentation and comment improvements
Authors: Justin Pryzby, Alexander Korotkov, Liudmila Mantrova
Reviewed by: all authors and Robert Haas
Discussion: https://www.postgresql.org/message-id/flat/20180502023025.GD7631%40telsasoft.com
Tom Lane [Wed, 9 May 2018 17:55:27 +0000 (13:55 -0400)]
Update time zone data files to tzdata release 2018e.
DST law changes in North Korea. Redefinition of "daylight savings" in
Ireland, as well as for some past years in Namibia and Czechoslovakia.
Additional historical corrections for Czechoslovakia.
With this change, the IANA database models Irish timekeeping as following
"standard time" in summer, and "daylight savings" in winter, so that the
daylight savings offset is one hour behind standard time not one hour
ahead. This does not change their UTC offset (+1:00 in summer, 0:00 in
winter) nor their timezone abbreviations (IST in summer, GMT in winter),
though now "IST" is more correctly read as "Irish Standard Time" not "Irish
Summer Time". However, the "is_dst" column in the pg_timezone_names view
will now be true in winter and false in summer for the Europe/Dublin zone.
Similar changes were made for Namibia between 1994 and 2017, and for
Czechoslovakia between 1946 and 1947.
So far as I can find, no Postgres internal logic cares about which way
tm_isdst is reported; in particular, since commit b2cbced9e we do not
rely on it to decide how to interpret ambiguous timestamps during DST
transitions. So I don't think this change will affect any Postgres
behavior other than the timezone-view outputs.
Commit 8b08f7d4820f failed to update these modules to at least give
non-broken error messages for partitioned indexes. Add appropriate
error support to them.
Peter G. was complaining about a problem of unfriendly error messages;
while we haven't fixed that yet, subsequent discussion let to discovery
of these unhandled cases.
Author: Michaël Paquier Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzkOKptQiE51Bh4_xeEHhaBwHkZkGtKizrFMgEkfUuRRQg@mail.gmail.com
Alvaro Herrera [Wed, 9 May 2018 16:44:50 +0000 (13:44 -0300)]
pgstatindex: HASH -> hash
Fix the lone error message in the whole source tree to use capitalized
HASH when referring to hash indexes, making it look like all the other
messages.
Someday it would be good to standardize 'B-Tree', 'B-tree', 'btree', and
random other spellings, too, but that's a larger patch ...
Alvaro Herrera [Wed, 9 May 2018 13:51:23 +0000 (10:51 -0300)]
Fix assorted partition pruning bugs
match_clause_to_partition_key failed to consider COERCION_PATH_ARRAYCOERCE
cases in scalar-op-array expressions, so it was possible to crash the
server easily. To handle this case properly (ie. prune partitions) we
would need to run a bit of executor code during planning. Maybe it can
be improved, but for now let's just not crash. Add a test case that
used to trigger the crash.
Author: Michaël Paquier
match_clause_to_partition_key failed to indicate that operators that
don't have a commutator in a btree opclass are unsupported. It is
possible for this to cause a crash later if such an operator is used in
a scalar-op-array expression. Add a test case that used to the crash.
Author: Amit Langote
One caller of gen_partprune_steps_internal in
match_clause_to_partition_key was too optimistic about the former never
returning an empty step list. Rid it of its innocence. (Having fixed
the bug above, I no longer know how to exploit this, so no test case for
it, but it remained a bug.) Revise code flow a little bit, for
succintness.
Author: Álvaro Herrera
Andrew Dunstan [Wed, 9 May 2018 14:14:46 +0000 (10:14 -0400)]
Restrict vertical tightness to parentheses in Perl code
The vertical tightness settings collapse vertical whitespace between
opening and closing brackets (parentheses, square brakets and braces).
This can make data structures in particular harder to read, and is not
very consistent with our style in non-Perl code. This patch restricts
that setting to parentheses only, and reformats all the perl code
accordingly. Not applying this to parentheses has some unfortunate
effects, so the consensus is to keep the setting for parentheses and not
for the others.
The diff for this patch does highlight some places where structures
should have trailing commas. They can be added manually, as there is no
automatic tool to do so.
Alvaro Herrera [Wed, 9 May 2018 13:40:21 +0000 (10:40 -0300)]
Make gen_partprune_steps static
There's no need to export this function, so don't. Michaël didn't
actually write the patch, but we list him as first author because with a
trivial one like this, intellectual authorship is as important (if not
more) as bit shovelling.
Author: Michaël Paquier, Amit Langote
Discussion: https://postgr.es/m/c91299c4-199b-0f16-339b-a29d6d2a39ee@lab.ntt.co.jp
Andrew Dunstan [Wed, 9 May 2018 11:55:23 +0000 (07:55 -0400)]
Add a script and a config file to run perlcritic
This is similar to what we do to run perltidy. For now we only run at
severity level 5. Over time we can improve our perl code and reduce the
severity level.
Teodor Sigaev [Wed, 9 May 2018 10:23:16 +0000 (13:23 +0300)]
Improve jsonb cast error message
Initial variant of error message didn't follow style of another casting error
messages and wasn't informative. Per gripe from Robert Haas.
Reviewer: Tom Lane
Discussion: https://www.postgresql.org/message-id/flat/CA%2BTgmob08StTV9yu04D0idRFNMh%2BUoyKax5Otvrix7rEZC8rMw%40mail.gmail.com#CA+Tgmob08StTV9yu04D0idRFNMh+UoyKax5Otvrix7rEZC8rMw@mail.gmail.com
Tom Lane [Wed, 9 May 2018 00:17:43 +0000 (20:17 -0400)]
Improve inefficient regexes in vacuumdb TAP test.
The regexes used in 102_vacuumdb_stages.pl to check the postmaster log
for expected output contained several places with ".*.*", which is
underdetermined and can cause exponential runtime growth in Perl's regex
matcher (since it's not bright enough not to waste time seeing whether
different splits of the same substring would allow a match). We were
fortunate that the amount of text in the postmaster log was generally not
enough to make the runtime go to the moon; although commit 6271fceb8 had
been on the hairy edge of an obvious problem, thanks to its increasing the
default log verbosity to DEBUG1. Experimentation shows that anyone who
tried to run this test case with an even higher log verbosity would have
been in for serious pain. But even at default logging level, fixing this
saves several hundred ms on my workstation, more on slower buildfarm
members.
Remove the extra ".*"s, restoring more-or-less-linear matching speed.
Back-patch to 9.4 where the test case was added, mostly in case anyone
tries to do related debugging in a back branch.
Tom Lane [Tue, 8 May 2018 19:59:01 +0000 (15:59 -0400)]
Improve initdb's query for generating default descriptions a little.
While poking into initdb's performance, I noticed that this query
wasn't being done very intelligently. By forcing it to execute
obj_description() for each pg_proc/pg_operator join row, we were
essentially setting up a nestloop join to pg_description, which
is not a bright query plan when there are hundreds of outer rows.
Convert the check for a "deprecated" operator into a NOT EXISTS
so that it can be done as a hashed antijoin. On my workstation
this reduces the time for this query from ~ 35ms to ~ 10ms.
Which is not a huge win, but it adds up over buildfarm runs.
In passing, insert forced query breaks (\n\n, in single-user mode)
after each SQL-query file that initdb sources, and after some
relatively new queries in setup_privileges(). This doesn't make
a lot of difference normally, but it will result in briefer, saner
error messages if anything goes wrong.
Tom Lane [Tue, 8 May 2018 04:20:19 +0000 (00:20 -0400)]
Count heap tuples in non-SnapshotAny path in IndexBuildHeapRangeScan().
Brown-paper-bag bug in commit 7c91a0364: when we rearranged the placement
of "reltuples += 1" statements, we missed including one in this code path.
The net effect of that was that CREATE INDEX CONCURRENTLY would set the
table's pg_class.reltuples to zero, as would index builds done during
bootstrap mode. (It seems like parallel index builds ought to fail
similarly, but they don't, perhaps because reltuples is computed in some
other way. You certainly couldn't figure that out from the abysmally
underdocumented parallelism code in this area.)
I was led to this by wondering why initdb seemed to have slowed down as
a result of 7c91a0364, as is evident in the buildfarm's timing history.
The reason is that every system catalog with indexes had pg_class.reltuples
= 0 after bootstrap, causing the planner to make some terrible choices for
queries in the post-bootstrap steps. On my workstation, this fix causes
the runtime of "initdb -N" to drop from ~2.0 sec to ~1.4 sec, which is
almost though not quite back to where it was in v10. That's not much of
a deal for production use perhaps, but it makes a noticeable difference
for buildfarm and "make check-world" runs, which do a lot of initdbs.
Andrew Dunstan [Mon, 7 May 2018 19:21:00 +0000 (15:21 -0400)]
Clean up some perlcritic warnings
In Catalog.pm, mark eval of a string instead of a block as allowed.
Disallow perlcritic completely in Gen_dummy_probes.pl, as it's
generated code.
Protect a couple of lines in plperl code from perltidy, so that the
annotation for perlcritic stays on the same line as the construct it
would otherwise object to.
Tom Lane [Mon, 7 May 2018 19:12:01 +0000 (15:12 -0400)]
Undo extra chattiness of postmaster logs in TAP tests.
Commit 6271fceb8 changed PostgresNode.pm to force log_min_messages = debug1
in all TAP tests, without any discussion and without a concrete need for
it. This makes some of the TAP tests noticeably slower (although much of
that may be due to poorly-written regexes), and for certain it's bloating
the buildfarm logs. Revert the change.
Tom Lane [Mon, 7 May 2018 18:32:04 +0000 (14:32 -0400)]
Update oidjoins regression test for v11.
Commit 86f575948 already manually updated the oidjoins test for the
new pg_constraint.conparentid => pg_constraint.oid relationship, but
failed to update findoidjoins/README, thus the apparent inconsistency
here.
Tom Lane [Mon, 7 May 2018 17:44:09 +0000 (13:44 -0400)]
Suppress compiler warnings when building with --enable-dtrace.
Most versions of "dtrace -h" drop const qualifiers from the declarations
of probe functions (though macOS gets it right). This causes compiler
warnings when we pass in pointers to const. Repair by extending our
existing post-processing of the probes.h file. To do so, assume that all
"char *" arguments should be "const char *"; that seems reasonably safe.
Tom Lane [Mon, 7 May 2018 17:13:27 +0000 (13:13 -0400)]
Last-minute updates for release notes.
The set of functions that need parallel-safety adjustments isn't the
same in 9.6 as 10, so I shouldn't have blindly back-patched that list.
Adjust as needed. Also, provide examples of the commands to issue.
There shouldn't be a line break between two adjacent tags, because that
will appear as whitespace in the output. (The rendering engine might in
turn collapse that whitespace away, so it might not actually make a
difference, but it's more correct this way.)
Stephen Frost [Mon, 7 May 2018 14:10:33 +0000 (10:10 -0400)]
adminpack: Revoke EXECUTE on pg_logfile_rotate()
In 9.6, we moved a number of functions over to using the GRANT system to
control access instead of having hard-coded superuser checks.
As it turns out, adminpack was creating another function in the catalog
for one of those backend functions where the superuser check was
removed, specifically pg_rotate_logfile(), but it didn't get the memo
about having to REVOKE EXECUTE on the alternative-name function
(pg_logfile_rotate()), meaning that in any installations with adminpack
on 9.6 and higher, any user is able to run the pg_logfile_rotate()
function, which then calls pg_rotate_logfile() and rotates the logfile.
Fix by adding a new version of adminpack (1.1) which handles the REVOKE.
As this function should have only been available to the superuser, this
is a security issue, albeit a minor one.
In HEAD, move the changes implemented for adminpack up to be adminpack
2.0 instead of 1.1.
projects / postgresql / log