Bruce Momjian [Sat, 23 Apr 2005 21:43:24 +0000 (21:43 +0000)]
Remove item, not sure what it refers to:
< * Allow ORDER BY ... LIMIT 1 to select high/low value without sort or
< index using a sequential scan for highest/lowest values
<
< If only one value is needed, there is no need to sort the entire
< table. Instead a sequential scan could get the matching value.
<
Bruce Momjian [Sat, 23 Apr 2005 21:39:27 +0000 (21:39 +0000)]
Update threading item:
< Solaris) might benefit from threading.
> Solaris) might benefit from threading. Also explore the idea of
> a single session using multiple threads to execute a query faster.
Tom Lane [Sat, 23 Apr 2005 21:32:34 +0000 (21:32 +0000)]
Remove explicit FreeExprContext calls during plan node shutdown. The
ExprContexts will be freed anyway when FreeExecutorState() is reached,
and letting that routine do the work is more efficient because it will
automatically free the ExprContexts in reverse creation order. The
existing coding was effectively freeing them in exactly the worst
possible order, resulting in O(N^2) behavior inside list_delete_ptr,
which becomes highly visible in cases with a few thousand plan nodes.
ExecFreeExprContext is now effectively a no-op and could be removed,
but I left it in place in case we ever want to put it back to use.
Bruce Momjian [Sat, 23 Apr 2005 18:57:25 +0000 (18:57 +0000)]
Update FAQ items to point to existing web pages rather than duplication
such information. Remove MySQL mention. Move server-side debug item to
developer's FAQ. Update URLs.
Tom Lane [Sat, 23 Apr 2005 17:22:16 +0000 (17:22 +0000)]
Define the right-hand input of AT TIME ZONE as a full a_expr instead of
c_expr. Perhaps the restriction was once needed to avoid bison errors,
but it seems to work just fine now --- and even generates a slightly
smaller state machine. This change allows examples like
SELECT '13:45'::timetz AT TIME ZONE '-07:00'::interval;
to work without parentheses around the right-hand input.
Tom Lane [Sat, 23 Apr 2005 04:42:53 +0000 (04:42 +0000)]
Turns out that my recent elimination of the 'redundant' flatten_andors()
code in prepqual.c had a small drawback: the flatten_andors code was
able to cope with deeply nested AND/OR structures (like 10000 ORs in
a row), whereas eval_const_expressions tends to recurse until it
overruns the stack. Revise eval_const_expressions so that it doesn't
choke on deeply nested ANDs or ORs.
Tom Lane [Sat, 23 Apr 2005 01:57:34 +0000 (01:57 +0000)]
Teach choose_bitmap_and() to actually be choosy --- that is, try to
make some estimate of which available indexes to AND together, rather
than blindly taking 'em all. This could probably stand further
improvement, but it seems to do OK in simple tests.
Tom Lane [Fri, 22 Apr 2005 21:58:32 +0000 (21:58 +0000)]
First cut at planner support for bitmap index scans. Lots to do yet,
but the code is basically working. Along the way, rewrite the entire
approach to processing OR index conditions, and make it work in join
cases for the first time ever. orindxpath.c is now basically obsolete,
but I left it in for the time being to allow easy comparison testing
against the old implementation.
Bruce Momjian [Fri, 22 Apr 2005 15:40:16 +0000 (15:40 +0000)]
Fix typo:
< Currently indexes do not have enough tuple tuple visibility
< information to allow data to be pulled from the index without
< also accessing the heap. One way to allow this is to set a bit
< to index tuples to indicate if a tuple is currently visible to
< all transactions when the first valid heap lookup happens. This
< bit would have to be cleared when a heap tuple is expired.
> Currently indexes do not have enough tuple visibility information
> to allow data to be pulled from the index without also accessing
> the heap. One way to allow this is to set a bit to index tuples
> to indicate if a tuple is currently visible to all transactions
> when the first valid heap lookup happens. This bit would have to
> be cleared when a heap tuple is expired.
Tom Lane [Thu, 21 Apr 2005 19:18:13 +0000 (19:18 +0000)]
Rethink original decision to use AND/OR Expr nodes to represent bitmap
logic operations during planning. Seems cleaner to create two new Path
node types, instead --- this avoids duplication of cost-estimation code.
Also, create an enable_bitmapscan GUC parameter to control use of bitmap
plans.
Bruce Momjian [Thu, 21 Apr 2005 15:20:39 +0000 (15:20 +0000)]
Updated text for bitmaps:
< Bitmap indexes index single columns that can be combined with other bitmap
< indexes to dynamically create a composite index to match a specific query.
< Each index is a bitmap, and the bitmaps are bitwise AND'ed or OR'ed to be
< combined. They can index by tid or can be lossy requiring a scan of the
< heap page to find matching rows, or perhaps use a mixed solution where
< tids are recorded for pages with only a few matches and per-page bitmaps
< are used for more dense pages. Another idea is to use a 32-bit bitmap
< for every page and set a bit based on the item number mod(32).
> This feature allows separate indexes to be ANDed or ORed together. This
> is particularly useful for data warehousing applications that need to
> query the database in an many permutations. This feature scans an index
> and creates an in-memory bitmap, and allows that bitmap to be combined
> with other bitmap created in a similar way. The bitmap can either index
> all TIDs, or be lossy, meaning it records just page numbers and each
> page tuple has to be checked for validity in a separate pass.
Tom Lane [Wed, 20 Apr 2005 15:48:36 +0000 (15:48 +0000)]
Minor performance improvement: avoid unnecessary creation/unioning of
bitmaps for multiple indexscans. Instead just let each indexscan add
TIDs directly into the BitmapOr node's result bitmap.
Tom Lane [Tue, 19 Apr 2005 22:35:18 +0000 (22:35 +0000)]
Create executor and planner-backend support for decoupled heap and index
scans, using in-memory tuple ID bitmaps as the intermediary. The planner
frontend (path creation and cost estimation) is not there yet, so none
of this code can be executed. I have tested it using some hacked planner
code that is far too ugly to see the light of day, however. Committing
now so that the bulk of the infrastructure changes go in before the tree
drifts under me.
Bruce Momjian [Tue, 19 Apr 2005 03:55:43 +0000 (03:55 +0000)]
>>>>Luckily, PG 8 is available for this. Do you have a short example?
>>>
>>>No, and I think it should be in the manual as an example.
>>>
>>>You will need to enter a loop that uses exception handling to detect
>>>unique_violation.
>>
>>Pursuant to an IRC discussion to which Dennis Bjorklund and
>>Christopher Kings-Lynne made most of the contributions, please find
>>enclosed an example patch demonstrating an UPSERT-like capability.
>>
Bruce Momjian [Tue, 19 Apr 2005 03:37:20 +0000 (03:37 +0000)]
> >Luckily, PG 8 is available for this. Do you have a short example?
>
> No, and I think it should be in the manual as an example.
>
> You will need to enter a loop that uses exception handling to detect
> unique_violation.
Pursuant to an IRC discussion to which Dennis Bjorklund and
Christopher Kings-Lynne made most of the contributions, please find
enclosed an example patch demonstrating an UPSERT-like capability.
Bruce Momjian [Tue, 19 Apr 2005 03:35:15 +0000 (03:35 +0000)]
The following patch should allow UPDATE_INTERVAL to be specified on the
command line. We find this useful because we frequently deal with
thousands of tables in an environment where neither the databases nor
the tables are updated frequently. This helps allow us to cut down on
the overhead of updating the list for every other primary loop of
pg_autovacuum.
I chose -i as the command-line argument and documented it briefly in
the README.
The patch was applied to the 7.4.7 version of pg_autovacuum in contrib.
Bruce Momjian [Tue, 19 Apr 2005 03:13:59 +0000 (03:13 +0000)]
Attached patch gets rid of the global timezone in the following steps:
* Changes the APIs to the timezone functions to take a pg_tz pointer as
an argument, representing the timezone to use for the selected
operation.
* Adds a global_timezone variable that represents the current timezone
in the backend as set by SET TIMEZONE (or guc, or env, etc).
* Implements a hash-table cache of loaded tables, so we don't have to
read and parse the TZ file everytime we change a timezone. While not
necesasry now (we don't change timezones very often), I beleive this
will be necessary (or at least good) when "multiple timezones in the
same query" is eventually implemented. And code-wise, this was the time
to do it.
There are no user-visible changes at this time. Implementing the
"multiple zones in one query" is a later step...
This also gets rid of some of the cruft needed to "back out a timezone
change", since we previously couldn't check a timezone unless it was
activated first.
Passes regression tests on win32, linux (slackware 10) and solaris x86.
Tom Lane [Mon, 18 Apr 2005 23:47:52 +0000 (23:47 +0000)]
pg_dumpall should enforce the server version check for itself, rather
than simply passing it down to pg_dump. Else, version-related failures
in pg_dumpall itself generate unhelpful error messages.
Tom Lane [Mon, 18 Apr 2005 17:11:05 +0000 (17:11 +0000)]
record_in and record_recv must be careful to return a separately
pfree'able result, since some callers expect to be able to pfree
the result of a pass-by-reference function. Per report from Chris Trawick.
Bruce Momjian [Mon, 18 Apr 2005 15:03:21 +0000 (15:03 +0000)]
Update PITR TODO items:
< failure.
> failure. This could be triggered by a user command or a timer.
< * Force archiving of partially-full WAL files when pg_stop_backup() is
< called or the server is stopped
> * Automatically force archiving of partially-filled WAL files when
> pg_stop_backup() is called or the server is stopped
Tom Lane [Sat, 16 Apr 2005 20:07:35 +0000 (20:07 +0000)]
Create a new 'MultiExecProcNode' call API for plan nodes that don't
return just a single tuple at a time. Currently the only such node
type is Hash, but I expect we will soon have indexscans that can return
tuple bitmaps. A side benefit is that EXPLAIN ANALYZE now shows the
correct tuple count for a Hash node.
Tom Lane [Fri, 15 Apr 2005 22:19:48 +0000 (22:19 +0000)]
Reduce PANIC to ERROR in several xlog routines that are used in both
critical and noncritical contexts (an example of noncritical being
post-checkpoint removal of dead xlog segments). In the critical cases
the CRIT_SECTION mechanism will cause ERROR to be promoted to PANIC
anyway, and in the noncritical cases we shouldn't let an error take
down the entire database. Arguably there should be *no* explicit
PANIC errors in this module, only more START/END_CRIT_SECTION calls,
but I didn't go that far. (Yet.)
Tom Lane [Fri, 15 Apr 2005 18:48:10 +0000 (18:48 +0000)]
Modify MoveOfflineLogs/InstallXLogFileSegment to avoid O(N^2) behavior
when recycling a large number of xlog segments during checkpoint.
The former behavior searched from the same start point each time,
requiring O(checkpoint_segments^2) stat() calls to relocate all the
segments. Instead keep track of where we stopped last time through.
Tom Lane [Fri, 15 Apr 2005 16:40:36 +0000 (16:40 +0000)]
Revert addition of poorly-thought-out DUMP TIMESTAMP archive entry,
which induced bug #1597 in addition to having several other misbehaviors
(like labeling the dump with a completion time having nothing to do with
reality). Instead just print out the desired strings where RestoreArchive
was already emitting the 'PostgreSQL database dump' and
'PostgreSQL database dump complete' strings.
Tom Lane [Thu, 14 Apr 2005 22:34:48 +0000 (22:34 +0000)]
Make equalTupleDescs() compare attlen/attbyval/attalign rather than
assuming comparison of atttypid is sufficient. In a dropped column
atttypid will be 0, and we'd better check the physical-storage data
to make sure the tupdescs are physically compatible.
I do not believe there is a real risk before 8.0, since before that
we only used this routine to compare successive states of the tupdesc
for a particular relation. But 8.0's typcache.c might be comparing
arbitrary tupdescs so we'd better play it safer.
Tom Lane [Thu, 14 Apr 2005 21:44:09 +0000 (21:44 +0000)]
Don't try to constant-fold functions returning RECORD, since the optimizer
isn't presently set up to pass them an expected tuple descriptor. Bug has
been there since 7.3 but was just recently reported by Thomas Hallgren.
Tom Lane [Thu, 14 Apr 2005 20:32:43 +0000 (20:32 +0000)]
Marginal hack to use a specialized hash function for dynahash hashtables
whose keys are OIDs. The only one that looks particularly performance
critical is the relcache hashtable, but as long as we've got the function
we may as well use it wherever it's applicable.
Tom Lane [Thu, 14 Apr 2005 20:03:27 +0000 (20:03 +0000)]
Completion of project to use fixed OIDs for all system catalogs and
indexes. Replace all heap_openr and index_openr calls by heap_open
and index_open. Remove runtime lookups of catalog OID numbers in
various places. Remove relcache's support for looking up system
catalogs by name. Bulky but mostly very boring patch ...
Tom Lane [Thu, 14 Apr 2005 01:38:22 +0000 (01:38 +0000)]
First phase of project to use fixed OIDs for all system catalogs and
indexes. Extend the macros in include/catalog/*.h to carry the info
about hand-assigned OIDs, and adjust the genbki script and bootstrap
code to make the relations actually get those OIDs. Remove the small
number of RelOid_pg_foo macros that we had in favor of a complete
set named like the catname.h and indexing.h macros. Next phase will
get rid of internal use of names for looking up catalogs and indexes;
but this completes the changes forcing an initdb, so it looks like a
good place to commit.
Along the way, I made the shared relations (pg_database etc) not be
'bootstrap' relations any more, so as to reduce the number of hardwired
entries and simplify changing those relations in future. I'm not
sure whether they ever really needed to be handled as bootstrap
relations, but it seems to work fine to not do so now.
Tom Lane [Wed, 13 Apr 2005 18:54:57 +0000 (18:54 +0000)]
Simplify initdb-time assignment of OIDs as I proposed yesterday, and
avoid encroaching on the 'user' range of OIDs by allowing automatic
OID assignment to use values below 16k until we reach normal operation.
initdb not forced since this doesn't make any incompatible change;
however a lot of stuff will have different OIDs after your next initdb.
Tom Lane [Wed, 13 Apr 2005 16:50:55 +0000 (16:50 +0000)]
Change addRangeTableEntryForRelation() to take a Relation pointer instead
of just a relation OID, thereby not having to open the relation for itself.
This actually saves code rather than adding it for most of the existing
callers, which had the rel open already. The main point though is to be
able to use this rather than plain addRangeTableEntry in setTargetTable,
thus saving one relation_openrv/relation_close cycle for every INSERT,
UPDATE, or DELETE. Seems to provide a several percent win on simple
INSERTs.
Tom Lane [Tue, 12 Apr 2005 19:45:43 +0000 (19:45 +0000)]
Adjust pg_cast.h so that the OIDs assigned to built-in casts come from
genbki.sh's pool (10000-16383) instead of being run-time assigned by
heap_insert. Might as well use the pool as long as it's there ...
I was a bit bemused to realize that it hadn't been in use at all since 7.2.
initdb not forced since this doesn't really affect anything. The OIDs
of casts and system indexes will change next time you do one, though.
Tom Lane [Tue, 12 Apr 2005 19:29:24 +0000 (19:29 +0000)]
Remove unnecessary UPDATE commands to assign explicit ACLs to functions
and PL languages during initdb. The default permissions for these objects
are the same as what we were assigning anyway, so there is no need to
expend space in the catalogs on them. The space cost is particularly
significant in pg_proc's indexes, which are bloated by about a factor of 2
by the full-table update, and can never really recover the space.
initdb not forced, since the change has no actual impact on behavior.
Tom Lane [Tue, 12 Apr 2005 04:26:34 +0000 (04:26 +0000)]
Add aggsortop column to pg_aggregate, so that MIN/MAX optimization can
be supported for all datatypes. Add CREATE AGGREGATE and pg_dump support
too. Add specialized min/max aggregates for bpchar, instead of depending
on text's min/max, because otherwise the possible use of bpchar indexes
cannot be recognized.
initdb forced because of catalog changes.
Tom Lane [Mon, 11 Apr 2005 23:06:57 +0000 (23:06 +0000)]
Create the planner mechanism for optimizing simple MIN and MAX queries
into indexscans on matching indexes. For the moment, it only handles
int4 and text datatypes; next step is to add a column to pg_aggregate
so that all MIN/MAX aggregates can be handled. Per my recent proposal.