Tom Lane [Sun, 22 Aug 1999 20:15:04 +0000 (20:15 +0000)]
Further planner/optimizer cleanups. Move all set_tlist_references
and fix_opids processing to a single recursive pass over the plan tree
executed at the very tail end of planning, rather than haphazardly here
and there at different places. Now that tlist Vars do not get modified
until the very end, it's possible to get rid of the klugy var_equal and
match_varid partial-matching routines, and just use plain equal()
throughout the optimizer. This is a step towards allowing merge and
hash joins to be done on expressions instead of only Vars ...
Tom Lane [Sat, 21 Aug 1999 03:49:17 +0000 (03:49 +0000)]
Major revision of sort-node handling: push knowledge of query
sort order down into planner, instead of handling it only at the very top
level of the planner. This fixes many things. An explicit sort is now
avoided if there is a cheaper alternative (typically an indexscan) not
only for ORDER BY, but also for the internal sort of GROUP BY. It works
even when there is no other reason (such as a WHERE condition) to consider
the indexscan. It works for indexes on functions. It works for indexes
on functions, backwards. It's just so cool...
CAUTION: I have changed the representation of SortClause nodes, therefore
THIS UPDATE BREAKS STORED RULES. You will need to initdb.
Tom Lane [Sat, 21 Aug 1999 03:06:58 +0000 (03:06 +0000)]
Cleanups for int8: guard against null inputs in comparison
operators (and some other places), fix rangechecks in int8 to int4
conversion (same problem we recently figured out in pg_atoi).
Tom Lane [Wed, 18 Aug 1999 04:15:16 +0000 (04:15 +0000)]
Remove extraneous SeqScan node that make_noname was inserting
above a Sort or Materialize node. As far as I can tell, the only place
that actually needed that was set_tlist_references, which was being lazy
about checking to see if it had a noname node to fix or not...
Tom Lane [Tue, 17 Aug 1999 21:21:22 +0000 (21:21 +0000)]
Add script that runs the regression tests with all valid
combinations of query-plan-type backend options. Good for testing
planner/optimizer. Tedious, though.
Tom Lane [Mon, 16 Aug 1999 23:07:20 +0000 (23:07 +0000)]
Assign sort keys properly when there are duplicate entries in
pathkey list --- corrects misbehavior seen with multiple mergejoin clauses
mentioning same variable.
Bruce Momjian [Mon, 16 Aug 1999 20:27:19 +0000 (20:27 +0000)]
I've sent 3 mails to pgsql-patches. There are two files, one for doc
and
for src/data directories, and one minor patch for doc/README.locale.
Please apply.
Tom Lane [Mon, 16 Aug 1999 02:17:58 +0000 (02:17 +0000)]
Major planner/optimizer revision: get rid of PathOrder node type,
store all ordering information in pathkeys lists (which are now lists of
lists of PathKeyItem nodes, not just lists of lists of vars). This was
a big win --- the code is smaller and IMHO more understandable than it
was, even though it handles more cases. I believe the node changes will
not force an initdb for anyone; planner nodes don't show up in stored
rules.
Repair the check for redundant UNIQUE and PRIMARY KEY indices.
Also, improve it so that it checks for multi-column constraints.
Thanks to Mark Dalphin <mdalphin@amgen.com> for reporting the problem.
Tom Lane [Sat, 14 Aug 1999 19:29:35 +0000 (19:29 +0000)]
LispUnion routine didn't generate a proper union: anytime
l2 contained more than one entry, there would be duplicates in the output
list. Miscellaneous code beautification in other routines, too.
Tom Lane [Thu, 12 Aug 1999 04:32:54 +0000 (04:32 +0000)]
Clean up optimizer's handling of indexscan quals that need to be
commuted (ie, the index var appears on the right). These are now handled
the same way as merge and hash join quals that need to be commuted: the
actual reversing of the clause only happens if we actually choose the path
and generate a plan from it. Furthermore, the clause is only reversed in
the 'indexqual' field of the plan, not in the 'indxqualorig' field. This
allows the clause to still be recognized and removed from qpquals of upper
level join plans. Also, simplify and generalize match_clause_to_indexkey;
now it recognizes binary-compatible indexes for join as well as restriction
clauses.
Tom Lane [Thu, 12 Aug 1999 00:42:43 +0000 (00:42 +0000)]
Add commentary to show that even though ExecInitIndexScan()
contains much code that looks like it will handle indexquals with the index
key on either side of the operator, in fact indexquals must have the index
key on the left because of limitations of the ScanKey machinery. Perhaps
someone will be motivated to fix that someday...
Tom Lane [Tue, 10 Aug 1999 02:58:56 +0000 (02:58 +0000)]
Revise create_nestloop_node's handling of inner indexscan to
work under a wider range of scenarios than it did --- it formerly did not
handle a multi-pass inner scan, nor cases in which the inner scan's
indxqualorig or non-index qual contained outer var references. I am not
sure that these limitations could be hit in the existing optimizer, but
they need to be fixed for future expansion.
Bruce Momjian [Mon, 9 Aug 1999 06:20:27 +0000 (06:20 +0000)]
> > Prevent sorting if result is already sorted
> >
> > was implemented by Jan Wieck.
> > His work is for ascending order cases.
> >
> > Here is a patch to prevent sorting also in descending
> > order cases.
> > Because I had already changed _bt_first() to position
> > backward correctly before v6.5,this patch would work.
> >
Hiroshi Inoue
Inoue@tpf.co.jp
Tom Lane [Mon, 9 Aug 1999 01:01:42 +0000 (01:01 +0000)]
Rewrite fix_indxqual_references, which was entirely bogus for
multi-scan indexscan plans; it tried to use the same table-to-index
attribute mapping for all the scans, even if they used different indexes.
It would klugily work as long as OR indexquals never used multikey indexes,
but that's not likely to hold up much longer...
Tom Lane [Mon, 9 Aug 1999 00:51:26 +0000 (00:51 +0000)]
Create a standardized expression_tree_mutator support routine
to go along with expression_tree_walker. (_walker is not suitable for
routines that need to alter the tree structure significantly.) Other minor
cleanups in clauses.c.
Tom Lane [Sun, 8 Aug 1999 20:12:52 +0000 (20:12 +0000)]
Fix nbtree's failure to clear BTScans list during xact abort.
Also, move responsibility for calling vc_abort into main xact.c list of
things-to-call-at-abort. What in the world was it doing down inside of
TransactionIdAbort()?
Fix cross-reference markup so that only the *title* of the Operators
chapter is included, not the chapter itself.
Thanks to Evelyn Mitchell <efm@tummy.com> for pointing it out.
Remove explicit references to ref/ path in file names; use vpath instead.
Fix rules for man pages to ensure double-pass to get cross references.
Add a few new man pages.
Try to clarify characteristics of the SERIAL type.
Fix source indenting, which does not affect output.
Note: still need docs on NUMERIC and DECIMAL
(and let's not talk about regression tests :()
Tom Lane [Fri, 6 Aug 1999 04:00:17 +0000 (04:00 +0000)]
Revise generation of hashjoin paths: generate one path per
hashjoinable clause, not one path for a randomly-chosen element of each
set of clauses with the same join operator. That is, if you wrote
SELECT ... WHERE t1.f1 = t2.f2 and t1.f3 = t2.f4,
and both '=' ops were the same opcode (say, all four fields are int4),
then the system would either consider hashing on f1=f2 or on f3=f4,
but it would *not* consider both possibilities. Boo hiss.
Also, revise estimation of hashjoin costs to include a penalty when the
inner join var has a high disbursion --- ie, the most common value is
pretty common. This tends to lead to badly skewed hash bucket occupancy
and way more comparisons than you'd expect on average.
I imagine that the cost calculation still needs tweaking, but at least
it generates a more reasonable plan than before on George Young's example.
Tom Lane [Thu, 5 Aug 1999 02:33:54 +0000 (02:33 +0000)]
Revise parse_coerce() to handle coercion of int and float
constants, not only string constants, at parse time. Get rid of
parser_typecast2(), which is bogus and redundant...
Tom Lane [Tue, 3 Aug 1999 00:09:32 +0000 (00:09 +0000)]
Fix ELF test so it doesn't spit up on all non-ELF systems...
use Autoconf-approved method of testing for predefined symbols, and move
it down to where we know what compiler to run and how to run it.
Tom Lane [Mon, 2 Aug 1999 02:05:41 +0000 (02:05 +0000)]
Further selectivity-estimation work. Speed up eqsel()
(it should just call the given operator, not look up an = operator).
Fix intltsel() so that all numeric data types are converted to double
before trying to estimate where the given comparison value is in the
known range of column values. intltsel() still needs work, or replacement,
for non-numeric data types ... but for nonintegral numeric types it
should now be delivering reasonable estimates.
Bruce Momjian [Sun, 1 Aug 1999 16:30:05 +0000 (16:30 +0000)]
I didn't see any further discussion so here is, I hope, a clean fix to
configure.in to determine if a system is ELF or not. Note that some
of the tests earlier may be redundant but I took the safest route.
Tom Lane [Sun, 1 Aug 1999 04:54:25 +0000 (04:54 +0000)]
First step in fixing selectivity-estimation code. eqsel and
neqsel now behave as per my suggestions in pghackers a few days ago.
selectivity for < > <= >= should work OK for integral types as well, but
still need work for nonintegral types. Since these routines have never
actually executed before :-(, this may result in some significant changes
in the optimizer's choices of execution plans. Let me know if you see
any serious misbehavior.
CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
Tom Lane [Fri, 30 Jul 1999 04:07:25 +0000 (04:07 +0000)]
Further cleanups of indexqual processing: simplify control
logic in indxpath.c, avoid generation of redundant indexscan paths for the
same relation and index.
Tom Lane [Fri, 30 Jul 1999 00:56:17 +0000 (00:56 +0000)]
Fix coredump seen when doing mergejoin between indexed tables,
for example in the regression test database, try
select * from tenk1 t1, tenk1 t2 where t1.unique1 = t2.unique2;
6.5 has this same bug ...
Tom Lane [Thu, 29 Jul 1999 02:48:05 +0000 (02:48 +0000)]
Add support for Case exprs to fix_indxqual_references,
so that Case works in WHERE join clauses. Temporary patch --- this routine
is one of many that ought to be changed to use centralized expression-tree-
walking logic.
Tom Lane [Wed, 28 Jul 1999 17:39:38 +0000 (17:39 +0000)]
Allow a_expr not just AexprConst in the right-hand list of
IN and NOT IN operators. Rewrite grotty implementation of IN-list
parsing ... look Ma, no global variable ...
Tom Lane [Tue, 27 Jul 1999 06:23:12 +0000 (06:23 +0000)]
Correct bug in best_innerjoin(): it should check all the
rels that the inner path needs to join to, but it was only checking for
the first one. Failure could only have been observed with an OR-clause
that mentions 3 or more tables, and then only if the bogus path was
actually selected as cheapest ...
Tom Lane [Tue, 27 Jul 1999 03:53:54 +0000 (03:53 +0000)]
My recent optimizer changes caused a different plan to
be picked for one of the complex joins in rules test ... leading to
a different output ordering ...
Tom Lane [Tue, 27 Jul 1999 03:51:11 +0000 (03:51 +0000)]
First cut at doing LIKE/regex indexing optimization in
optimizer rather than parser. This has many advantages, such as not
getting fooled by chance uses of operator names ~ and ~~ (the operators
are identified by OID now), and not creating useless comparison operations
in contexts where the comparisons will not actually be used as indexquals.
The new code also recognizes exact-match LIKE and regex patterns, and
produces an = indexqual instead of >= and <=.
This change does NOT fix the problem with non-ASCII locales: the code
still doesn't know how to generate an upper bound indexqual for non-ASCII
collation order. But it's no worse than before, just the same deficiency
in a different place...
Also, dike out loc_restrictinfo fields in Plan nodes. These were doing
nothing useful in the absence of 'expensive functions' optimization,
and they took a considerable amount of processing to fill in.
Tom Lane [Sun, 25 Jul 1999 23:07:26 +0000 (23:07 +0000)]
Further work on planning of indexscans. Cleaned up interfaces
to index_selectivity so that it can be handed an indexqual clause list
rather than a bunch of assorted derivative data.
Tom Lane [Sun, 25 Jul 1999 17:53:27 +0000 (17:53 +0000)]
Remove 'restrictinfojoinid' field from RestrictInfo nodes.
The only place it was being used was as temporary storage in indxpath.c,
and the logic was wrong: the same restrictinfo node could get chosen to
carry the info for two different joins. Right fix is to return a second
list of unjoined-relids parallel to the list of clause groups.
Tom Lane [Sat, 24 Jul 1999 23:21:14 +0000 (23:21 +0000)]
Clean up messy clause-selectivity code in clausesel.c; repair bug
identified by Hiroshi (incorrect cost attributed to OR clauses
after multiple passes through set_rest_selec()). I think the code
was trying to allow selectivities of OR subclauses to be passed in
from outside, but noplace was actually passing any useful data, and
set_rest_selec() was passing wrong data.
Restructure representation of "indexqual" in IndexPath nodes so that
it is the same as for indxqual in completed IndexScan nodes: namely,
a toplevel list with an entry for each pass of the index scan, having
sublists that are implicitly-ANDed index qual conditions for that pass.
You don't want to know what the old representation was :-(
Improve documentation of OR-clause indexscan functions.
Remove useless 'notclause' field from RestrictInfo nodes. (This might
force an initdb for anyone who has stored rules containing RestrictInfos,
but I do not think that RestrictInfo ever appears in completed plans.)
Tom Lane [Fri, 23 Jul 1999 03:00:10 +0000 (03:00 +0000)]
Exit cleanups I made yesterday caused pq_close() to be
invoked during exit from a standalone backend, leading to core dump.
This is the cause of the recently reported initdb-time crash :-(.
Sorry folks...