Tom Lane [Mon, 5 Sep 2005 17:25:01 +0000 (17:25 +0000)]
For non-projecting plan node types such as Limit, set_plan_references
has to recopy the input plan node's targetlist if it removes a
SubqueryScan node just below the non-projecting node. For simplicity
I made it recopy always. Per bug report from Allan Wang and Michael Fuhr.
Bruce Momjian [Sat, 3 Sep 2005 00:41:30 +0000 (00:41 +0000)]
Add idea:
< cmin/cmax pair and is stored in local memory.
> cmin/cmax pair and is stored in local memory. Another idea is to
> store both cmin and cmax only in local memory.
Bruce Momjian [Fri, 2 Sep 2005 21:23:05 +0000 (21:23 +0000)]
Update.
< have its heap and index files truncated. One issue is
< that no other backend should be able to add to the table
< at the same time, which is something that is currently
< allowed.
> removed or have its heap and index files truncated. One
> issue is that no other backend should be able to add to
> the table at the same time, which is something that is
> currently allowed.
Bruce Momjian [Fri, 2 Sep 2005 21:10:59 +0000 (21:10 +0000)]
Update:
> o Allow COPY on a newly-created table to skip WAL logging 450a452,456
> On crash recovery, the table involved in the COPY would
> have its heap and index files truncated. One issue is
> that no other backend should be able to add to the table
> at the same time, which is something that is currently
> allowed.
Bruce Momjian [Fri, 2 Sep 2005 20:11:34 +0000 (20:11 +0000)]
Add:
> * Use UTF8 encoding for NLS messages so all server encodings can
> read them properly
< o %Add support for Unicode
<
< To fix this, the data needs to be converted to/from UTF16/UTF8
< so the Win32 wcscoll() can be used, and perhaps other functions
< like towupper(). However, UTF8 already works with normal
< locales but provides no ordering or character set classes.
Bruce Momjian [Fri, 2 Sep 2005 19:51:37 +0000 (19:51 +0000)]
Update wording:
< could only see committed rows from another transaction. However,
> could only see rows from another completed transaction. However, 981c981
< proper visibility of the row, for example, for cursors.
> proper visibility of the row's cmin, for example, for cursors.
Tom Lane [Fri, 2 Sep 2005 19:02:20 +0000 (19:02 +0000)]
Clean up a couple of ad-hoc computations of the maximum number of tuples
on a page, as suggested by ITAGAKI Takahiro. Also, change a few places
that were using some other estimates of max-items-per-page to consistently
use MaxOffsetNumber. This is conservatively large --- we could have used
the new MaxHeapTuplesPerPage macro, or a similar one for index tuples ---
but those places are simply declaring a fixed-size buffer and assuming it
will work, rather than actively testing for overrun. It seems safer to
size these buffers in a way that can't overflow even if the page is
corrupt.
Tom Lane [Fri, 2 Sep 2005 18:55:32 +0000 (18:55 +0000)]
In copy_file, use a palloc'd buffer instead of just a local char array;
a local array isn't guaranteed to have any particular alignment, and
so it could slow down the data transfer.
Bruce Momjian [Fri, 2 Sep 2005 15:22:44 +0000 (15:22 +0000)]
Add text:
* Merge xmin/xmax/cmin/cmax back into three header fields
Before subtransactions, there used to be only three fields needed to
store these four values. This was possible because only the current
transaction looks at the cmin/cmax values. If the current transaction
created and expired the row the fields stored where xmin (same as
xmax), cmin, cmax, and if the transaction was expiring a row from a
another transaction, the fields stored were xmin (cmin was not
needed), xmax, and cmax. Such a system worked because a transaction
could only see committed rows from another transaction. However,
subtransactions can see rows from outer transactions, and once the
subtransaction completes, the outer transaction continues, requiring
the storage of all four fields. With subtransactions, an outer
transaction can create a row, a subtransaction expire it, and when the
subtransaction completes, the outer transaction still has to have
proper visibility of the row, for example, for cursors.
One possible solution is to create a phantom cid which represents a
cmin/cmax pair and is stored in local memory.
Tom Lane [Fri, 2 Sep 2005 00:57:57 +0000 (00:57 +0000)]
Update performance discussion for 8.1. Add a little more explanatory
material in the EXPLAIN section, update examples to match current reality,
show examples of bitmap indexscans as well as plain ones.
Bruce Momjian [Fri, 2 Sep 2005 00:49:57 +0000 (00:49 +0000)]
Duplicate:
< * Maintain a map of recently-expired rows
<
< This allows vacuum to target specific pages for possible free space
< without requiring a sequential scan.
<
Update entry:
> One complexity is that index entries still have to be vacuumed, and
> doing this without an index scan (by using the heap values to find the
> index entry) might be slow and unreliable, especially for user-defined
> index functions.
Tom Lane [Thu, 1 Sep 2005 18:15:42 +0000 (18:15 +0000)]
Short-circuit AllocSetReset if nothing has been palloc'd in the memory
context since the previous AllocSetReset. Original patch by Atsushi Ogawa,
editorialized on a little bit by Tom Lane.
Tom Lane [Tue, 30 Aug 2005 02:47:37 +0000 (02:47 +0000)]
Log send() failures when sending to statistics process (but only in
assert-enabled builds). This is a temporary measure to see if we can
learn anything about those intermittent stats test failures in the
buildfarm.
Tom Lane [Mon, 29 Aug 2005 21:38:18 +0000 (21:38 +0000)]
Reduce default value of max_prepared_transactions from 50 to 5. This
saves nearly 700kB in the default shared memory segment size, which seems
worthwhile, and it is a feature that many users won't use anyway. Per
Heikki's argument, there is no point in a compromise value --- those who
are using 2PC at all will probably want it at least equal to max_connections.
But we can't set it to zero by default without breaking the prepared_xacts
regression test.
Tom Lane [Mon, 29 Aug 2005 19:39:39 +0000 (19:39 +0000)]
Fix platform-specific test for path prefix-ness: move it into path.c where
it can be done right. Allow explicit use of absolute DataDir path.
Per Dave Page.
Tom Lane [Sun, 28 Aug 2005 22:47:20 +0000 (22:47 +0000)]
Tweak nodeBitmapAnd to stop evaluating sub-plan scans if it finds it's
got an empty bitmap after any step; the remaining subplans can no longer
affect the result. Per a suggestion from Ilia Kantor.
Tom Lane [Sun, 28 Aug 2005 16:31:37 +0000 (16:31 +0000)]
dumpUserConfig failed (in a pretty harmless way, but failed nonetheless)
to cope with a group name when dumping from a pre-8.1 installation.
Per report from Stefan Kaltenbrunner.
Tom Lane [Sat, 27 Aug 2005 22:13:44 +0000 (22:13 +0000)]
Change the division of labor between grouping_planner and query_planner
so that the latter estimates the number of groups that grouping will
produce. This is needed because it is primarily query_planner that
makes the decision between fast-start and fast-finish plans, and in the
original coding it was unable to make more than a crude rule-of-thumb
choice when the query involved grouping. This revision helps us make
saner choices for queries like SELECT ... GROUP BY ... LIMIT, as in a
recent example from Mark Kirkwood. Also move the responsibility for
canonicalizing sort_pathkeys and group_pathkeys into query_planner;
this information has to be available anyway to support the first change,
and doing it this way lets us get rid of compare_noncanonical_pathkeys
entirely.
Tom Lane [Sat, 27 Aug 2005 18:04:49 +0000 (18:04 +0000)]
Fix two separate bugs in setrefs.c. set_subqueryscan_references needs
to copy the whole plan tree before invoking adjust_plan_varnos(); else
if there is any multiply-linked substructure, the latter might increment
some Var's varno twice. Previously there were some retail copyObject
calls inside adjust_plan_varnos, but it seems a lot safer to just dup the
whole tree first. Also, set_inner_join_references was trying to avoid
work by not recursing if a BitmapHeapScan's bitmapqualorig contained no
outer references; which was OK at the time the code was written, I think,
but now that create_bitmap_scan_plan removes duplicate clauses from
bitmapqualorig it is possible for that field to be NULL while outer
references still remain in the qpqual and/or contained indexscan nodes.
For safety, always recurse even if the BitmapHeapScan looks to be outer
reference free. Per reports from Michael Fuhr and Oleg Bartunov.
Tom Lane [Sat, 27 Aug 2005 16:22:48 +0000 (16:22 +0000)]
Get the MIPS assembler syntax right. Also add a separate sync command;
the reference I consulted yesterday said SC does a SYNC, but apparently
this is not true on newer MIPS processors, so be safe.
Bruce Momjian [Sat, 27 Aug 2005 13:38:43 +0000 (13:38 +0000)]
Update:
>
> Another issue is whether underlying table changes should be reflected
> in the view, e.g. should SELECT * show additional columns if they
> are added after the view is created.
Bruce Momjian [Fri, 26 Aug 2005 20:38:57 +0000 (20:38 +0000)]
Add:
> o Issue a warning if a change-on-restart-only postgresql.conf value
> is modified and the server config files are reloaded
> o Mark change-on-restart-only values in postgresql.conf 205a209
> o Fix SELECT '0.01 years'::interval, '0.01 months'::interval
Bruce Momjian [Fri, 26 Aug 2005 19:41:18 +0000 (19:41 +0000)]
Add description and item:
>
> Currently, while \e saves a single query as one entry, interactive
> queries are saved one line at a time. Ideally all queries
> whould be saved like \e does.
>
> o Allow multi-line column values to align in the proper columns
>
> If the second output column value is 'a\nb', the 'b' should appear
> in the second display column, rather than the first column as it
> does now.
Bruce Momjian [Fri, 26 Aug 2005 19:32:43 +0000 (19:32 +0000)]
Update based on user comments:
< in PL/PgSQL is to use EXECUTE.
> in PL/PgSQL is to use EXECUTE. One complexity is that a function
> might itself drop and recreate dependent tables, causing it to
> invalidate its own query plan.
Bruce Momjian [Fri, 26 Aug 2005 19:10:48 +0000 (19:10 +0000)]
Update constraint_exclusion items:
< inheritance, and allow it to work for UPDATE and DELETE queries
> inheritance, allow it to work for UPDATE and DELETE queries, and allow
> it to be used for all queries with little performance impact
< * Allow constraint_elimination to be automatically performed
<
< This requires additional code to reduce the performance loss caused by
< constraint elimination.
Bruce Momjian [Fri, 26 Aug 2005 18:59:48 +0000 (18:59 +0000)]
Remove completed items:
< * -Allow limits on per-db/role connections
43d41
< * -Prevent dropping user that still owns objects, or auto-drop the objects
49d46
< * -Add the client IP address and port to pg_stat_activity
< * -Add session start time and last statement time to pg_stat_activity
< * -Add a function that returns the start time of the postmaster 230d224
< o -Allow MIN()/MAX() on arrays
< o -Modify array literal representation to handle array index lower bound
< of other than one 253d244
< * -Add function to return compressed length of TOAST data values
< * -Prevent to_char() on interval from returning meaningless values
<
< For example, to_char('1 month', 'mon') is meaningless. Basically,
< most date-related parameters to to_char() are meaningless for
< intervals because interval is not anchored to a date.
<
< * -Have views on temporary tables exist in the temporary namespace
< * -Allow temporary views on non-temporary tables 329d311
< * -Add BETWEEN SYMMETRIC/ASYMMETRIC
< * -Add E'' escape string marker so eventually ordinary strings can treat
< backslashes literally, for portability
<
< * -Allow additional tables to be specified in DELETE for joins
<
< UPDATE already allows this (UPDATE...FROM) but we need similar
< functionality in DELETE. It's been agreed that the keyword should
< be USING, to avoid anything as confusing as DELETE FROM a FROM b.
< 341d313
< * -Allow REINDEX to rebuild all database indexes
< * -Add an option to automatically use savepoints for each statement in a
< multi-statement transaction.
<
< When enabled, this would allow errors in multi-statement transactions
< to be automatically ignored.
< 426d391
< o -Allow FOR UPDATE queries to do NOWAIT locks 473d437
< o -Allow COPY to understand \x as a hex byte
< o -Allow COPY to optionally include column headings in the first line
< o -Allow COPY FROM ... CSV to interpret newlines and carriage
< returns in data 525d485
< o -Have SHOW ALL show descriptions for server-side variables
< o -Allow PL/PgSQL's RAISE function to take expressions
<
< Currently only constants are supported.
<
< o -Change PL/PgSQL to use palloc() instead of malloc() 545d499
< o -Allow PL/pgSQL EXECUTE query_var INTO record_var; 550d503
< o -Pass arrays natively instead of as text between plperl and postgres 598d550
< o -Add dumping and restoring of LOB comments 638d589
< * -Implement shared row locks and use them in RI triggers 642d592
< * -Allow triggers to be disabled
< * -Add two-phase commit
<
<
< * -Prevent inherited tables from expanding temporary subtables of other
< sessions
< * -Use indexes for MIN() and MAX()
<
< MIN/MAX queries can already be rewritten as SELECT col FROM tab ORDER
< BY col {DESC} LIMIT 1. Completing this item involves doing this
< transformation automatically.
<
< * -Use index to restrict rows returned by multi-key index when used with
< non-consecutive keys to reduce heap accesses
<
< For an index on col1,col2,col3, and a WHERE clause of col1 = 5 and
< col3 = 9, spin though the index checking for col1 and col3 matches,
< rather than just col1; also called skip-scanning.
<
< * -Fetch heap pages matching index entries in sequential order
<
< Rather than randomly accessing heap pages based on index entries, mark
< heap pages needing access in a bitmap and do the lookups in sequential
< order. Another method would be to sort heap ctids matching the index
< before accessing the heap rows.
<
< * -Allow non-bitmap indexes to be combined by creating bitmaps in memory
<
< This feature allows separate indexes to be ANDed or ORed together. This
< is particularly useful for data warehousing applications that need to
< query the database in an many permutations. This feature scans an index
< and creates an in-memory bitmap, and allows that bitmap to be combined
< with other bitmap created in a similar way. The bitmap can either index
< all TIDs, or be lossy, meaning it records just page numbers and each
< page tuple has to be checked for validity in a separate pass.
<
< * -Fix incorrect rtree results due to wrong assumptions about "over"
< operator semantics 782d694
< o -Add concurrency to GIST 813d724
< * -Allow multiple blocks to be written to WAL with one write()
< * -Consider use of open/fcntl(O_DIRECT) to minimize OS caching,
< for WAL writes
<
< O_DIRECT doesn't have the same media write guarantees as fsync, so it
< is in addition to the fsync method, not in place of it.
<
< * -Cache last known per-tuple offsets to speed long tuple access
< * -Allow the size of the buffer cache used by temporary objects to be
< specified as a GUC variable
<
< Larger local buffer cache sizes requires more efficient handling of
< local cache lookups.
<
< * -Improve the background writer
<
< Allow the background writer to more efficiently write dirty buffers
< from the end of the LRU cache and use a clock sweep algorithm to
< write other dirty buffers to reduced checkpoint I/O
< 897d788
< * -Add a warning when the free space map is too small 917d807
< o -Move into the backend code
< * -Make locking of shared data structures more fine-grained
<
< This requires that more locks be acquired but this would reduce lock
< contention, improving concurrency.
<
< * -Improve SMP performance on i386 machines
<
< i386-based SMP machines can generate excessive context switching
< caused by lock failure in high concurrency situations. This may be
< caused by CPU cache line invalidation inefficiencies.
< 979d857
< o -Add ability to turn off full page writes
< * -Eliminate WAL logging for CREATE TABLE AS when not doing WAL archiving
< * -Change WAL to use 32-bit CRC, for performance reasons
<
< * -Use CHECK constraints to influence optimizer decisions
<
< CHECK constraints contain information about the distribution of values
< within the table. This is also useful for implementing subtables where
< a tables content is distributed across several subtables.
< 1045d913
< * -ANALYZE should record a pg_statistic entry for an all-NULL column 1099d966
< * -Remove kerberos4 from source tree 1103d969
< * -Make src/port/snprintf.c thread-safe 1118d983
< * -Add C code on Unix to copy directories for use in creating new databases 1133d997
< o -Improve dlerror() reporting string
Bruce Momjian [Fri, 26 Aug 2005 18:52:44 +0000 (18:52 +0000)]
Great TODO updates from Tom:
< Currently SIGTERM of a backend can lead to lock table corruption.
> Lock table corruption following SIGTERM of an individual backend
> has been reported in 8.0. A possible cause was fixed in 8.1, but
> it is unknown whether other problems exist. This item mostly
> requires additional testing rather than of writing any new code.
< o Allow postgresql.conf values to be set so they can not be changed
< by the user 166c167,171
< * %Remove Money type, add money formatting for decimal type
> * Improve the MONEY data type
>
> Change the MONEY data type to use DECIMAL internally, with special
> locale-aware output formatting.
> 225c230
< o %Allow MIN()/MAX() on arrays
> o -Allow MIN()/MAX() on arrays 228c233
< o Modify array literal representation to handle array index lower bound
> o -Modify array literal representation to handle array index lower bound 235a241
> o Auto-delete large objects when referencing row is deleted
< Currently large objects entries do not have owners. Permissions can
< only be set at the pg_largeobject table level.
> /contrib/lo offers this functionality. 240d244
< o Auto-delete large objects when referencing row is deleted
< * %Have views on temporary tables exist in the temporary namespace
< * Allow temporary views on non-temporary tables
< * %Allow RULE recompilation
> * -Have views on temporary tables exist in the temporary namespace
> * -Allow temporary views on non-temporary tables
> * Allow VIEW/RULE recompilation when the underlying tables change 340a345,347
>
> This is like DELETE CASCADE, but truncates.
> 381c388
< * Make row-wise comparisons work per SQL spec
> * %Make row-wise comparisons work per SQL spec
< o Currently the system uses the operating system COPY command to
< create a new database. Add ON COMMIT capability to CREATE TABLE AS
< SELECT
> o Add ON COMMIT capability to CREATE TABLE AS ... SELECT 427c432
< o %Add ALTER DOMAIN TYPE
> o Add ALTER DOMAIN to modify the underlying data type
< o %Disallow dropping of an inherited constraint
< o -Allow objects to be moved to different schemas
> o Add missing object types for ALTER ... SET SCHEMA
< o %Prevent child tables from altering constraints like CHECK that were
< inherited from the parent table
> o %Disallow dropping of an inherited constraint
> o %Prevent child tables from altering or dropping constraints
> like CHECK that were inherited from the parent table
< o Handle references to temporary tables that are created, destroyed,
< then recreated during a session, and EXECUTE is not used
<
< This requires the cached PL/PgSQL byte code to be invalidated when
< an object referenced in the function is changed.
<
< o Add table function support to pltcl, plperl, plpython?
< o Allow PL/pgSQL to name columns by ordinal position, e.g. rec.(3)
> o Add table function support to pltcl, plpython 549a548
> o Allow function argument names to be queries from PL/PgSQL
< o Pass arrays natively instead of as text between plperl and postgres
< o Add support for polymorphic arguments and return types to plperl
> o -Pass arrays natively instead of as text between plperl and postgres
> o Add support for polymorphic arguments and return types to
> languages other than PL/PgSQL
> o Add support for OUT and INOUT parameters to languages other
> than PL/PgSQL
< * Allow libpq to access SQLSTATE so pg_ctl can test for connection failure
<
< This would be used for checking if the server is up.
< 565c563
< * Have initdb set DateStyle based on locale?
> * Have initdb set the input DateStyle (MDY or DMY) based on locale? 567d564
< * Add a schema option to createlang
< o Add pg_dumpall custom format dumps.
<
< This is probably best done by combining pg_dump and pg_dumpall
< into a single binary.
<
> o Add pg_dumpall custom format dumps? 612c605,606
< o Remove unnecessary abstractions in pg_dump source code
> o Remove unnecessary function pointer abstractions in pg_dump source
> code
< * %Remove CREATE CONSTRAINT TRIGGER
<
< This was used in older releases to dump referential integrity
< constraints.
< 682a672,675
> This is particularly important for references to temporary tables
> in PL/PgSQL because PL/PgSQL caches query plans. The only workaround
> in PL/PgSQL is to use EXECUTE.
> 748c741
< * Fetch heap pages matching index entries in sequential order
> * -Fetch heap pages matching index entries in sequential order 797c790
< Currently no only one hash bucket can be stored on a page. Ideally
> Currently only one hash bucket can be stored on a page. Ideally 806a800,802
> o Add WAL logging for crash recovery
> o Allow multi-column hash indexes
> 812a809,812
>
> Ideally this requires a separate test program that can be run
> at initdb time or optionally later.
> 867c867
< * Improve the background writer
> * -Improve the background writer
< For large table adjustements during vacuum, it is faster to reindex
< rather than update the index.
> For large table adjustements during VACUUM FULL, it is faster to
> reindex rather than update the index.
< * Reduce lock time by moving tuples with read lock, then write
< lock and truncate table
> * Reduce lock time during VACUUM FULL by moving tuples with read lock,
> then write lock and truncate table 919c919,920
< o %Suggest VACUUM FULL if a table is nearly empty
> o %Issue log message to suggest VACUUM FULL if a table is nearly
> empty? 995d995
< * Add WAL index reliability improvement to non-btree indexes 1045c1045
< * ANALYZE should record a pg_statistic entry for an all-NULL column
> * -ANALYZE should record a pg_statistic entry for an all-NULL column 1047a1048,1051
> * Allow constraint_elimination to be automatically performed
>
> This requires additional code to reduce the performance loss caused by
> constraint elimination. 1090c1094
< * Remove memory/file descriptor freeing before ereport(ERROR)
> * %Remove memory/file descriptor freeing before ereport(ERROR)
< * Promote debug_query_string into a server-side function current_query()
< * Allow the identifier length to be increased via a configure option
> * %Promote debug_query_string into a server-side function current_query()
> * %Allow the identifier length to be increased via a configure option 1113d1116
< * Fix cross-compiling of time zone database via 'zic' 1130c1133
< o Improve dlerror() reporting string
> o -Improve dlerror() reporting string 1132c1135
< o Add support for Unicode
> o %Add support for Unicode
Tom Lane [Fri, 26 Aug 2005 03:08:15 +0000 (03:08 +0000)]
Arrange for indexes and toast tables to inherit their ownership from
the parent table, even if the command that creates them is executed by
someone else (such as a superuser or a member of the owning role).
Per gripe from Michael Fuhr.
Tom Lane [Thu, 25 Aug 2005 05:01:43 +0000 (05:01 +0000)]
Factor out the common subexpression month_remainder * DAYS_PER_MONTH
in interval_mul and interval_div. This avoids an optimization bug
in A Certain Company's compiler (and given their explanation, I wouldn't
be surprised if other compilers blow it too). Besides the code seems
more clear this way --- in the original formulation, you had to mentally
recognize the common subexpression in order to understand what was going
on.
Tom Lane [Thu, 25 Aug 2005 02:22:59 +0000 (02:22 +0000)]
Original assumption that our own getaddrinfo routine would never support
IPv6 is obsoleted by recent Windows patch. Perform the runtime test
whenever HAVE_IPV6 is set. This should be OK since initdb can get
getaddrinfo from libpgport if needed.
Tom Lane [Wed, 24 Aug 2005 22:13:23 +0000 (22:13 +0000)]
Add code to use Windows getaddrinfo and friends if they exist. This
has to be tested dynamically if we want to make portable executables.
Chuck McDevitt and Andrew Dunstan.
Tom Lane [Wed, 24 Aug 2005 20:20:38 +0000 (20:20 +0000)]
Back off allow_nonpic_in_shlib for FreeBSD builds: only set it on i386.
This might be overly conservative, but we know it does not work on amd64.
Per buildfarm results.
Bruce Momjian [Wed, 24 Aug 2005 19:43:26 +0000 (19:43 +0000)]
Update:
< Currently, if a variable is commented out, it keeps the
< previous uncommented value until a server restarted.
> Currently, if a variable is commented out, it keeps the
> previous uncommented value until a server restarted.
> Logically, a reload should set the same values as a
> server restart.
Tom Lane [Wed, 24 Aug 2005 18:16:58 +0000 (18:16 +0000)]
Fix up plperl 'use_strict' so that it can be enabled or disabled on the
fly. Fix problem with incompletely duplicated setup code. Andrew Dunstan,
from an idea of Michael Fuhr's.
Tom Lane [Wed, 24 Aug 2005 17:50:00 +0000 (17:50 +0000)]
Add small hack to support use of Unicode-based locales on WIN32. This
is not adequately tested yet, but let's get it into beta1 so it can be
tested. Magnus Hagander and Tom Lane.