From: Bruce Momjian Date: Tue, 5 Mar 2002 05:33:31 +0000 (+0000) Subject: I attach a version of my toast-slicing patch, against current CVS X-Git-Tag: REL7_3~1970 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=03194432de712f7afb4ddc2ade2bc44f0536dae1;p=postgresql I attach a version of my toast-slicing patch, against current CVS (current as of a few hours ago.) This patch: 1. Adds PG_GETARG_xxx_P_SLICE() macros and associated support routines. 2. Adds routines in src/backend/access/tuptoaster.c for fetching only necessary chunks of a toasted value. (Modelled on latest changes to assume chunks are returned in order). 3. Amends text_substr and bytea_substr to use new methods. It now handles multibyte cases -and should still lead to a performance improvement in the multibyte case where the substring is near the beginning of the string. 4. Added new command: ALTER TABLE tabname ALTER COLUMN colname SET STORAGE {PLAIN | EXTERNAL | EXTENDED | MAIN} to parser and documented in alter-table.sgml. (NB I used ColId as the item type for the storage mode string, rather than a new production - I hope this makes sense!). All this does is sets attstorage for the specified column. 4. AlterTableAlterColumnStatistics is now AlterTableAlterColumnFlags and handles both statistics and storage (it uses the subtype code to distinguish). The previous version of my patch also re-arranged other code in backend/commands/command.c but I have dropped that from this patch.(I plan to return to it separately). 5. Documented new macros (and also the PG_GETARG_xxx_P_COPY macros) in xfunc.sgml. ref/alter_table.sgml also contains documentation for ALTER COLUMN SET STORAGE. John Gray --- diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 6bc5ac5445..2d87902b2f 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -1,5 +1,5 @@ @@ -30,6 +30,8 @@ ALTER TABLE [ ONLY ] table [ * ] class="PARAMETER">value | DROP DEFAULT } ALTER TABLE [ ONLY ] table [ * ] ALTER [ COLUMN ] column SET STATISTICS integer +ALTER TABLE [ ONLY ] table [ * ] + ALTER [ COLUMN ] column SET STORAGE {PLAIN | EXTERNAL | EXTENDED | MAIN} ALTER TABLE [ ONLY ] table [ * ] RENAME [ COLUMN ] column TO newcolumn @@ -169,6 +171,17 @@ ALTER TABLE table The ALTER COLUMN SET STATISTICS form allows you to set the statistics-gathering target for subsequent operations. + The ALTER COLUMN SET STORAGE form allows the + column storage mode to be set. This controls whether this column is + held inline or in a supplementary table, and whether the data + should be compressed or not. PLAIN must be used + for fixed-length values such as INTEGER and is + inline, uncompressed. MAIN is for inline, + compressible data. EXTERNAL is for external, + uncompressed data and EXTENDED is for external, + compressed data. The use of EXTERNAL will make + substring operations on a column faster, at the penalty of + increased storage space. The RENAME clause causes the name of a table, column, index, or sequence to change without changing any of the data. The data will remain of the same type and size after the diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index 20341077c9..94c664cbce 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -1,5 +1,5 @@ @@ -1296,6 +1296,35 @@ concat_text(PG_FUNCTION_ARGS) this works in both strict and nonstrict functions. + + Other options provided in the new-style interface are two + variants of the + PG_GETARG_xxx() + macros. The first of these, + PG_GETARG_xxx_COPY() + guarantees to return a copy of the specified parameter which is + safe for writing into. (The normal macros will sometimes return a + pointer to the value which must not be written to. Using the + PG_GETARG_xxx_COPY() + macros guarantees a writable result.) + + + + The second variant consists of the + PG_GETARG_xxx_SLICE() + macros which take three parameters. The first is the number of the + parameter (as above). The second and third are the offset and + length of the segment to be returned. Offsets are counted from + zero, and a negative length requests that the remainder of the + value be returned. These routines provide more efficient access to + parts of large values in the case where they have storage type + "external". (The storage type of a column can be specified using + ALTER TABLE tablename ALTER + COLUMN colname SET STORAGE + storagetype. Storage type is one of + plain, external, extended or main.) + + The version-1 function call conventions make it possible to return set results and implement trigger functions and diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c index 48a15cf5d3..2a0b5c2762 100644 --- a/src/backend/access/heap/tuptoaster.c +++ b/src/backend/access/heap/tuptoaster.c @@ -8,7 +8,7 @@ * * * IDENTIFICATION - * $Header: /cvsroot/pgsql/src/backend/access/heap/tuptoaster.c,v 1.27 2002/01/16 20:29:01 tgl Exp $ + * $Header: /cvsroot/pgsql/src/backend/access/heap/tuptoaster.c,v 1.28 2002/03/05 05:33:06 momjian Exp $ * * * INTERFACE ROUTINES @@ -47,6 +47,8 @@ static void toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup); static Datum toast_save_datum(Relation rel, Datum value); static varattrib *toast_fetch_datum(varattrib *attr); +static varattrib *toast_fetch_datum_slice(varattrib *attr, + int32 sliceoffset, int32 length); /* ---------- @@ -162,6 +164,80 @@ heap_tuple_untoast_attr(varattrib *attr) } +/* ---------- + * heap_tuple_untoast_attr_slice - + * + * Public entry point to get back part of a toasted value + * from compression or external storage. + * ---------- + */ +varattrib * +heap_tuple_untoast_attr_slice(varattrib *attr, int32 sliceoffset, int32 slicelength) +{ + varattrib *preslice; + varattrib *result; + int32 attrsize; + + if (VARATT_IS_COMPRESSED(attr)) + { + varattrib *tmp; + + if (VARATT_IS_EXTERNAL(attr)) + { + tmp = toast_fetch_datum(attr); + } + else + { + tmp = attr; /* compressed in main tuple */ + } + + preslice = (varattrib *) palloc(attr->va_content.va_external.va_rawsize + + VARHDRSZ); + VARATT_SIZEP(preslice) = attr->va_content.va_external.va_rawsize + VARHDRSZ; + pglz_decompress((PGLZ_Header *) tmp, VARATT_DATA(preslice)); + + if (tmp != attr) + pfree(tmp); + } + else + { + /* Plain value */ + if (VARATT_IS_EXTERNAL(attr)) + { + /* fast path */ + return (toast_fetch_datum_slice(attr, sliceoffset, slicelength)); + } + else + { + preslice = attr; + } + } + + /* slicing of datum for compressed cases and plain value */ + + attrsize = VARSIZE(preslice) - VARHDRSZ; + if (sliceoffset >= attrsize) + { + sliceoffset = 0; + slicelength = 0; + } + + if (((sliceoffset + slicelength) > attrsize) || slicelength < 0) + { + slicelength = attrsize - sliceoffset; + } + + result = (varattrib *) palloc(slicelength + VARHDRSZ); + VARATT_SIZEP(result) = slicelength + VARHDRSZ; + + memcpy(VARDATA(result), VARDATA(preslice) + sliceoffset, slicelength); + + if (preslice != attr) pfree(preslice); + + return result; +} + + /* ---------- * toast_raw_datum_size - * @@ -981,7 +1057,7 @@ toast_fetch_datum(varattrib *attr) VARATT_SIZEP(result) |= VARATT_FLAG_COMPRESSED; /* - * Open the toast relation and it's index + * Open the toast relation and its index */ toastrel = heap_open(attr->va_content.va_external.va_toastrelid, AccessShareLock); @@ -1081,4 +1157,198 @@ toast_fetch_datum(varattrib *attr) return result; } +/* ---------- + * toast_fetch_datum_slice - + * + * Reconstruct a segment of a varattrib from the chunks saved + * in the toast relation + * ---------- + */ +static varattrib * +toast_fetch_datum_slice(varattrib *attr, int32 sliceoffset, int32 length) +{ + Relation toastrel; + Relation toastidx; + ScanKeyData toastkey[3]; + IndexScanDesc toastscan; + HeapTupleData toasttup; + HeapTuple ttup; + TupleDesc toasttupDesc; + RetrieveIndexResult indexRes; + Buffer buffer; + + varattrib *result; + int32 attrsize; + int32 nscankeys; + int32 residx; + int32 nextidx; + int numchunks; + int startchunk; + int endchunk; + int32 startoffset; + int32 endoffset; + int totalchunks; + Pointer chunk; + bool isnull; + int32 chunksize; + int32 chcpystrt; + int32 chcpyend; + + attrsize = attr->va_content.va_external.va_extsize; + totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1; + + if (sliceoffset >= attrsize) + { + sliceoffset = 0; + length = 0; + } + + if (((sliceoffset + length) > attrsize) || length < 0) + { + length = attrsize - sliceoffset; + } + + result = (varattrib *) palloc(length + VARHDRSZ); + VARATT_SIZEP(result) = length + VARHDRSZ; + + if (VARATT_IS_COMPRESSED(attr)) + VARATT_SIZEP(result) |= VARATT_FLAG_COMPRESSED; + + if (length == 0) return (result); /* Can save a lot of work at this point! */ + + startchunk = sliceoffset / TOAST_MAX_CHUNK_SIZE; + endchunk = (sliceoffset + length - 1) / TOAST_MAX_CHUNK_SIZE; + numchunks = (endchunk - startchunk ) + 1; + + startoffset = sliceoffset % TOAST_MAX_CHUNK_SIZE; + endoffset = (sliceoffset + length - 1) % TOAST_MAX_CHUNK_SIZE; + + /* + * Open the toast relation and it's index + */ + toastrel = heap_open(attr->va_content.va_external.va_toastrelid, + AccessShareLock); + toasttupDesc = toastrel->rd_att; + toastidx = index_open(toastrel->rd_rel->reltoastidxid); + + /* + * Setup a scan key to fetch from the index. This is either two keys + * or three depending on the number of chunks. + */ + ScanKeyEntryInitialize(&toastkey[0], + (bits16) 0, + (AttrNumber) 1, + (RegProcedure) F_OIDEQ, + ObjectIdGetDatum(attr->va_content.va_external.va_valueid)); + /* + * Now dependent on number of chunks: + */ + + if (numchunks == 1) + { + ScanKeyEntryInitialize(&toastkey[1], + (bits16) 0, + (AttrNumber) 2, + (RegProcedure) F_INT4EQ, + Int32GetDatum(startchunk)); + nscankeys = 2; + } + else + { + ScanKeyEntryInitialize(&toastkey[1], + (bits16) 0, + (AttrNumber) 2, + (RegProcedure) F_INT4GE, + Int32GetDatum(startchunk)); + ScanKeyEntryInitialize(&toastkey[2], + (bits16) 0, + (AttrNumber) 2, + (RegProcedure) F_INT4LE, + Int32GetDatum(endchunk)); + nscankeys = 3; + } + + /* + * Read the chunks by index + * + * The index is on (valueid, chunkidx) so they will come in order + */ + nextidx = startchunk; + toastscan = index_beginscan(toastidx, false, nscankeys, &toastkey[0]); + while ((indexRes = index_getnext(toastscan, ForwardScanDirection)) != NULL) + { + toasttup.t_self = indexRes->heap_iptr; + heap_fetch(toastrel, SnapshotToast, &toasttup, &buffer, toastscan); + pfree(indexRes); + + if (toasttup.t_data == NULL) + continue; + ttup = &toasttup; + + /* + * Have a chunk, extract the sequence number and the data + */ + residx = DatumGetInt32(heap_getattr(ttup, 2, toasttupDesc, &isnull)); + Assert(!isnull); + chunk = DatumGetPointer(heap_getattr(ttup, 3, toasttupDesc, &isnull)); + Assert(!isnull); + chunksize = VARATT_SIZE(chunk) - VARHDRSZ; + + /* + * Some checks on the data we've found + */ + if ((residx != nextidx) || (residx > endchunk) || (residx < startchunk)) + elog(ERROR, "unexpected chunk number %d (expected %d) for toast value %u", + residx, nextidx, + attr->va_content.va_external.va_valueid); + if (residx < totalchunks - 1) + { + if (chunksize != TOAST_MAX_CHUNK_SIZE) + elog(ERROR, "unexpected chunk size %d in chunk %d for toast value %u", + chunksize, residx, + attr->va_content.va_external.va_valueid); + } + else + { + if ((residx * TOAST_MAX_CHUNK_SIZE + chunksize) != attrsize) + elog(ERROR, "unexpected chunk size %d in chunk %d for toast value %u", + chunksize, residx, + attr->va_content.va_external.va_valueid); + } + + /* + * Copy the data into proper place in our result + */ + chcpystrt = 0; + chcpyend = chunksize - 1; + if (residx == startchunk) chcpystrt = startoffset; + if (residx == endchunk) chcpyend = endoffset; + + memcpy(((char *) VARATT_DATA(result)) + + (residx * TOAST_MAX_CHUNK_SIZE - sliceoffset) +chcpystrt, + VARATT_DATA(chunk) + chcpystrt, + (chcpyend - chcpystrt) + 1); + + ReleaseBuffer(buffer); + nextidx++; + } + + /* + * Final checks that we successfully fetched the datum + */ + if ( nextidx != (endchunk + 1)) + elog(ERROR, "missing chunk number %d for toast value %u", + nextidx, + attr->va_content.va_external.va_valueid); + + /* + * End scan and close relations + */ + index_endscan(toastscan); + index_close(toastidx); + heap_close(toastrel, AccessShareLock); + + return result; +} + #endif /* TUPLE_TOASTER_ACTIVE */ diff --git a/src/backend/commands/command.c b/src/backend/commands/command.c index eefbe269f3..e49c8ca321 100644 --- a/src/backend/commands/command.c +++ b/src/backend/commands/command.c @@ -8,7 +8,7 @@ * * * IDENTIFICATION - * $Header: /cvsroot/pgsql/src/backend/commands/Attic/command.c,v 1.157 2002/03/02 21:39:22 momjian Exp $ + * $Header: /cvsroot/pgsql/src/backend/commands/Attic/command.c,v 1.158 2002/03/05 05:33:08 momjian Exp $ * * NOTES * The PerformAddAttribute() code, like most of the relation @@ -714,20 +714,27 @@ drop_default(Oid relid, int16 attnum) /* - * ALTER TABLE ALTER COLUMN SET STATISTICS + * ALTER TABLE ALTER COLUMN SET STATISTICS / STORAGE */ void -AlterTableAlterColumnStatistics(const char *relationName, +AlterTableAlterColumnFlags(const char *relationName, bool inh, const char *colName, - Node *statsTarget) + Node *flagValue, const char *flagType) { Relation rel; Oid myrelid; - int newtarget; + int newtarget = 1; + char newstorage = 'x'; + char *storagemode; Relation attrelation; HeapTuple tuple; - /* we allow this on system tables */ + /* we allow statistics case for system tables */ + + if (*flagType =='M' && !allowSystemTableMods && IsSystemRelationName(relationName)) + elog(ERROR, "ALTER TABLE: relation \"%s\" is a system catalog", + relationName); + #ifndef NO_SECURITY if (!pg_ownercheck(GetUserId(), relationName, RELNAME)) elog(ERROR, "ALTER TABLE: permission denied"); @@ -742,6 +749,50 @@ AlterTableAlterColumnStatistics(const char *relationName, myrelid = RelationGetRelid(rel); heap_close(rel, NoLock); /* close rel, but keep lock! */ + + /* + * Check the supplied parameters before anything else + */ + if (*flagType == 'S') /* + * STATISTICS + */ + { + Assert(IsA(flagValue, Integer)); + newtarget = intVal(flagValue); + + /* + * Limit target to sane range (should we raise an error instead?) + */ + if (newtarget < 0) + newtarget = 0; + else if (newtarget > 1000) + newtarget = 1000; + } + else if (*flagType == 'M') /* + * STORAGE + */ + { + Assert(IsA(flagValue, Value)); + + storagemode = strVal(flagValue); + if (strcasecmp(storagemode, "plain") == 0) + newstorage = 'p'; + else if (strcasecmp(storagemode, "external") == 0) + newstorage = 'e'; + else if (strcasecmp(storagemode, "extended") == 0) + newstorage = 'x'; + else if (strcasecmp(storagemode, "main") == 0) + newstorage = 'm'; + else + elog(ERROR, "ALTER TABLE: \"%s\" storage not recognized", + storagemode); + } + else + { + elog(ERROR, "ALTER TABLE: Invalid column flag: %c", + (int) *flagType); + } + /* * Propagate to children if desired */ @@ -765,23 +816,14 @@ AlterTableAlterColumnStatistics(const char *relationName, if (childrelid == myrelid) continue; rel = heap_open(childrelid, AccessExclusiveLock); - AlterTableAlterColumnStatistics(RelationGetRelationName(rel), - false, colName, statsTarget); + AlterTableAlterColumnFlags(RelationGetRelationName(rel), + false, colName, flagValue, flagType); heap_close(rel, AccessExclusiveLock); } } /* -= now do the thing on this relation =- */ - Assert(IsA(statsTarget, Integer)); - newtarget = intVal(statsTarget); - - /* Limit target to sane range (should we raise an error instead?) */ - if (newtarget < 0) - newtarget = 0; - else if (newtarget > 1000) - newtarget = 1000; - attrelation = heap_openr(AttributeRelationName, RowExclusiveLock); tuple = SearchSysCacheCopy(ATTNAME, @@ -795,9 +837,22 @@ AlterTableAlterColumnStatistics(const char *relationName, if (((Form_pg_attribute) GETSTRUCT(tuple))->attnum < 0) elog(ERROR, "ALTER TABLE: cannot change system attribute \"%s\"", colName); - - ((Form_pg_attribute) GETSTRUCT(tuple))->attstattarget = newtarget; - + /* + * Now change the appropriate field + */ + if (*flagType == 'S') + ((Form_pg_attribute) GETSTRUCT(tuple))->attstattarget = newtarget; + else + { + if ((newstorage == 'p') || + (((Form_pg_attribute) GETSTRUCT(tuple))->attlen == -1)) + ((Form_pg_attribute) GETSTRUCT(tuple))->attstorage = newstorage; + else + { + elog(ERROR, + "ALTER TABLE: Fixed-length columns can only have storage \"plain\""); + } + } simple_heap_update(attrelation, &tuple->t_self, tuple); /* keep system catalog indices current */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 9ff44e9d93..dfc8898653 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -11,7 +11,7 @@ * * * IDENTIFICATION - * $Header: /cvsroot/pgsql/src/backend/parser/gram.y,v 2.283 2002/03/02 21:39:27 momjian Exp $ + * $Header: /cvsroot/pgsql/src/backend/parser/gram.y,v 2.284 2002/03/05 05:33:14 momjian Exp $ * * HISTORY * AUTHOR DATE MAJOR EVENT @@ -364,7 +364,7 @@ static void doNegateFloat(Value *v); OFFSET, OIDS, OPERATOR, OWNER, PASSWORD, PROCEDURAL, REINDEX, RENAME, RESET, RETURNS, ROW, RULE, SEQUENCE, SETOF, SHARE, SHOW, START, STATEMENT, - STATISTICS, STDIN, STDOUT, SYSID, + STATISTICS, STDIN, STDOUT, STORAGE, SYSID, TEMP, TEMPLATE, TOAST, TRUNCATE, TRUSTED, UNLISTEN, UNTIL, VACUUM, VALID, VERBOSE, VERSION @@ -1117,6 +1117,17 @@ AlterTableStmt: n->def = (Node *) makeInteger($9); $$ = (Node *)n; } +/* ALTER TABLE ALTER [COLUMN] SET STORAGE */ + | ALTER TABLE relation_expr ALTER opt_column ColId SET STORAGE ColId + { + AlterTableStmt *n = makeNode(AlterTableStmt); + n->subtype = 'M'; + n->relname = $3->relname; + n->inhOpt = $3->inhOpt; + n->name = $6; + n->def = (Node *) makeString($9); + $$ = (Node *)n; + } /* ALTER TABLE DROP [COLUMN] {RESTRICT|CASCADE} */ | ALTER TABLE relation_expr DROP opt_column ColId drop_behavior { @@ -5959,6 +5970,7 @@ unreserved_keyword: | STATISTICS { $$ = "statistics"; } | STDIN { $$ = "stdin"; } | STDOUT { $$ = "stdout"; } + | STORAGE { $$ = "storage"; } | SYSID { $$ = "sysid"; } | TEMP { $$ = "temp"; } | TEMPLATE { $$ = "template"; } diff --git a/src/backend/parser/keywords.c b/src/backend/parser/keywords.c index 12b05c6bcd..eaa0f7fa2b 100644 --- a/src/backend/parser/keywords.c +++ b/src/backend/parser/keywords.c @@ -8,7 +8,7 @@ * * * IDENTIFICATION - * $Header: /cvsroot/pgsql/src/backend/parser/keywords.c,v 1.100 2002/02/18 23:11:18 petere Exp $ + * $Header: /cvsroot/pgsql/src/backend/parser/keywords.c,v 1.101 2002/03/05 05:33:15 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -243,6 +243,7 @@ static ScanKeyword ScanKeywords[] = { {"statistics", STATISTICS}, {"stdin", STDIN}, {"stdout", STDOUT}, + {"storage", STORAGE}, {"substring", SUBSTRING}, {"sysid", SYSID}, {"table", TABLE}, diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c index 4e2b89508e..528b93012c 100644 --- a/src/backend/tcop/utility.c +++ b/src/backend/tcop/utility.c @@ -10,7 +10,7 @@ * * * IDENTIFICATION - * $Header: /cvsroot/pgsql/src/backend/tcop/utility.c,v 1.128 2002/03/01 22:45:14 petere Exp $ + * $Header: /cvsroot/pgsql/src/backend/tcop/utility.c,v 1.129 2002/03/05 05:33:19 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -425,10 +425,12 @@ ProcessUtility(Node *parsetree, stmt->def); break; case 'S': /* ALTER COLUMN STATISTICS */ - AlterTableAlterColumnStatistics(stmt->relname, + case 'M': /* ALTER COLUMN STORAGE */ + AlterTableAlterColumnFlags(stmt->relname, interpretInhOption(stmt->inhOpt), stmt->name, - stmt->def); + stmt->def, + &(stmt->subtype)); break; case 'D': /* DROP COLUMN */ AlterTableDropColumn(stmt->relname, diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 7e55bfa4ba..b10ec7d685 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -8,7 +8,7 @@ * * * IDENTIFICATION - * $Header: /cvsroot/pgsql/src/backend/utils/adt/varlena.c,v 1.78 2001/11/19 19:15:07 momjian Exp $ + * $Header: /cvsroot/pgsql/src/backend/utils/adt/varlena.c,v 1.79 2002/03/05 05:33:19 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -332,49 +332,71 @@ textcat(PG_FUNCTION_ARGS) * Changed behavior if starting position is less than one to conform to SQL92 behavior. * Formerly returned the entire string; now returns a portion. * - Thomas Lockhart 1998-12-10 + * Now uses faster TOAST-slicing interface + * - John Gray 2002-02-22 */ Datum text_substr(PG_FUNCTION_ARGS) { - text *string = PG_GETARG_TEXT_P(0); + text *string; int32 m = PG_GETARG_INT32(1); int32 n = PG_GETARG_INT32(2); - text *ret; - int len; - + int32 sm; + int32 sn; + int eml = 1; #ifdef MULTIBYTE int i; + int len; + text *ret; char *p; -#endif - - len = VARSIZE(string) - VARHDRSZ; -#ifdef MULTIBYTE - len = pg_mbstrlen_with_len(VARDATA(string), len); -#endif - - /* starting position after the end of the string? */ - if (m > len) - { - m = 1; - n = 0; - } +#endif /* * starting position before the start of the string? then offset into * the string per SQL92 spec... */ - else if (m < 1) + if (m < 1) { n += (m - 1); m = 1; } + /* Check for m > octet length is made in TOAST access routine */ /* m will now become a zero-based starting position */ + sm = m - 1; + sn = n; + +#ifdef MULTIBYTE + eml = pg_database_encoding_max_length (); + + if (eml > 1) + { + sm = 0; + sn = (m + n) * eml + 3; /* +3 to avoid mb characters overhanging slice end */ + } +#endif + + string = PG_GETARG_TEXT_P_SLICE (0, sm, sn); + + if (eml == 1) + { + PG_RETURN_TEXT_P (string); + } +#ifndef MULTIBYTE + PG_RETURN_NULL(); /* notreached: suppress compiler warning */ +#endif +#ifdef MULTIBYTE + len = pg_mbstrlen_with_len (VARDATA (string), sn - 3); + + if (m > len) + { + m = 1; + n = 0; + } m--; if (((m + n) > len) || (n < 0)) n = (len - m); -#ifdef MULTIBYTE p = VARDATA(string); for (i = 0; i < m; i++) p += pg_mblen(p); @@ -382,7 +404,6 @@ text_substr(PG_FUNCTION_ARGS) for (i = 0; i < n; i++) p += pg_mblen(p); n = p - (VARDATA(string) + m); -#endif ret = (text *) palloc(VARHDRSZ + n); VARATT_SIZEP(ret) = VARHDRSZ + n; @@ -390,6 +411,7 @@ text_substr(PG_FUNCTION_ARGS) memcpy(VARDATA(ret), VARDATA(string) + m, n); PG_RETURN_TEXT_P(ret); +#endif } /* @@ -740,26 +762,14 @@ byteacat(PG_FUNCTION_ARGS) Datum bytea_substr(PG_FUNCTION_ARGS) { - bytea *string = PG_GETARG_BYTEA_P(0); int32 m = PG_GETARG_INT32(1); int32 n = PG_GETARG_INT32(2); - bytea *ret; - int len; - - len = VARSIZE(string) - VARHDRSZ; - - /* starting position after the end of the string? */ - if (m > len) - { - m = 1; - n = 0; - } /* * starting position before the start of the string? then offset into * the string per SQL92 spec... */ - else if (m < 1) + if (m < 1) { n += (m - 1); m = 1; @@ -767,15 +777,8 @@ bytea_substr(PG_FUNCTION_ARGS) /* m will now become a zero-based starting position */ m--; - if (((m + n) > len) || (n < 0)) - n = (len - m); - - ret = (bytea *) palloc(VARHDRSZ + n); - VARATT_SIZEP(ret) = VARHDRSZ + n; - - memcpy(VARDATA(ret), VARDATA(string) + m, n); - PG_RETURN_BYTEA_P(ret); + PG_RETURN_BYTEA_P(PG_GETARG_BYTEA_P_SLICE (0, m, n)); } /* diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c index 6fffd7bab8..64988a2077 100644 --- a/src/backend/utils/fmgr/fmgr.c +++ b/src/backend/utils/fmgr/fmgr.c @@ -8,7 +8,7 @@ * * * IDENTIFICATION - * $Header: /cvsroot/pgsql/src/backend/utils/fmgr/fmgr.c,v 1.57 2001/11/05 17:46:30 momjian Exp $ + * $Header: /cvsroot/pgsql/src/backend/utils/fmgr/fmgr.c,v 1.58 2002/03/05 05:33:20 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -1520,3 +1520,10 @@ pg_detoast_datum_copy(struct varlena * datum) return result; } } + +struct varlena * +pg_detoast_datum_slice(struct varlena * datum, int32 first, int32 count) +{ + /* Only get the specified portion from the toast rel */ + return (struct varlena *) heap_tuple_untoast_attr_slice((varattrib *) datum, first, count); +} diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h index 7bc55dc53a..95a1fe8e00 100644 --- a/src/include/access/tuptoaster.h +++ b/src/include/access/tuptoaster.h @@ -6,7 +6,7 @@ * * Copyright (c) 2000, PostgreSQL Development Team * - * $Id: tuptoaster.h,v 1.13 2001/11/05 17:46:31 momjian Exp $ + * $Id: tuptoaster.h,v 1.14 2002/03/05 05:33:25 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -99,6 +99,17 @@ extern varattrib *heap_tuple_fetch_attr(varattrib *attr); */ extern varattrib *heap_tuple_untoast_attr(varattrib *attr); +/* ---------- + * heap_tuple_untoast_attr_slice() - + * + * Fetches only the specified portion of an attribute. + * (Handles all cases for attribute storage) + * ---------- + */ +extern varattrib *heap_tuple_untoast_attr_slice(varattrib *attr, + int32 sliceoffset, + int32 slicelength); + /* ---------- * toast_compress_datum - * diff --git a/src/include/commands/command.h b/src/include/commands/command.h index ee4e2c0aa3..cf09111af4 100644 --- a/src/include/commands/command.h +++ b/src/include/commands/command.h @@ -7,7 +7,7 @@ * Portions Copyright (c) 1996-2001, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * - * $Id: command.h,v 1.32 2002/02/26 22:47:10 tgl Exp $ + * $Id: command.h,v 1.33 2002/03/05 05:33:29 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -47,9 +47,9 @@ extern void AlterTableAlterColumnDefault(const char *relationName, bool inh, const char *colName, Node *newDefault); -extern void AlterTableAlterColumnStatistics(const char *relationName, +extern void AlterTableAlterColumnFlags(const char *relationName, bool inh, const char *colName, - Node *statsTarget); + Node *flagValue, const char *flagType); extern void AlterTableDropColumn(const char *relationName, bool inh, const char *colName, diff --git a/src/include/fmgr.h b/src/include/fmgr.h index 51adf1c5cd..017f73fb75 100644 --- a/src/include/fmgr.h +++ b/src/include/fmgr.h @@ -11,7 +11,7 @@ * Portions Copyright (c) 1996-2001, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * - * $Id: fmgr.h,v 1.18 2001/11/05 17:46:31 momjian Exp $ + * $Id: fmgr.h,v 1.19 2002/03/05 05:33:22 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -135,11 +135,16 @@ extern void fmgr_info_copy(FmgrInfo *dstinfo, FmgrInfo *srcinfo, */ extern struct varlena *pg_detoast_datum(struct varlena * datum); extern struct varlena *pg_detoast_datum_copy(struct varlena * datum); +extern struct varlena *pg_detoast_datum_slice(struct varlena * datum, + int32 first, int32 count); #define PG_DETOAST_DATUM(datum) \ pg_detoast_datum((struct varlena *) DatumGetPointer(datum)) #define PG_DETOAST_DATUM_COPY(datum) \ pg_detoast_datum_copy((struct varlena *) DatumGetPointer(datum)) +#define PG_DETOAST_DATUM_SLICE(datum,f,c) \ + pg_detoast_datum_slice((struct varlena *) DatumGetPointer(datum), \ + (int32) f, (int32) c) /* * Support for cleaning up detoasted copies of inputs. This must only @@ -187,6 +192,11 @@ extern struct varlena *pg_detoast_datum_copy(struct varlena * datum); #define DatumGetTextPCopy(X) ((text *) PG_DETOAST_DATUM_COPY(X)) #define DatumGetBpCharPCopy(X) ((BpChar *) PG_DETOAST_DATUM_COPY(X)) #define DatumGetVarCharPCopy(X) ((VarChar *) PG_DETOAST_DATUM_COPY(X)) +/* Variants which return n bytes starting at pos. m */ +#define DatumGetByteaPSlice(X,m,n) ((bytea *) PG_DETOAST_DATUM_SLICE(X,m,n)) +#define DatumGetTextPSlice(X,m,n) ((text *) PG_DETOAST_DATUM_SLICE(X,m,n)) +#define DatumGetBpCharPSlice(X,m,n) ((BpChar *) PG_DETOAST_DATUM_SLICE(X,m,n)) +#define DatumGetVarCharPSlice(X,m,n) ((VarChar *) PG_DETOAST_DATUM_SLICE(X,m,n)) /* GETARG macros for varlena types will typically look like this: */ #define PG_GETARG_BYTEA_P(n) DatumGetByteaP(PG_GETARG_DATUM(n)) #define PG_GETARG_TEXT_P(n) DatumGetTextP(PG_GETARG_DATUM(n)) @@ -197,6 +207,11 @@ extern struct varlena *pg_detoast_datum_copy(struct varlena * datum); #define PG_GETARG_TEXT_P_COPY(n) DatumGetTextPCopy(PG_GETARG_DATUM(n)) #define PG_GETARG_BPCHAR_P_COPY(n) DatumGetBpCharPCopy(PG_GETARG_DATUM(n)) #define PG_GETARG_VARCHAR_P_COPY(n) DatumGetVarCharPCopy(PG_GETARG_DATUM(n)) +/* And a b-byte slice from position a -also OK to write */ +#define PG_GETARG_BYTEA_P_SLICE(n,a,b) DatumGetByteaPSlice(PG_GETARG_DATUM(n),a,b) +#define PG_GETARG_TEXT_P_SLICE(n,a,b) DatumGetTextPSlice(PG_GETARG_DATUM(n),a,b) +#define PG_GETARG_BPCHAR_P_SLICE(n,a,b) DatumGetBpCharPSlice(PG_GETARG_DATUM(n),a,b) +#define PG_GETARG_VARCHAR_P_SLICE(n,a,b) DatumGetVarCharPSlice(PG_GETARG_DATUM(n),a,b) /* To return a NULL do this: */ #define PG_RETURN_NULL() \ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 564985ae0e..c6b1feb79b 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -7,7 +7,7 @@ * Portions Copyright (c) 1996-2001, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * - * $Id: parsenodes.h,v 1.155 2002/03/01 22:45:18 petere Exp $ + * $Id: parsenodes.h,v 1.156 2002/03/05 05:33:31 momjian Exp $ * *------------------------------------------------------------------------- */ @@ -123,6 +123,7 @@ typedef struct AlterTableStmt * A = add column * T = alter column default * S = alter column statistics + * M = alter column storage * D = drop column * C = add constraint * X = drop constraint