From: Tom Lane Date: Sun, 20 Feb 2011 05:17:18 +0000 (-0500) Subject: Implement an API to let foreign-data wrappers actually be functional. X-Git-Tag: REL9_1_ALPHA4~126 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=bb742407947ad1cbf19355d24282380d576e7654;p=postgresql Implement an API to let foreign-data wrappers actually be functional. This commit provides the core code and documentation needed. A contrib module test case will follow shortly. Shigeru Hanada, Jan Urbanski, Heikki Linnakangas --- diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml index a65b4bcd33..12f7c3706e 100644 --- a/doc/src/sgml/ddl.sgml +++ b/doc/src/sgml/ddl.sgml @@ -2986,6 +2986,53 @@ ANALYZE measurement; + + Foreign Data + + + foreign data + + + foreign table + + + + PostgreSQL implements portions of the SQL/MED + specification, allowing you to access data that resides outside + PostgreSQL using regular SQL queries. Such data is referred to as + foreign data. (Note that this usage is not to be confused + with foreign keys, which are a type of constraint within the database.) + + + + Foreign data is accessed with help from a + foreign data wrapper. A foreign data wrapper is a + library that can communicate with an external data source, hiding the + details of connecting to the data source and fetching data from it. There + are several foreign data wrappers available, which can for example read + plain data files residing on the server, or connect to another PostgreSQL + instance. If none of the existing foreign data wrappers suit your needs, + you can write your own; see . + + + + To access foreign data, you need to create a foreign server + object, which defines how to connect to a particular external data source, + according to the set of options used by a particular foreign data + wrapper. Then you need to create one or more foreign + tables, which define the structure of the remote data. A + foreign table can be used in queries just like a normal table, but a + foreign table has no storage in the PostgreSQL server. Whenever it is + used, PostgreSQL asks the foreign data wrapper to fetch the data from the + external source. + + + + Currently, foreign tables are read-only. This limitation may be fixed + in a future release. + + + Other Database Objects diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml new file mode 100644 index 0000000000..fc07f129b7 --- /dev/null +++ b/doc/src/sgml/fdwhandler.sgml @@ -0,0 +1,212 @@ + + + + Writing A Foreign Data Wrapper + + + foreign data wrapper + handler for + + + + All operations on a foreign table are handled through its foreign data + wrapper, which consists of a set of functions that the planner and + executor call. The foreign data wrapper is responsible for fetching + data from the remote data source and returning it to the + PostgreSQL executor. This chapter outlines how + to write a new foreign data wrapper. + + + + The FDW author needs to implement a handler function, and optionally + a validator function. Both functions must be written in a compiled + language such as C, using the version-1 interface. + For details on C language calling conventions and dynamic loading, + see . + + + + The handler function simply returns a struct of function pointers to + callback functions that will be called by the planner and executor. + Most of the effort in writing an FDW is in implementing these callback + functions. + The handler function must be registered with + PostgreSQL as taking no arguments and returning + the special pseudo-type fdw_handler. + The callback functions are plain C functions and are not visible or + callable at the SQL level. + + + + The validator function is responsible for validating options given in the + CREATE FOREIGN DATA WRAPPER, CREATE + SERVER and CREATE FOREIGN TABLE commands. + The validator function must be registered as taking two arguments, a text + array containing the options to be validated, and an OID representing the + type of object the options are associated with (in the form of the OID + of the system catalog the object would be stored in). If no validator + function is supplied, the options are not checked at object creation time. + + + + The foreign data wrappers included in the standard distribution are good + references when trying to write your own. Look into the + contrib/file_fdw subdirectory of the source tree. + The reference page also has + some useful details. + + + + + The SQL standard specifies an interface for writing foreign data wrappers. + However, PostgreSQL does not implement that API, because the effort to + accommodate it into PostgreSQL would be large, and the standard API hasn't + gained wide adoption anyway. + + + + + Foreign Data Wrapper Callback Routines + + + The FDW handler function returns a palloc'd FdwRoutine + struct containing pointers to the following callback functions: + + + + +FdwPlan * +PlanForeignScan (Oid foreigntableid, + PlannerInfo *root, + RelOptInfo *baserel); + + + Plan a scan on a foreign table. This is called when a query is planned. + foreigntableid is the pg_class OID of the + foreign table. root is the planner's global information + about the query, and baserel is the planner's information + about this table. + The function must return a palloc'd struct that contains cost estimates + plus any FDW-private information that is needed to execute the foreign + scan at a later time. (Note that the private information must be + represented in a form that copyObject knows how to copy.) + + + + The information in root and baserel can be used + to reduce the amount of information that has to be fetched from the + foreign table (and therefore reduce the cost estimate). + baserel->baserestrictinfo is particularly interesting, as + it contains restriction quals (WHERE clauses) that can be + used to filter the rows to be fetched. (The FDW is not required to + enforce these quals, as the finished plan will recheck them anyway.) + baserel->reltargetlist can be used to determine which + columns need to be fetched. + + + + In addition to returning cost estimates, the function should update + baserel->rows to be the expected number of rows returned + by the scan, after accounting for the filtering done by the restriction + quals. The initial value of baserel->rows is just a + constant default estimate, which should be replaced if at all possible. + The function may also choose to update baserel->width if + it can compute a better estimate of the average result row width. + + + + +void +ExplainForeignScan (ForeignScanState *node, + ExplainState *es); + + + Print additional EXPLAIN output for a foreign table scan. + This can just return if there is no need to print anything. + Otherwise, it should call ExplainPropertyText and + related functions to add fields to the EXPLAIN output. + The flag fields in es can be used to determine what to + print, and the state of the ForeignScanState node + can be inspected to provide runtime statistics in the EXPLAIN + ANALYZE case. + + + + +void +BeginForeignScan (ForeignScanState *node, + int eflags); + + + Begin executing a foreign scan. This is called during executor startup. + It should perform any initialization needed before the scan can start. + The ForeignScanState node has already been created, but + its fdw_state field is still NULL. Information about + the table to scan is accessible through the + ForeignScanState node (in particular, from the underlying + ForeignScan plan node, which contains a pointer to the + FdwPlan structure returned by + PlanForeignScan). + + + + Note that when (eflags & EXEC_FLAG_EXPLAIN_ONLY) is + true, this function should not perform any externally-visible actions; + it should only do the minimum required to make the node state valid + for ExplainForeignScan and EndForeignScan. + + + + +TupleTableSlot * +IterateForeignScan (ForeignScanState *node); + + + Fetch one row from the foreign source, returning it in a tuple table slot + (the node's ScanTupleSlot should be used for this + purpose). Return NULL if no more rows are available. The tuple table + slot infrastructure allows either a physical or virtual tuple to be + returned; in most cases the latter choice is preferable from a + performance standpoint. Note that this is called in a short-lived memory + context that will be reset between invocations. Create a memory context + in BeginForeignScan if you need longer-lived storage, or use + the es_query_cxt of the node's EState. + + + + The rows returned must match the column signature of the foreign table + being scanned. If you choose to optimize away fetching columns that + are not needed, you should insert nulls in those column positions. + + + + +void +ReScanForeignScan (ForeignScanState *node); + + + Restart the scan from the beginning. Note that any parameters the + scan depends on may have changed value, so the new scan does not + necessarily return exactly the same rows. + + + + +void +EndForeignScan (ForeignScanState *node); + + + End the scan and release resources. It is normally not important + to release palloc'd memory, but for example open files and connections + to remote servers should be cleaned up. + + + + The FdwRoutine and FdwPlan struct types + are declared in src/include/foreign/fdwapi.h, which see + for additional details. + + + + + diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index b9d4ea59b1..659bcba7c7 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -86,6 +86,7 @@ + diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml index 4d32f7db25..98d19a5c73 100644 --- a/doc/src/sgml/postgres.sgml +++ b/doc/src/sgml/postgres.sgml @@ -238,6 +238,7 @@ &sources; &nls; &plhandler; + &fdwhandler; &geqo; &indexam; &gist; diff --git a/doc/src/sgml/ref/create_foreign_data_wrapper.sgml b/doc/src/sgml/ref/create_foreign_data_wrapper.sgml index 711f32b118..3093ebcb4a 100644 --- a/doc/src/sgml/ref/create_foreign_data_wrapper.sgml +++ b/doc/src/sgml/ref/create_foreign_data_wrapper.sgml @@ -119,18 +119,13 @@ CREATE FOREIGN DATA WRAPPER name Notes - At the moment, the foreign-data wrapper functionality is very - rudimentary. The purpose of foreign-data wrappers, foreign - servers, and user mappings is to store this information in a - standard way so that it can be queried by interested applications. - One such application is dblink; - see . The functionality to actually query - external data through a foreign-data wrapper library does not exist - yet. + At the moment, the foreign-data wrapper functionality is rudimentary. + There is no support for updating a foreign table, and optimization of + queries is primitive (and mostly left to the wrapper, too). - There is currently one foreign-data wrapper validator function + There is one built-in foreign-data wrapper validator function provided: postgresql_fdw_validator, which accepts options corresponding to libpq connection diff --git a/doc/src/sgml/ref/create_foreign_table.sgml b/doc/src/sgml/ref/create_foreign_table.sgml index ac2e1393e3..77c62140f2 100644 --- a/doc/src/sgml/ref/create_foreign_table.sgml +++ b/doc/src/sgml/ref/create_foreign_table.sgml @@ -131,8 +131,8 @@ CREATE FOREIGN TABLE [ IF NOT EXISTS ] table_name Options to be associated with the new foreign table. The allowed option names and values are specific to each foreign - data wrapper and are validated using the foreign-data wrapper - library. Option names must be unique. + data wrapper and are validated using the foreign-data wrapper's + validator function. Option names must be unique. diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index 65345907d0..ccae1f9d84 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -22,6 +22,7 @@ #include "commands/trigger.h" #include "executor/hashjoin.h" #include "executor/instrument.h" +#include "foreign/fdwapi.h" #include "optimizer/clauses.h" #include "optimizer/planner.h" #include "optimizer/var.h" @@ -80,25 +81,15 @@ static void show_sort_keys_common(PlanState *planstate, List *ancestors, ExplainState *es); static void show_sort_info(SortState *sortstate, ExplainState *es); static void show_hash_info(HashState *hashstate, ExplainState *es); +static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es); static const char *explain_get_index_name(Oid indexId); static void ExplainScanTarget(Scan *plan, ExplainState *es); static void ExplainMemberNodes(List *plans, PlanState **planstates, List *ancestors, ExplainState *es); static void ExplainSubPlans(List *plans, List *ancestors, const char *relationship, ExplainState *es); -static void ExplainPropertyList(const char *qlabel, List *data, - ExplainState *es); static void ExplainProperty(const char *qlabel, const char *value, bool numeric, ExplainState *es); - -#define ExplainPropertyText(qlabel, value, es) \ - ExplainProperty(qlabel, value, false, es) -static void ExplainPropertyInteger(const char *qlabel, int value, - ExplainState *es); -static void ExplainPropertyLong(const char *qlabel, long value, - ExplainState *es); -static void ExplainPropertyFloat(const char *qlabel, double value, int ndigits, - ExplainState *es); static void ExplainOpenGroup(const char *objtype, const char *labelname, bool labeled, ExplainState *es); static void ExplainCloseGroup(const char *objtype, const char *labelname, @@ -705,6 +696,9 @@ ExplainNode(PlanState *planstate, List *ancestors, case T_WorkTableScan: pname = sname = "WorkTable Scan"; break; + case T_ForeignScan: + pname = sname = "Foreign Scan"; + break; case T_Material: pname = sname = "Materialize"; break; @@ -854,6 +848,7 @@ ExplainNode(PlanState *planstate, List *ancestors, case T_ValuesScan: case T_CteScan: case T_WorkTableScan: + case T_ForeignScan: ExplainScanTarget((Scan *) plan, es); break; case T_BitmapIndexScan: @@ -1057,6 +1052,10 @@ ExplainNode(PlanState *planstate, List *ancestors, show_scan_qual(plan->qual, "Filter", planstate, ancestors, es); } break; + case T_ForeignScan: + show_scan_qual(plan->qual, "Filter", planstate, ancestors, es); + show_foreignscan_info((ForeignScanState *) planstate, es); + break; case T_NestLoop: show_upper_qual(((NestLoop *) plan)->join.joinqual, "Join Filter", planstate, ancestors, es); @@ -1523,6 +1522,18 @@ show_hash_info(HashState *hashstate, ExplainState *es) } } +/* + * Show extra information for a ForeignScan node. + */ +static void +show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es) +{ + FdwRoutine *fdwroutine = fsstate->fdwroutine; + + /* Let the FDW emit whatever fields it wants */ + fdwroutine->ExplainForeignScan(fsstate, es); +} + /* * Fetch the name of an index in an EXPLAIN * @@ -1570,6 +1581,7 @@ ExplainScanTarget(Scan *plan, ExplainState *es) case T_IndexScan: case T_BitmapHeapScan: case T_TidScan: + case T_ForeignScan: /* Assert it's on a real relation */ Assert(rte->rtekind == RTE_RELATION); objectname = get_rel_name(rte->relid); @@ -1695,7 +1707,7 @@ ExplainSubPlans(List *plans, List *ancestors, * Explain a property, such as sort keys or targets, that takes the form of * a list of unlabeled items. "data" is a list of C strings. */ -static void +void ExplainPropertyList(const char *qlabel, List *data, ExplainState *es) { ListCell *lc; @@ -1817,10 +1829,19 @@ ExplainProperty(const char *qlabel, const char *value, bool numeric, } } +/* + * Explain a string-valued property. + */ +void +ExplainPropertyText(const char *qlabel, const char *value, ExplainState *es) +{ + ExplainProperty(qlabel, value, false, es); +} + /* * Explain an integer-valued property. */ -static void +void ExplainPropertyInteger(const char *qlabel, int value, ExplainState *es) { char buf[32]; @@ -1832,7 +1853,7 @@ ExplainPropertyInteger(const char *qlabel, int value, ExplainState *es) /* * Explain a long-integer-valued property. */ -static void +void ExplainPropertyLong(const char *qlabel, long value, ExplainState *es) { char buf[32]; @@ -1845,7 +1866,7 @@ ExplainPropertyLong(const char *qlabel, long value, ExplainState *es) * Explain a float-valued property, using the specified number of * fractional digits. */ -static void +void ExplainPropertyFloat(const char *qlabel, double value, int ndigits, ExplainState *es) { diff --git a/src/backend/executor/Makefile b/src/backend/executor/Makefile index da4fbb440b..a854c9a5dc 100644 --- a/src/backend/executor/Makefile +++ b/src/backend/executor/Makefile @@ -23,6 +23,6 @@ OBJS = execAmi.o execCurrent.o execGrouping.o execJunk.o execMain.o \ nodeSeqscan.o nodeSetOp.o nodeSort.o nodeUnique.o \ nodeValuesscan.o nodeCtescan.o nodeWorktablescan.o \ nodeGroup.o nodeSubplan.o nodeSubqueryscan.o nodeTidscan.o \ - nodeWindowAgg.o tstoreReceiver.o spi.o + nodeForeignscan.o nodeWindowAgg.o tstoreReceiver.o spi.o include $(top_srcdir)/src/backend/common.mk diff --git a/src/backend/executor/execAmi.c b/src/backend/executor/execAmi.c index 6bdc60c352..01775ce449 100644 --- a/src/backend/executor/execAmi.c +++ b/src/backend/executor/execAmi.c @@ -21,6 +21,7 @@ #include "executor/nodeBitmapIndexscan.h" #include "executor/nodeBitmapOr.h" #include "executor/nodeCtescan.h" +#include "executor/nodeForeignscan.h" #include "executor/nodeFunctionscan.h" #include "executor/nodeGroup.h" #include "executor/nodeGroup.h" @@ -186,6 +187,10 @@ ExecReScan(PlanState *node) ExecReScanWorkTableScan((WorkTableScanState *) node); break; + case T_ForeignScanState: + ExecReScanForeignScan((ForeignScanState *) node); + break; + case T_NestLoopState: ExecReScanNestLoop((NestLoopState *) node); break; diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 7df7f5cdf0..68509fbfc6 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -738,6 +738,13 @@ InitPlan(QueryDesc *queryDesc, int eflags) break; } + /* if foreign table, tuples can't be locked */ + if (relation && relation->rd_rel->relkind == RELKIND_FOREIGN_TABLE) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("SELECT FOR UPDATE/SHARE cannot be used with foreign table \"%s\"", + RelationGetRelationName(relation)))); + erm = (ExecRowMark *) palloc(sizeof(ExecRowMark)); erm->relation = relation; erm->rti = rc->rti; diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index e8ebec1234..17788761d7 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -85,6 +85,7 @@ #include "executor/nodeBitmapIndexscan.h" #include "executor/nodeBitmapOr.h" #include "executor/nodeCtescan.h" +#include "executor/nodeForeignscan.h" #include "executor/nodeFunctionscan.h" #include "executor/nodeGroup.h" #include "executor/nodeHash.h" @@ -232,6 +233,11 @@ ExecInitNode(Plan *node, EState *estate, int eflags) estate, eflags); break; + case T_ForeignScan: + result = (PlanState *) ExecInitForeignScan((ForeignScan *) node, + estate, eflags); + break; + /* * join nodes */ @@ -422,6 +428,10 @@ ExecProcNode(PlanState *node) result = ExecWorkTableScan((WorkTableScanState *) node); break; + case T_ForeignScanState: + result = ExecForeignScan((ForeignScanState *) node); + break; + /* * join nodes */ @@ -650,6 +660,10 @@ ExecEndNode(PlanState *node) ExecEndWorkTableScan((WorkTableScanState *) node); break; + case T_ForeignScanState: + ExecEndForeignScan((ForeignScanState *) node); + break; + /* * join nodes */ diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c new file mode 100644 index 0000000000..c4309a981e --- /dev/null +++ b/src/backend/executor/nodeForeignscan.c @@ -0,0 +1,209 @@ +/*------------------------------------------------------------------------- + * + * nodeForeignscan.c + * Routines to support scans of foreign tables + * + * Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/executor/nodeForeignscan.c + * + *------------------------------------------------------------------------- + */ +/* + * INTERFACE ROUTINES + * + * ExecForeignScan scans a foreign table. + * ExecInitForeignScan creates and initializes state info. + * ExecReScanForeignScan rescans the foreign relation. + * ExecEndForeignScan releases any resources allocated. + */ +#include "postgres.h" + +#include "executor/executor.h" +#include "executor/nodeForeignscan.h" +#include "foreign/fdwapi.h" + +static TupleTableSlot *ForeignNext(ForeignScanState *node); +static bool ForeignRecheck(ForeignScanState *node, TupleTableSlot *slot); + + +/* ---------------------------------------------------------------- + * ForeignNext + * + * This is a workhorse for ExecForeignScan + * ---------------------------------------------------------------- + */ +static TupleTableSlot * +ForeignNext(ForeignScanState *node) +{ + TupleTableSlot *slot; + ForeignScan *plan = (ForeignScan *) node->ss.ps.plan; + ExprContext *econtext = node->ss.ps.ps_ExprContext; + MemoryContext oldcontext; + + /* Call the Iterate function in short-lived context */ + oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); + slot = node->fdwroutine->IterateForeignScan(node); + MemoryContextSwitchTo(oldcontext); + + /* + * If any system columns are requested, we have to force the tuple into + * physical-tuple form to avoid "cannot extract system attribute from + * virtual tuple" errors later. We also insert a valid value for + * tableoid, which is the only actually-useful system column. + */ + if (plan->fsSystemCol && !TupIsNull(slot)) + { + HeapTuple tup = ExecMaterializeSlot(slot); + + tup->t_tableOid = RelationGetRelid(node->ss.ss_currentRelation); + } + + return slot; +} + +/* + * ForeignRecheck -- access method routine to recheck a tuple in EvalPlanQual + */ +static bool +ForeignRecheck(ForeignScanState *node, TupleTableSlot *slot) +{ + /* There are no access-method-specific conditions to recheck. */ + return true; +} + +/* ---------------------------------------------------------------- + * ExecForeignScan(node) + * + * Fetches the next tuple from the FDW, checks local quals, and + * returns it. + * We call the ExecScan() routine and pass it the appropriate + * access method functions. + * ---------------------------------------------------------------- + */ +TupleTableSlot * +ExecForeignScan(ForeignScanState *node) +{ + return ExecScan((ScanState *) node, + (ExecScanAccessMtd) ForeignNext, + (ExecScanRecheckMtd) ForeignRecheck); +} + + +/* ---------------------------------------------------------------- + * ExecInitForeignScan + * ---------------------------------------------------------------- + */ +ForeignScanState * +ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) +{ + ForeignScanState *scanstate; + Relation currentRelation; + FdwRoutine *fdwroutine; + + /* check for unsupported flags */ + Assert(!(eflags & (EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK))); + + /* + * create state structure + */ + scanstate = makeNode(ForeignScanState); + scanstate->ss.ps.plan = (Plan *) node; + scanstate->ss.ps.state = estate; + + /* + * Miscellaneous initialization + * + * create expression context for node + */ + ExecAssignExprContext(estate, &scanstate->ss.ps); + + scanstate->ss.ps.ps_TupFromTlist = false; + + /* + * initialize child expressions + */ + scanstate->ss.ps.targetlist = (List *) + ExecInitExpr((Expr *) node->scan.plan.targetlist, + (PlanState *) scanstate); + scanstate->ss.ps.qual = (List *) + ExecInitExpr((Expr *) node->scan.plan.qual, + (PlanState *) scanstate); + + /* + * tuple table initialization + */ + ExecInitResultTupleSlot(estate, &scanstate->ss.ps); + ExecInitScanTupleSlot(estate, &scanstate->ss); + + /* + * open the base relation and acquire appropriate lock on it. + */ + currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid); + scanstate->ss.ss_currentRelation = currentRelation; + + /* + * get the scan type from the relation descriptor. + */ + ExecAssignScanType(&scanstate->ss, RelationGetDescr(currentRelation)); + + /* + * Initialize result tuple type and projection info. + */ + ExecAssignResultTypeFromTL(&scanstate->ss.ps); + ExecAssignScanProjectionInfo(&scanstate->ss); + + /* + * Acquire function pointers from the FDW's handler, and init fdw_state. + */ + fdwroutine = GetFdwRoutineByRelId(RelationGetRelid(currentRelation)); + scanstate->fdwroutine = fdwroutine; + scanstate->fdw_state = NULL; + + /* + * Tell the FDW to initiate the scan. + */ + fdwroutine->BeginForeignScan(scanstate, eflags); + + return scanstate; +} + +/* ---------------------------------------------------------------- + * ExecEndForeignScan + * + * frees any storage allocated through C routines. + * ---------------------------------------------------------------- + */ +void +ExecEndForeignScan(ForeignScanState *node) +{ + /* Let the FDW shut down */ + node->fdwroutine->EndForeignScan(node); + + /* Free the exprcontext */ + ExecFreeExprContext(&node->ss.ps); + + /* clean out the tuple table */ + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + ExecClearTuple(node->ss.ss_ScanTupleSlot); + + /* close the relation. */ + ExecCloseScanRelation(node->ss.ss_currentRelation); +} + +/* ---------------------------------------------------------------- + * ExecReScanForeignScan + * + * Rescans the relation. + * ---------------------------------------------------------------- + */ +void +ExecReScanForeignScan(ForeignScanState *node) +{ + node->fdwroutine->ReScanForeignScan(node); + + ExecScanReScan(&node->ss); +} diff --git a/src/backend/foreign/foreign.c b/src/backend/foreign/foreign.c index 6e391025ff..44cd18177c 100644 --- a/src/backend/foreign/foreign.c +++ b/src/backend/foreign/foreign.c @@ -16,8 +16,10 @@ #include "catalog/namespace.h" #include "catalog/pg_foreign_data_wrapper.h" #include "catalog/pg_foreign_server.h" +#include "catalog/pg_foreign_table.h" #include "catalog/pg_type.h" #include "catalog/pg_user_mapping.h" +#include "foreign/fdwapi.h" #include "foreign/foreign.h" #include "funcapi.h" #include "miscadmin.h" @@ -54,19 +56,22 @@ GetForeignDataWrapper(Oid fdwid) fdwform = (Form_pg_foreign_data_wrapper) GETSTRUCT(tp); - fdw = palloc(sizeof(ForeignDataWrapper)); + fdw = (ForeignDataWrapper *) palloc(sizeof(ForeignDataWrapper)); fdw->fdwid = fdwid; fdw->owner = fdwform->fdwowner; fdw->fdwname = pstrdup(NameStr(fdwform->fdwname)); fdw->fdwhandler = fdwform->fdwhandler; fdw->fdwvalidator = fdwform->fdwvalidator; - /* Extract the options */ + /* Extract the fdwoptions */ datum = SysCacheGetAttr(FOREIGNDATAWRAPPEROID, tp, Anum_pg_foreign_data_wrapper_fdwoptions, &isnull); - fdw->options = untransformRelOptions(datum); + if (isnull) + fdw->options = NIL; + else + fdw->options = untransformRelOptions(datum); ReleaseSysCache(tp); @@ -88,7 +93,8 @@ GetForeignDataWrapperOidByName(const char *fdwname, bool missing_ok) if (!OidIsValid(fdwId) && !missing_ok) ereport(ERROR, (errcode(ERRCODE_UNDEFINED_OBJECT), - errmsg("foreign-data wrapper \"%s\" does not exist", fdwname))); + errmsg("foreign-data wrapper \"%s\" does not exist", + fdwname))); return fdwId; } @@ -103,7 +109,7 @@ GetForeignDataWrapperByName(const char *fdwname, bool missing_ok) { Oid fdwId = GetForeignDataWrapperOidByName(fdwname, missing_ok); - if (!OidIsValid(fdwId) && missing_ok) + if (!OidIsValid(fdwId)) return NULL; return GetForeignDataWrapper(fdwId); @@ -129,7 +135,7 @@ GetForeignServer(Oid serverid) serverform = (Form_pg_foreign_server) GETSTRUCT(tp); - server = palloc(sizeof(ForeignServer)); + server = (ForeignServer *) palloc(sizeof(ForeignServer)); server->serverid = serverid; server->servername = pstrdup(NameStr(serverform->srvname)); server->owner = serverform->srvowner; @@ -154,9 +160,10 @@ GetForeignServer(Oid serverid) tp, Anum_pg_foreign_server_srvoptions, &isnull); - - /* untransformRelOptions does exactly what we want - avoid duplication */ - server->options = untransformRelOptions(datum); + if (isnull) + server->options = NIL; + else + server->options = untransformRelOptions(datum); ReleaseSysCache(tp); @@ -191,7 +198,7 @@ GetForeignServerByName(const char *srvname, bool missing_ok) { Oid serverid = GetForeignServerOidByName(srvname, missing_ok); - if (!OidIsValid(serverid) && missing_ok) + if (!OidIsValid(serverid)) return NULL; return GetForeignServer(serverid); @@ -233,16 +240,19 @@ GetUserMapping(Oid userid, Oid serverid) umform = (Form_pg_user_mapping) GETSTRUCT(tp); + um = (UserMapping *) palloc(sizeof(UserMapping)); + um->userid = userid; + um->serverid = serverid; + /* Extract the umoptions */ datum = SysCacheGetAttr(USERMAPPINGUSERSERVER, tp, Anum_pg_user_mapping_umoptions, &isnull); - - um = palloc(sizeof(UserMapping)); - um->userid = userid; - um->serverid = serverid; - um->options = untransformRelOptions(datum); + if (isnull) + um->options = NIL; + else + um->options = untransformRelOptions(datum); ReleaseSysCache(tp); @@ -250,6 +260,116 @@ GetUserMapping(Oid userid, Oid serverid) } +/* + * GetForeignTable - look up the foreign table definition by relation oid. + */ +ForeignTable * +GetForeignTable(Oid relid) +{ + Form_pg_foreign_table tableform; + ForeignTable *ft; + HeapTuple tp; + Datum datum; + bool isnull; + + tp = SearchSysCache1(FOREIGNTABLEREL, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for foreign table %u", relid); + tableform = (Form_pg_foreign_table) GETSTRUCT(tp); + + ft = (ForeignTable *) palloc(sizeof(ForeignTable)); + ft->relid = relid; + ft->serverid = tableform->ftserver; + + /* Extract the ftoptions */ + datum = SysCacheGetAttr(FOREIGNTABLEREL, + tp, + Anum_pg_foreign_table_ftoptions, + &isnull); + if (isnull) + ft->options = NIL; + else + ft->options = untransformRelOptions(datum); + + ReleaseSysCache(tp); + + return ft; +} + + +/* + * GetFdwRoutine - call the specified foreign-data wrapper handler routine + * to get its FdwRoutine struct. + */ +FdwRoutine * +GetFdwRoutine(Oid fdwhandler) +{ + Datum datum; + FdwRoutine *routine; + + datum = OidFunctionCall0(fdwhandler); + routine = (FdwRoutine *) DatumGetPointer(datum); + + if (routine == NULL || !IsA(routine, FdwRoutine)) + elog(ERROR, "foreign-data wrapper handler function %u did not return an FdwRoutine struct", + fdwhandler); + + return routine; +} + + +/* + * GetFdwRoutineByRelId - look up the handler of the foreign-data wrapper + * for the given foreign table, and retrieve its FdwRoutine struct. + */ +FdwRoutine * +GetFdwRoutineByRelId(Oid relid) +{ + HeapTuple tp; + Form_pg_foreign_data_wrapper fdwform; + Form_pg_foreign_server serverform; + Form_pg_foreign_table tableform; + Oid serverid; + Oid fdwid; + Oid fdwhandler; + + /* Get server OID for the foreign table. */ + tp = SearchSysCache1(FOREIGNTABLEREL, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for foreign table %u", relid); + tableform = (Form_pg_foreign_table) GETSTRUCT(tp); + serverid = tableform->ftserver; + ReleaseSysCache(tp); + + /* Get foreign-data wrapper OID for the server. */ + tp = SearchSysCache1(FOREIGNSERVEROID, ObjectIdGetDatum(serverid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for foreign server %u", serverid); + serverform = (Form_pg_foreign_server) GETSTRUCT(tp); + fdwid = serverform->srvfdw; + ReleaseSysCache(tp); + + /* Get handler function OID for the FDW. */ + tp = SearchSysCache1(FOREIGNDATAWRAPPEROID, ObjectIdGetDatum(fdwid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for foreign-data wrapper %u", fdwid); + fdwform = (Form_pg_foreign_data_wrapper) GETSTRUCT(tp); + fdwhandler = fdwform->fdwhandler; + + /* Complain if FDW has been set to NO HANDLER. */ + if (!OidIsValid(fdwhandler)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("foreign-data wrapper \"%s\" has no handler", + NameStr(fdwform->fdwname)))); + + ReleaseSysCache(tp); + + /* And finally, call the handler function. */ + return GetFdwRoutine(fdwhandler); +} + + /* * deflist_to_tuplestore - Helper function to convert DefElem list to * tuplestore usable in SRF. @@ -261,7 +381,7 @@ deflist_to_tuplestore(ReturnSetInfo *rsinfo, List *options) TupleDesc tupdesc; Tuplestorestate *tupstore; Datum values[2]; - bool nulls[2] = {0}; + bool nulls[2]; MemoryContext per_query_ctx; MemoryContext oldcontext; @@ -294,6 +414,7 @@ deflist_to_tuplestore(ReturnSetInfo *rsinfo, List *options) values[0] = CStringGetTextDatum(def->defname); values[1] = CStringGetTextDatum(((Value *) def->arg)->val.str); + nulls[0] = nulls[1] = false; tuplestore_putvalues(tupstore, tupdesc, values, nulls); } @@ -313,7 +434,8 @@ pg_options_to_table(PG_FUNCTION_ARGS) { Datum array = PG_GETARG_DATUM(0); - deflist_to_tuplestore((ReturnSetInfo *) fcinfo->resultinfo, untransformRelOptions(array)); + deflist_to_tuplestore((ReturnSetInfo *) fcinfo->resultinfo, + untransformRelOptions(array)); return (Datum) 0; } @@ -407,7 +529,8 @@ postgresql_fdw_validator(PG_FUNCTION_ARGS) ereport(ERROR, (errcode(ERRCODE_SYNTAX_ERROR), errmsg("invalid option \"%s\"", def->defname), - errhint("Valid options in this context are: %s", buf.data))); + errhint("Valid options in this context are: %s", + buf.data))); PG_RETURN_BOOL(false); } diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index 9a8a97c4be..1ba746bb4d 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -23,6 +23,7 @@ #include "postgres.h" #include "miscadmin.h" +#include "foreign/fdwapi.h" #include "nodes/plannodes.h" #include "nodes/relation.h" #include "utils/datum.h" @@ -549,6 +550,43 @@ _copyWorkTableScan(WorkTableScan *from) return newnode; } +/* + * _copyForeignScan + */ +static ForeignScan * +_copyForeignScan(ForeignScan *from) +{ + ForeignScan *newnode = makeNode(ForeignScan); + + /* + * copy node superclass fields + */ + CopyScanFields((Scan *) from, (Scan *) newnode); + + /* + * copy remainder of node + */ + COPY_SCALAR_FIELD(fsSystemCol); + COPY_NODE_FIELD(fdwplan); + + return newnode; +} + +/* + * _copyFdwPlan + */ +static FdwPlan * +_copyFdwPlan(FdwPlan *from) +{ + FdwPlan *newnode = makeNode(FdwPlan); + + COPY_SCALAR_FIELD(startup_cost); + COPY_SCALAR_FIELD(total_cost); + COPY_NODE_FIELD(fdw_private); + + return newnode; +} + /* * CopyJoinFields * @@ -3839,6 +3877,12 @@ copyObject(void *from) case T_WorkTableScan: retval = _copyWorkTableScan(from); break; + case T_ForeignScan: + retval = _copyForeignScan(from); + break; + case T_FdwPlan: + retval = _copyFdwPlan(from); + break; case T_Join: retval = _copyJoin(from); break; diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index 192c042291..10f630e27f 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -24,6 +24,7 @@ #include #include "lib/stringinfo.h" +#include "foreign/fdwapi.h" #include "nodes/plannodes.h" #include "nodes/relation.h" #include "utils/datum.h" @@ -537,6 +538,27 @@ _outWorkTableScan(StringInfo str, WorkTableScan *node) WRITE_INT_FIELD(wtParam); } +static void +_outForeignScan(StringInfo str, ForeignScan *node) +{ + WRITE_NODE_TYPE("FOREIGNSCAN"); + + _outScanInfo(str, (Scan *) node); + + WRITE_BOOL_FIELD(fsSystemCol); + WRITE_NODE_FIELD(fdwplan); +} + +static void +_outFdwPlan(StringInfo str, FdwPlan *node) +{ + WRITE_NODE_TYPE("FDWPLAN"); + + WRITE_FLOAT_FIELD(startup_cost, "%.2f"); + WRITE_FLOAT_FIELD(total_cost, "%.2f"); + WRITE_NODE_FIELD(fdw_private); +} + static void _outJoin(StringInfo str, Join *node) { @@ -1507,6 +1529,16 @@ _outTidPath(StringInfo str, TidPath *node) WRITE_NODE_FIELD(tidquals); } +static void +_outForeignPath(StringInfo str, ForeignPath *node) +{ + WRITE_NODE_TYPE("FOREIGNPATH"); + + _outPathInfo(str, (Path *) node); + + WRITE_NODE_FIELD(fdwplan); +} + static void _outAppendPath(StringInfo str, AppendPath *node) { @@ -2672,6 +2704,12 @@ _outNode(StringInfo str, void *obj) case T_WorkTableScan: _outWorkTableScan(str, obj); break; + case T_ForeignScan: + _outForeignScan(str, obj); + break; + case T_FdwPlan: + _outFdwPlan(str, obj); + break; case T_Join: _outJoin(str, obj); break; @@ -2877,6 +2915,9 @@ _outNode(StringInfo str, void *obj) case T_TidPath: _outTidPath(str, obj); break; + case T_ForeignPath: + _outForeignPath(str, obj); + break; case T_AppendPath: _outAppendPath(str, obj); break; diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README index 825be2cd3a..aaa754cbb1 100644 --- a/src/backend/optimizer/README +++ b/src/backend/optimizer/README @@ -347,6 +347,7 @@ RelOptInfo - a relation or joined relations IndexPath - index scan BitmapHeapPath - top of a bitmapped index scan TidPath - scan by CTID + ForeignPath - scan a foreign table AppendPath - append multiple subpaths together MergeAppendPath - merge multiple subpaths, preserving their common sort order ResultPath - a Result plan node (used for FROM-less SELECT) diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c index 37d83323d6..c835a954ed 100644 --- a/src/backend/optimizer/path/allpaths.c +++ b/src/backend/optimizer/path/allpaths.c @@ -17,6 +17,7 @@ #include +#include "catalog/pg_class.h" #include "nodes/nodeFuncs.h" #ifdef OPTIMIZER_DEBUG #include "nodes/print.h" @@ -34,6 +35,7 @@ #include "parser/parse_clause.h" #include "parser/parsetree.h" #include "rewrite/rewriteManip.h" +#include "utils/lsyscache.h" /* These parameters are set by GUC */ @@ -63,6 +65,8 @@ static void set_cte_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte); static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte); +static void set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, + RangeTblEntry *rte); static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist); static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery, bool *differentTypes); @@ -197,9 +201,17 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, } else { - /* Plain relation */ Assert(rel->rtekind == RTE_RELATION); - set_plain_rel_pathlist(root, rel, rte); + if (get_rel_relkind(rte->relid) == RELKIND_FOREIGN_TABLE) + { + /* Foreign table */ + set_foreign_pathlist(root, rel, rte); + } + else + { + /* Plain relation */ + set_plain_rel_pathlist(root, rel, rte); + } } #ifdef OPTIMIZER_DEBUG @@ -904,6 +916,23 @@ set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte) set_cheapest(rel); } +/* + * set_foreign_pathlist + * Build the (single) access path for a foreign table RTE + */ +static void +set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte) +{ + /* Mark rel with estimated output rows, width, etc */ + set_foreign_size_estimates(root, rel); + + /* Generate appropriate path */ + add_path(rel, (Path *) create_foreignscan_path(root, rel)); + + /* Select cheapest path (pretty easy in this case...) */ + set_cheapest(rel); +} + /* * make_rel_from_joinlist * Build access paths using a "joinlist" to guide the join path search. @@ -1503,6 +1532,9 @@ print_path(PlannerInfo *root, Path *path, int indent) case T_TidPath: ptype = "TidScan"; break; + case T_ForeignPath: + ptype = "ForeignScan"; + break; case T_AppendPath: ptype = "Append"; break; diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c index ffb066283f..c292e24ac3 100644 --- a/src/backend/optimizer/path/costsize.c +++ b/src/backend/optimizer/path/costsize.c @@ -3324,6 +3324,34 @@ set_cte_size_estimates(PlannerInfo *root, RelOptInfo *rel, Plan *cteplan) set_baserel_size_estimates(root, rel); } +/* + * set_foreign_size_estimates + * Set the size estimates for a base relation that is a foreign table. + * + * There is not a whole lot that we can do here; the foreign-data wrapper + * is responsible for producing useful estimates. We can do a decent job + * of estimating baserestrictcost, so we set that, and we also set up width + * using what will be purely datatype-driven estimates from the targetlist. + * There is no way to do anything sane with the rows value, so we just put + * a default estimate and hope that the wrapper can improve on it. The + * wrapper's PlanForeignScan function will be called momentarily. + * + * The rel's targetlist and restrictinfo list must have been constructed + * already. + */ +void +set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel) +{ + /* Should only be applied to base relations */ + Assert(rel->relid > 0); + + rel->rows = 1000; /* entirely bogus default estimate */ + + cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root); + + set_rel_width(root, rel); +} + /* * set_rel_width diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index f01114c673..4895858df6 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -20,6 +20,7 @@ #include #include "access/skey.h" +#include "foreign/fdwapi.h" #include "miscadmin.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -71,6 +72,8 @@ static CteScan *create_ctescan_plan(PlannerInfo *root, Path *best_path, List *tlist, List *scan_clauses); static WorkTableScan *create_worktablescan_plan(PlannerInfo *root, Path *best_path, List *tlist, List *scan_clauses); +static ForeignScan *create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path, + List *tlist, List *scan_clauses); static NestLoop *create_nestloop_plan(PlannerInfo *root, NestPath *best_path, Plan *outer_plan, Plan *inner_plan); static MergeJoin *create_mergejoin_plan(PlannerInfo *root, MergePath *best_path, @@ -112,6 +115,8 @@ static CteScan *make_ctescan(List *qptlist, List *qpqual, Index scanrelid, int ctePlanId, int cteParam); static WorkTableScan *make_worktablescan(List *qptlist, List *qpqual, Index scanrelid, int wtParam); +static ForeignScan *make_foreignscan(List *qptlist, List *qpqual, + Index scanrelid, bool fsSystemCol, FdwPlan *fdwplan); static BitmapAnd *make_bitmap_and(List *bitmapplans); static BitmapOr *make_bitmap_or(List *bitmapplans); static NestLoop *make_nestloop(List *tlist, @@ -206,6 +211,7 @@ create_plan_recurse(PlannerInfo *root, Path *best_path) case T_ValuesScan: case T_CteScan: case T_WorkTableScan: + case T_ForeignScan: plan = create_scan_plan(root, best_path); break; case T_HashJoin: @@ -346,6 +352,13 @@ create_scan_plan(PlannerInfo *root, Path *best_path) scan_clauses); break; + case T_ForeignScan: + plan = (Plan *) create_foreignscan_plan(root, + (ForeignPath *) best_path, + tlist, + scan_clauses); + break; + default: elog(ERROR, "unrecognized node type: %d", (int) best_path->pathtype); @@ -468,6 +481,7 @@ disuse_physical_tlist(Plan *plan, Path *path) case T_ValuesScan: case T_CteScan: case T_WorkTableScan: + case T_ForeignScan: plan->targetlist = build_relation_tlist(path->parent); break; default: @@ -1755,6 +1769,56 @@ create_worktablescan_plan(PlannerInfo *root, Path *best_path, return scan_plan; } +/* + * create_foreignscan_plan + * Returns a foreignscan plan for the base relation scanned by 'best_path' + * with restriction clauses 'scan_clauses' and targetlist 'tlist'. + */ +static ForeignScan * +create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path, + List *tlist, List *scan_clauses) +{ + ForeignScan *scan_plan; + RelOptInfo *rel = best_path->path.parent; + Index scan_relid = rel->relid; + RangeTblEntry *rte; + bool fsSystemCol; + int i; + + /* it should be a base rel... */ + Assert(scan_relid > 0); + Assert(rel->rtekind == RTE_RELATION); + rte = planner_rt_fetch(scan_relid, root); + Assert(rte->rtekind == RTE_RELATION); + + /* Sort clauses into best execution order */ + scan_clauses = order_qual_clauses(root, scan_clauses); + + /* Reduce RestrictInfo list to bare expressions; ignore pseudoconstants */ + scan_clauses = extract_actual_clauses(scan_clauses, false); + + /* Detect whether any system columns are requested from rel */ + fsSystemCol = false; + for (i = rel->min_attr; i < 0; i++) + { + if (!bms_is_empty(rel->attr_needed[i - rel->min_attr])) + { + fsSystemCol = true; + break; + } + } + + scan_plan = make_foreignscan(tlist, + scan_clauses, + scan_relid, + fsSystemCol, + best_path->fdwplan); + + copy_path_costsize(&scan_plan->scan.plan, &best_path->path); + + return scan_plan; +} + /***************************************************************************** * @@ -2978,6 +3042,28 @@ make_worktablescan(List *qptlist, return node; } +static ForeignScan * +make_foreignscan(List *qptlist, + List *qpqual, + Index scanrelid, + bool fsSystemCol, + FdwPlan *fdwplan) +{ + ForeignScan *node = makeNode(ForeignScan); + Plan *plan = &node->scan.plan; + + /* cost should be inserted by caller */ + plan->targetlist = qptlist; + plan->qual = qpqual; + plan->lefttree = NULL; + plan->righttree = NULL; + node->scan.scanrelid = scanrelid; + node->fsSystemCol = fsSystemCol; + node->fdwplan = fdwplan; + + return node; +} + Append * make_append(List *appendplans, List *tlist) { diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 01b90383cf..b73b872b31 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -1914,7 +1914,8 @@ preprocess_rowmarks(PlannerInfo *root) newrc->rti = newrc->prti = i; newrc->rowmarkId = ++(root->glob->lastRowMarkId); /* real tables support REFERENCE, anything else needs COPY */ - if (rte->rtekind == RTE_RELATION) + if (rte->rtekind == RTE_RELATION && + get_rel_relkind(rte->relid) != RELKIND_FOREIGN_TABLE) newrc->markType = ROW_MARK_REFERENCE; else newrc->markType = ROW_MARK_COPY; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index d004f6cf12..b1c181a1cc 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -402,6 +402,18 @@ set_plan_refs(PlannerGlobal *glob, Plan *plan, int rtoffset) fix_scan_list(glob, splan->scan.plan.qual, rtoffset); } break; + case T_ForeignScan: + { + ForeignScan *splan = (ForeignScan *) plan; + + splan->scan.scanrelid += rtoffset; + splan->scan.plan.targetlist = + fix_scan_list(glob, splan->scan.plan.targetlist, rtoffset); + splan->scan.plan.qual = + fix_scan_list(glob, splan->scan.plan.qual, rtoffset); + } + break; + case T_NestLoop: case T_MergeJoin: case T_HashJoin: diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c index 29eb9dced4..96a257f6af 100644 --- a/src/backend/optimizer/plan/subselect.c +++ b/src/backend/optimizer/plan/subselect.c @@ -2054,6 +2054,10 @@ finalize_plan(PlannerInfo *root, Plan *plan, Bitmapset *valid_params, context.paramids = bms_add_members(context.paramids, scan_params); break; + case T_ForeignScan: + context.paramids = bms_add_members(context.paramids, scan_params); + break; + case T_ModifyTable: { ModifyTable *mtplan = (ModifyTable *) plan; diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c index d1010af662..1158f7715b 100644 --- a/src/backend/optimizer/util/pathnode.c +++ b/src/backend/optimizer/util/pathnode.c @@ -17,6 +17,7 @@ #include #include "catalog/pg_operator.h" +#include "foreign/fdwapi.h" #include "miscadmin.h" #include "nodes/nodeFuncs.h" #include "optimizer/clauses.h" @@ -1419,6 +1420,41 @@ create_worktablescan_path(PlannerInfo *root, RelOptInfo *rel) return pathnode; } +/* + * create_foreignscan_path + * Creates a path corresponding to a scan of a foreign table, + * returning the pathnode. + */ +ForeignPath * +create_foreignscan_path(PlannerInfo *root, RelOptInfo *rel) +{ + ForeignPath *pathnode = makeNode(ForeignPath); + RangeTblEntry *rte; + FdwRoutine *fdwroutine; + FdwPlan *fdwplan; + + pathnode->path.pathtype = T_ForeignScan; + pathnode->path.parent = rel; + pathnode->path.pathkeys = NIL; /* result is always unordered */ + + /* Get FDW's callback info */ + rte = planner_rt_fetch(rel->relid, root); + fdwroutine = GetFdwRoutineByRelId(rte->relid); + + /* Let the FDW do its planning */ + fdwplan = fdwroutine->PlanForeignScan(rte->relid, root, rel); + if (fdwplan == NULL || !IsA(fdwplan, FdwPlan)) + elog(ERROR, "foreign-data wrapper PlanForeignScan function for relation %u did not return an FdwPlan struct", + rte->relid); + pathnode->fdwplan = fdwplan; + + /* use costs estimated by FDW */ + pathnode->path.startup_cost = fdwplan->startup_cost; + pathnode->path.total_cost = fdwplan->total_cost; + + return pathnode; +} + /* * create_nestloop_path * Creates a pathnode corresponding to a nestloop join between two diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index 40df626b62..b9c4cbe7f0 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -90,12 +90,6 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, */ relation = heap_open(relationObjectId, NoLock); - /* Foreign table scans are not implemented yet. */ - if (relation->rd_rel->relkind == RELKIND_FOREIGN_TABLE) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - errmsg("foreign table scans are not yet supported"))); - rel->min_attr = FirstLowInvalidHeapAttributeNumber + 1; rel->max_attr = RelationGetNumberOfAttributes(relation); rel->reltablespace = RelationGetForm(relation)->reltablespace; @@ -463,6 +457,11 @@ estimate_rel_size(Relation rel, int32 *attr_widths, *pages = 1; *tuples = 1; break; + case RELKIND_FOREIGN_TABLE: + /* Just use whatever's in pg_class */ + *pages = rel->rd_rel->relpages; + *tuples = rel->rd_rel->reltuples; + break; default: /* else it has no disk storage; probably shouldn't get here? */ *pages = 0; diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c index 22447f92a2..c7a7a23076 100644 --- a/src/backend/parser/analyze.c +++ b/src/backend/parser/analyze.c @@ -40,6 +40,7 @@ #include "parser/parse_target.h" #include "parser/parsetree.h" #include "rewrite/rewriteManip.h" +#include "utils/lsyscache.h" #include "utils/rel.h" @@ -2176,9 +2177,14 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc, switch (rte->rtekind) { case RTE_RELATION: - applyLockingClause(qry, i, - lc->forUpdate, lc->noWait, pushedDown); - rte->requiredPerms |= ACL_SELECT_FOR_UPDATE; + /* ignore foreign tables */ + if (get_rel_relkind(rte->relid) != RELKIND_FOREIGN_TABLE) + { + applyLockingClause(qry, i, + lc->forUpdate, lc->noWait, + pushedDown); + rte->requiredPerms |= ACL_SELECT_FOR_UPDATE; + } break; case RTE_SUBQUERY: applyLockingClause(qry, i, @@ -2225,6 +2231,12 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc, switch (rte->rtekind) { case RTE_RELATION: + if (get_rel_relkind(rte->relid) == RELKIND_FOREIGN_TABLE) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("SELECT FOR UPDATE/SHARE cannot be used with foreign table \"%s\"", + get_rel_name(rte->relid)), + parser_errposition(pstate, thisrel->location))); applyLockingClause(qry, i, lc->forUpdate, lc->noWait, pushedDown); diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index c6491e1519..3a50642fce 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1392,8 +1392,12 @@ markQueryForLocking(Query *qry, Node *jtnode, if (rte->rtekind == RTE_RELATION) { - applyLockingClause(qry, rti, forUpdate, noWait, pushedDown); - rte->requiredPerms |= ACL_SELECT_FOR_UPDATE; + /* ignore foreign tables */ + if (get_rel_relkind(rte->relid) != RELKIND_FOREIGN_TABLE) + { + applyLockingClause(qry, rti, forUpdate, noWait, pushedDown); + rte->requiredPerms |= ACL_SELECT_FOR_UPDATE; + } } else if (rte->rtekind == RTE_SUBQUERY) { diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c index d05e4d2dd8..a115a29da4 100644 --- a/src/backend/utils/fmgr/fmgr.c +++ b/src/backend/utils/fmgr/fmgr.c @@ -1621,6 +1621,26 @@ FunctionCall9(FmgrInfo *flinfo, Datum arg1, Datum arg2, * by FunctionCallN(). If the same function is to be invoked repeatedly, * do the fmgr_info() once and then use FunctionCallN(). */ +Datum +OidFunctionCall0(Oid functionId) +{ + FmgrInfo flinfo; + FunctionCallInfoData fcinfo; + Datum result; + + fmgr_info(functionId, &flinfo); + + InitFunctionCallInfoData(fcinfo, &flinfo, 0, NULL, NULL); + + result = FunctionCallInvoke(&fcinfo); + + /* Check for null result, since caller is clearly not expecting one */ + if (fcinfo.isnull) + elog(ERROR, "function %u returned NULL", flinfo.fn_oid); + + return result; +} + Datum OidFunctionCall1(Oid functionId, Datum arg1) { diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index f17548338f..2c38c92ae5 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -72,4 +72,15 @@ extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); +extern void ExplainPropertyList(const char *qlabel, List *data, + ExplainState *es); +extern void ExplainPropertyText(const char *qlabel, const char *value, + ExplainState *es); +extern void ExplainPropertyInteger(const char *qlabel, int value, + ExplainState *es); +extern void ExplainPropertyLong(const char *qlabel, long value, + ExplainState *es); +extern void ExplainPropertyFloat(const char *qlabel, double value, int ndigits, + ExplainState *es); + #endif /* EXPLAIN_H */ diff --git a/src/include/executor/nodeForeignscan.h b/src/include/executor/nodeForeignscan.h new file mode 100644 index 0000000000..5cfcfe72d0 --- /dev/null +++ b/src/include/executor/nodeForeignscan.h @@ -0,0 +1,24 @@ +/*------------------------------------------------------------------------- + * + * nodeForeignscan.h + * + * + * + * Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/executor/nodeForeignscan.h + * + *------------------------------------------------------------------------- + */ +#ifndef NODEFOREIGNSCAN_H +#define NODEFOREIGNSCAN_H + +#include "nodes/execnodes.h" + +extern ForeignScanState *ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags); +extern TupleTableSlot *ExecForeignScan(ForeignScanState *node); +extern void ExecEndForeignScan(ForeignScanState *node); +extern void ExecReScanForeignScan(ForeignScanState *node); + +#endif /* NODEFOREIGNSCAN_H */ diff --git a/src/include/fmgr.h b/src/include/fmgr.h index 9720117a7f..7539acace8 100644 --- a/src/include/fmgr.h +++ b/src/include/fmgr.h @@ -488,6 +488,7 @@ extern Datum FunctionCall9(FmgrInfo *flinfo, Datum arg1, Datum arg2, * by FunctionCallN(). If the same function is to be invoked repeatedly, * do the FunctionLookup() once and then use FunctionCallN(). */ +extern Datum OidFunctionCall0(Oid functionId); extern Datum OidFunctionCall1(Oid functionId, Datum arg1); extern Datum OidFunctionCall2(Oid functionId, Datum arg1, Datum arg2); extern Datum OidFunctionCall3(Oid functionId, Datum arg1, Datum arg2, diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h new file mode 100644 index 0000000000..2873187388 --- /dev/null +++ b/src/include/foreign/fdwapi.h @@ -0,0 +1,98 @@ +/*------------------------------------------------------------------------- + * + * fdwapi.h + * API for foreign-data wrappers + * + * Copyright (c) 2010-2011, PostgreSQL Global Development Group + * + * src/include/foreign/fdwapi.h + * + *------------------------------------------------------------------------- + */ +#ifndef FDWAPI_H +#define FDWAPI_H + +#include "nodes/execnodes.h" +#include "nodes/relation.h" + +/* To avoid including explain.h here, reference ExplainState thus: */ +struct ExplainState; + + +/* + * FdwPlan is the information returned to the planner by PlanForeignScan. + */ +typedef struct FdwPlan +{ + NodeTag type; + + /* + * Cost estimation info. The startup_cost is time before retrieving + * the first row, so it should include costs of connecting to the remote + * host, sending over the query, etc. Note that PlanForeignScan also + * ought to set baserel->rows and baserel->width if it can produce any + * usable estimates of those values. + */ + Cost startup_cost; /* cost expended before fetching any tuples */ + Cost total_cost; /* total cost (assuming all tuples fetched) */ + + /* + * FDW private data, which will be available at execution time. + * + * Note that everything in this list must be copiable by copyObject(). + * One way to store an arbitrary blob of bytes is to represent it as a + * bytea Const. Usually, though, you'll be better off choosing a + * representation that can be dumped usefully by nodeToString(). + */ + List *fdw_private; +} FdwPlan; + + +/* + * Callback function signatures --- see fdwhandler.sgml for more info. + */ + +typedef FdwPlan * (*PlanForeignScan_function) (Oid foreigntableid, + PlannerInfo *root, + RelOptInfo *baserel); + +typedef void (*ExplainForeignScan_function) (ForeignScanState *node, + struct ExplainState *es); + +typedef void (*BeginForeignScan_function) (ForeignScanState *node, + int eflags); + +typedef TupleTableSlot * (*IterateForeignScan_function) (ForeignScanState *node); + +typedef void (*ReScanForeignScan_function) (ForeignScanState *node); + +typedef void (*EndForeignScan_function) (ForeignScanState *node); + + +/* + * FdwRoutine is the struct returned by a foreign-data wrapper's handler + * function. It provides pointers to the callback functions needed by the + * planner and executor. + * + * Currently, all functions must be supplied. Later there may be optional + * additions. It's recommended that the handler initialize the struct with + * makeNode(FdwRoutine) so that all fields are set to zero. + */ +typedef struct FdwRoutine +{ + NodeTag type; + + PlanForeignScan_function PlanForeignScan; + ExplainForeignScan_function ExplainForeignScan; + BeginForeignScan_function BeginForeignScan; + IterateForeignScan_function IterateForeignScan; + ReScanForeignScan_function ReScanForeignScan; + EndForeignScan_function EndForeignScan; +} FdwRoutine; + + +/* Functions in foreign/foreign.c */ +extern FdwRoutine *GetFdwRoutine(Oid fdwhandler); +extern FdwRoutine *GetFdwRoutineByRelId(Oid relid); + +#endif /* FDWAPI_H */ diff --git a/src/include/foreign/foreign.h b/src/include/foreign/foreign.h index 2cf0eaa09c..d676f3fce7 100644 --- a/src/include/foreign/foreign.h +++ b/src/include/foreign/foreign.h @@ -60,6 +60,13 @@ typedef struct UserMapping List *options; /* useoptions as DefElem list */ } UserMapping; +typedef struct ForeignTable +{ + Oid relid; /* relation Oid */ + Oid serverid; /* server Oid */ + List *options; /* ftoptions as DefElem list */ +} ForeignTable; + extern ForeignServer *GetForeignServer(Oid serverid); extern ForeignServer *GetForeignServerByName(const char *name, bool missing_ok); @@ -69,5 +76,6 @@ extern ForeignDataWrapper *GetForeignDataWrapper(Oid fdwid); extern ForeignDataWrapper *GetForeignDataWrapperByName(const char *name, bool missing_ok); extern Oid GetForeignDataWrapperOidByName(const char *name, bool missing_ok); +extern ForeignTable *GetForeignTable(Oid relid); #endif /* FOREIGN_H */ diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 44da70ff60..0a6b829de4 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -1403,6 +1403,20 @@ typedef struct WorkTableScanState RecursiveUnionState *rustate; } WorkTableScanState; +/* ---------------- + * ForeignScanState information + * + * ForeignScan nodes are used to scan foreign-data tables. + * ---------------- + */ +typedef struct ForeignScanState +{ + ScanState ss; /* its first field is NodeTag */ + /* use struct pointer to avoid including fdwapi.h here */ + struct FdwRoutine *fdwroutine; + void *fdw_state; /* foreign-data wrapper can keep state here */ +} ForeignScanState; + /* ---------------------------------------------------------------- * Join State Information * ---------------------------------------------------------------- diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h index e0d05748da..cbaf123ee9 100644 --- a/src/include/nodes/nodes.h +++ b/src/include/nodes/nodes.h @@ -60,6 +60,8 @@ typedef enum NodeTag T_ValuesScan, T_CteScan, T_WorkTableScan, + T_ForeignScan, + T_FdwPlan, T_Join, T_NestLoop, T_MergeJoin, @@ -103,6 +105,7 @@ typedef enum NodeTag T_ValuesScanState, T_CteScanState, T_WorkTableScanState, + T_ForeignScanState, T_JoinState, T_NestLoopState, T_MergeJoinState, @@ -217,6 +220,7 @@ typedef enum NodeTag T_MergePath, T_HashPath, T_TidPath, + T_ForeignPath, T_AppendPath, T_MergeAppendPath, T_ResultPath, @@ -409,7 +413,8 @@ typedef enum NodeTag T_ReturnSetInfo, /* in nodes/execnodes.h */ T_WindowObjectData, /* private in nodeWindowAgg.c */ T_TIDBitmap, /* in nodes/tidbitmap.h */ - T_InlineCodeBlock /* in nodes/parsenodes.h */ + T_InlineCodeBlock, /* in nodes/parsenodes.h */ + T_FdwRoutine /* in foreign/fdwapi.h */ } NodeTag; /* diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index b4fb4f211b..90d61256e9 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -436,6 +436,18 @@ typedef struct WorkTableScan int wtParam; /* ID of Param representing work table */ } WorkTableScan; +/* ---------------- + * ForeignScan node + * ---------------- + */ +typedef struct ForeignScan +{ + Scan scan; + bool fsSystemCol; /* true if any "system column" is needed */ + /* use struct pointer to avoid including fdwapi.h here */ + struct FdwPlan *fdwplan; +} ForeignScan; + /* * ========== diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h index 6f51b6c5fe..ab708351ed 100644 --- a/src/include/nodes/relation.h +++ b/src/include/nodes/relation.h @@ -749,6 +749,16 @@ typedef struct TidPath List *tidquals; /* qual(s) involving CTID = something */ } TidPath; +/* + * ForeignPath represents a scan of a foreign table + */ +typedef struct ForeignPath +{ + Path path; + /* use struct pointer to avoid including fdwapi.h here */ + struct FdwPlan *fdwplan; +} ForeignPath; + /* * AppendPath represents an Append plan, ie, successive execution of * several member plans. diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h index 3112bf75cc..e3cf7464df 100644 --- a/src/include/optimizer/cost.h +++ b/src/include/optimizer/cost.h @@ -127,6 +127,7 @@ extern void set_function_size_estimates(PlannerInfo *root, RelOptInfo *rel); extern void set_values_size_estimates(PlannerInfo *root, RelOptInfo *rel); extern void set_cte_size_estimates(PlannerInfo *root, RelOptInfo *rel, Plan *cteplan); +extern void set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel); /* * prototypes for clausesel.c diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h index ff220e32a7..7a24da2c51 100644 --- a/src/include/optimizer/pathnode.h +++ b/src/include/optimizer/pathnode.h @@ -61,6 +61,7 @@ extern Path *create_functionscan_path(PlannerInfo *root, RelOptInfo *rel); extern Path *create_valuesscan_path(PlannerInfo *root, RelOptInfo *rel); extern Path *create_ctescan_path(PlannerInfo *root, RelOptInfo *rel); extern Path *create_worktablescan_path(PlannerInfo *root, RelOptInfo *rel); +extern ForeignPath *create_foreignscan_path(PlannerInfo *root, RelOptInfo *rel); extern NestPath *create_nestloop_path(PlannerInfo *root, RelOptInfo *joinrel, diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out index 780098c237..00730f25cb 100644 --- a/src/test/regress/expected/foreign_data.out +++ b/src/test/regress/expected/foreign_data.out @@ -670,9 +670,9 @@ Has OIDs: no CREATE INDEX id_ft1_c2 ON ft1 (c2); -- ERROR ERROR: "ft1" is not a table SELECT * FROM ft1; -- ERROR -ERROR: foreign table scans are not yet supported +ERROR: foreign-data wrapper "dummy" has no handler EXPLAIN SELECT * FROM ft1; -- ERROR -ERROR: foreign table scans are not yet supported +ERROR: foreign-data wrapper "dummy" has no handler -- ALTER FOREIGN TABLE COMMENT ON FOREIGN TABLE ft1 IS 'foreign table'; COMMENT ON FOREIGN TABLE ft1 IS NULL;