1 <!-- doc/src/sgml/plpython.sgml -->
3 <chapter id="plpython">
4 <title>PL/Python - Python Procedural Language</title>
6 <indexterm zone="plpython"><primary>PL/Python</></>
7 <indexterm zone="plpython"><primary>Python</></>
10 The <application>PL/Python</application> procedural language allows
11 <productname>PostgreSQL</productname> functions to be written in the
12 <ulink url="http://www.python.org">Python language</ulink>.
16 To install PL/Python in a particular database, use
17 <literal>CREATE EXTENSION plpythonu</>, or from the shell command line use
18 <literal>createlang plpythonu <replaceable>dbname</></literal> (but
19 see also <xref linkend="plpython-python23">).
24 If a language is installed into <literal>template1</>, all subsequently
25 created databases will have the language installed automatically.
30 PL/Python is only available as an <quote>untrusted</> language, meaning
31 it does not offer any way of restricting what users can do in it and
32 is therefore named <literal>plpythonu</>. A trusted
33 variant <literal>plpython</> might become available in the future
34 if a secure execution mechanism is developed in Python. The
35 writer of a function in untrusted PL/Python must take care that the
36 function cannot be used to do anything unwanted, since it will be
37 able to do anything that could be done by a user logged in as the
38 database administrator. Only superusers can create functions in
39 untrusted languages such as <literal>plpythonu</literal>.
44 Users of source packages must specially enable the build of
45 PL/Python during the installation process. (Refer to the
46 installation instructions for more information.) Users of binary
47 packages might find PL/Python in a separate subpackage.
51 <sect1 id="plpython-python23">
52 <title>Python 2 vs. Python 3</title>
55 PL/Python supports both the Python 2 and Python 3 language
56 variants. (The PostgreSQL installation instructions might contain
57 more precise information about the exact supported minor versions
58 of Python.) Because the Python 2 and Python 3 language variants
59 are incompatible in some important aspects, the following naming
60 and transitioning scheme is used by PL/Python to avoid mixing them:
65 The PostgreSQL language named <literal>plpython2u</literal>
66 implements PL/Python based on the Python 2 language variant.
72 The PostgreSQL language named <literal>plpython3u</literal>
73 implements PL/Python based on the Python 3 language variant.
79 The language named <literal>plpythonu</literal> implements
80 PL/Python based on the default Python language variant, which is
81 currently Python 2. (This default is independent of what any
82 local Python installations might consider to be
83 their <quote>default</quote>, for example,
84 what <filename>/usr/bin/python</filename> might be.) The
85 default will probably be changed to Python 3 in a distant future
86 release of PostgreSQL, depending on the progress of the
87 migration to Python 3 in the Python community.
92 This scheme is analogous to the recommendations in <ulink
93 url="http://www.python.org/dev/peps/pep-0394/">PEP 394</ulink> regarding the
94 naming and transitioning of the <command>python</command> command.
98 It depends on the build configuration or the installed packages
99 whether PL/Python for Python 2 or Python 3 or both are available.
104 The built variant depends on which Python version was found during
105 the installation or which version was explicitly set using
106 the <envar>PYTHON</envar> environment variable;
107 see <xref linkend="install-procedure">. To make both variants of
108 PL/Python available in one installation, the source tree has to be
109 configured and built twice.
114 This results in the following usage and migration strategy:
119 Existing users and users who are currently not interested in
120 Python 3 use the language name <literal>plpythonu</literal> and
121 don't have to change anything for the foreseeable future. It is
122 recommended to gradually <quote>future-proof</quote> the code
123 via migration to Python 2.6/2.7 to simplify the eventual
124 migration to Python 3.
128 In practice, many PL/Python functions will migrate to Python 3
129 with few or no changes.
135 Users who know that they have heavily Python 2 dependent code
136 and don't plan to ever change it can make use of
137 the <literal>plpython2u</literal> language name. This will
138 continue to work into the very distant future, until Python 2
139 support might be completely dropped by PostgreSQL.
145 Users who want to dive into Python 3 can use
146 the <literal>plpython3u</literal> language name, which will keep
147 working forever by today's standards. In the distant future,
148 when Python 3 might become the default, they might like to
149 remove the <quote>3</quote> for aesthetic reasons.
155 Daredevils, who want to build a Python-3-only operating system
156 environment, can change the contents of
157 <link linkend="catalog-pg-pltemplate"><structname>pg_pltemplate</structname></link>
158 to make <literal>plpythonu</literal> be equivalent
159 to <literal>plpython3u</literal>, keeping in mind that this
160 would make their installation incompatible with most of the rest
169 document <ulink url="http://docs.python.org/py3k/whatsnew/3.0.html">What's
170 New In Python 3.0</ulink> for more information about porting to
175 It is not allowed to use PL/Python based on Python 2 and PL/Python
176 based on Python 3 in the same session, because the symbols in the
177 dynamic modules would clash, which could result in crashes of the
178 PostgreSQL server process. There is a check that prevents mixing
179 Python major versions in a session, which will abort the session if
180 a mismatch is detected. It is possible, however, to use both
181 PL/Python variants in the same database, from separate sessions.
185 <sect1 id="plpython-funcs">
186 <title>PL/Python Functions</title>
189 Functions in PL/Python are declared via the
190 standard <xref linkend="sql-createfunction"> syntax:
193 CREATE FUNCTION <replaceable>funcname</replaceable> (<replaceable>argument-list</replaceable>)
194 RETURNS <replaceable>return-type</replaceable>
196 # PL/Python function body
197 $$ LANGUAGE plpythonu;
202 The body of a function is simply a Python script. When the function
203 is called, its arguments are passed as elements of the list
204 <varname>args</varname>; named arguments are also passed as
205 ordinary variables to the Python script. Use of named arguments is
206 usually more readable. The result is returned from the Python code
207 in the usual way, with <literal>return</literal> or
208 <literal>yield</literal> (in case of a result-set statement). If
209 you do not provide a return value, Python returns the default
210 <symbol>None</symbol>. <application>PL/Python</application> translates
211 Python's <symbol>None</symbol> into the SQL null value.
215 For example, a function to return the greater of two integers can be
219 CREATE FUNCTION pymax (a integer, b integer)
225 $$ LANGUAGE plpythonu;
228 The Python code that is given as the body of the function definition
229 is transformed into a Python function. For example, the above results in:
232 def __plpython_procedure_pymax_23456():
238 assuming that 23456 is the OID assigned to the function by
239 <productname>PostgreSQL</productname>.
243 The arguments are set as global variables. Because of the scoping
244 rules of Python, this has the subtle consequence that an argument
245 variable cannot be reassigned inside the function to the value of
246 an expression that involves the variable name itself, unless the
247 variable is redeclared as global in the block. For example, the
248 following won't work:
250 CREATE FUNCTION pystrip(x text)
253 x = x.strip() # error
255 $$ LANGUAGE plpythonu;
257 because assigning to <varname>x</varname>
258 makes <varname>x</varname> a local variable for the entire block,
259 and so the <varname>x</varname> on the right-hand side of the
260 assignment refers to a not-yet-assigned local
261 variable <varname>x</varname>, not the PL/Python function
262 parameter. Using the <literal>global</literal> statement, this can
265 CREATE FUNCTION pystrip(x text)
269 x = x.strip() # ok now
271 $$ LANGUAGE plpythonu;
273 But it is advisable not to rely on this implementation detail of
274 PL/Python. It is better to treat the function parameters as
279 <sect1 id="plpython-data">
280 <title>Data Values</title>
282 Generally speaking, the aim of PL/Python is to provide
283 a <quote>natural</quote> mapping between the PostgreSQL and the
284 Python worlds. This informs the data mapping rules described
289 <title>Data Type Mapping</title>
291 When a PL/Python function is called, its arguments are converted from
292 their PostgreSQL data type to a corresponding Python type:
297 PostgreSQL <type>boolean</type> is converted to Python <type>bool</type>.
303 PostgreSQL <type>smallint</type> and <type>int</type> are
304 converted to Python <type>int</type>.
305 PostgreSQL <type>bigint</type> and <type>oid</type> are converted
306 to <type>long</type> in Python 2 and to <type>int</type> in
313 PostgreSQL <type>real</type> and <type>double</type> are converted to
314 Python <type>float</type>.
320 PostgreSQL <type>numeric</type> is converted to
321 Python <type>Decimal</type>. This type is imported from
322 the <literal>cdecimal</literal> package if that is available.
324 <literal>decimal.Decimal</literal> from the standard library will be
325 used. <literal>cdecimal</literal> is significantly faster
326 than <literal>decimal</literal>. In Python 3.3 and up,
327 however, <literal>cdecimal</literal> has been integrated into the
328 standard library under the name <literal>decimal</literal>, so there is
329 no longer any difference.
335 PostgreSQL <type>bytea</type> is converted to
336 Python <type>str</type> in Python 2 and to <type>bytes</type>
337 in Python 3. In Python 2, the string should be treated as a
338 byte sequence without any character encoding.
344 All other data types, including the PostgreSQL character string
345 types, are converted to a Python <type>str</type>. In Python
346 2, this string will be in the PostgreSQL server encoding; in
347 Python 3, it will be a Unicode string like all strings.
353 For nonscalar data types, see below.
360 When a PL/Python function returns, its return value is converted to the
361 function's declared PostgreSQL return data type as follows:
366 When the PostgreSQL return type is <type>boolean</type>, the
367 return value will be evaluated for truth according to the
368 <emphasis>Python</emphasis> rules. That is, 0 and empty string
369 are false, but notably <literal>'f'</literal> is true.
375 When the PostgreSQL return type is <type>bytea</type>, the
376 return value will be converted to a string (Python 2) or bytes
377 (Python 3) using the respective Python built-ins, with the
378 result being converted to <type>bytea</type>.
384 For all other PostgreSQL return types, the return value is converted
385 to a string using the Python built-in <literal>str</literal>, and the
386 result is passed to the input function of the PostgreSQL data type.
387 (If the Python value is a <type>float</>, it is converted using
388 the <literal>repr</> built-in instead of <literal>str</literal>, to
389 avoid loss of precision.)
393 Strings in Python 2 are required to be in the PostgreSQL server
394 encoding when they are passed to PostgreSQL. Strings that are
395 not valid in the current server encoding will raise an error,
396 but not all encoding mismatches can be detected, so garbage
397 data can still result when this is not done correctly. Unicode
398 strings are converted to the correct encoding automatically, so
399 it can be safer and more convenient to use those. In Python 3,
400 all strings are Unicode strings.
406 For nonscalar data types, see below.
411 Note that logical mismatches between the declared PostgreSQL
412 return type and the Python data type of the actual return object
413 are not flagged; the value will be converted in any case.
418 <title>Null, None</title>
420 If an SQL null value<indexterm><primary>null value</primary><secondary
421 sortas="PL/Python">in PL/Python</secondary></indexterm> is passed to a
422 function, the argument value will appear as <symbol>None</symbol> in
423 Python. For example, the function definition of <function>pymax</function>
424 shown in <xref linkend="plpython-funcs"> will return the wrong answer for null
425 inputs. We could add <literal>STRICT</literal> to the function definition
426 to make <productname>PostgreSQL</productname> do something more reasonable:
427 if a null value is passed, the function will not be called at all,
428 but will just return a null result automatically. Alternatively,
429 we could check for null inputs in the function body:
432 CREATE FUNCTION pymax (a integer, b integer)
435 if (a is None) or (b is None):
440 $$ LANGUAGE plpythonu;
443 As shown above, to return an SQL null value from a PL/Python
444 function, return the value <symbol>None</symbol>. This can be done whether the
445 function is strict or not.
449 <sect2 id="plpython-arrays">
450 <title>Arrays, Lists</title>
452 SQL array values are passed into PL/Python as a Python list. To
453 return an SQL array value out of a PL/Python function, return a
454 Python sequence, for example a list or tuple:
457 CREATE FUNCTION return_arr()
460 return (1, 2, 3, 4, 5)
461 $$ LANGUAGE plpythonu;
470 Note that in Python, strings are sequences, which can have
471 undesirable effects that might be familiar to Python programmers:
474 CREATE FUNCTION return_str_arr()
478 $$ LANGUAGE plpythonu;
480 SELECT return_str_arr();
490 <title>Composite Types</title>
492 Composite-type arguments are passed to the function as Python mappings. The
493 element names of the mapping are the attribute names of the composite type.
494 If an attribute in the passed row has the null value, it has the value
495 <symbol>None</symbol> in the mapping. Here is an example:
498 CREATE TABLE employee (
504 CREATE FUNCTION overpaid (e employee)
507 if e["salary"] > 200000:
509 if (e["age"] < 30) and (e["salary"] > 100000):
512 $$ LANGUAGE plpythonu;
517 There are multiple ways to return row or composite types from a Python
518 function. The following examples assume we have:
521 CREATE TYPE named_value AS (
527 A composite result can be returned as a:
531 <term>Sequence type (a tuple or list, but not a set because
532 it is not indexable)</term>
535 Returned sequence objects must have the same number of items as the
536 composite result type has fields. The item with index 0 is assigned to
537 the first field of the composite type, 1 to the second and so on. For
541 CREATE FUNCTION make_pair (name text, value integer)
544 return [ name, value ]
545 # or alternatively, as tuple: return ( name, value )
546 $$ LANGUAGE plpythonu;
549 To return a SQL null for any column, insert <symbol>None</symbol> at
550 the corresponding position.
556 <term>Mapping (dictionary)</term>
559 The value for each result type column is retrieved from the mapping
560 with the column name as key. Example:
563 CREATE FUNCTION make_pair (name text, value integer)
566 return { "name": name, "value": value }
567 $$ LANGUAGE plpythonu;
570 Any extra dictionary key/value pairs are ignored. Missing keys are
572 To return a SQL null value for any column, insert
573 <symbol>None</symbol> with the corresponding column name as the key.
579 <term>Object (any object providing method <literal>__getattr__</literal>)</term>
582 This works the same as a mapping.
586 CREATE FUNCTION make_pair (name text, value integer)
590 def __init__ (self, n, v):
593 return named_value(name, value)
600 $$ LANGUAGE plpythonu;
609 Functions with <literal>OUT</literal> parameters are also supported. For example:
611 CREATE FUNCTION multiout_simple(OUT i integer, OUT j integer) AS $$
613 $$ LANGUAGE plpythonu;
615 SELECT * FROM multiout_simple();
621 <title>Set-returning Functions</title>
623 A <application>PL/Python</application> function can also return sets of
624 scalar or composite types. There are several ways to achieve this because
625 the returned object is internally turned into an iterator. The following
626 examples assume we have composite type:
629 CREATE TYPE greeting AS (
635 A set result can be returned from a:
639 <term>Sequence type (tuple, list, set)</term>
643 CREATE FUNCTION greet (how text)
644 RETURNS SETOF greeting
646 # return tuple containing lists as composite types
647 # all other combinations work also
648 return ( [ how, "World" ], [ how, "PostgreSQL" ], [ how, "PL/Python" ] )
649 $$ LANGUAGE plpythonu;
656 <term>Iterator (any object providing <symbol>__iter__</symbol> and
657 <symbol>next</symbol> methods)</term>
661 CREATE FUNCTION greet (how text)
662 RETURNS SETOF greeting
665 def __init__ (self, how, who):
675 if self.ndx == len(self.who):
677 return ( self.how, self.who[self.ndx] )
679 return producer(how, [ "World", "PostgreSQL", "PL/Python" ])
680 $$ LANGUAGE plpythonu;
687 <term>Generator (<literal>yield</literal>)</term>
691 CREATE FUNCTION greet (how text)
692 RETURNS SETOF greeting
694 for who in [ "World", "PostgreSQL", "PL/Python" ]:
696 $$ LANGUAGE plpythonu;
702 <ulink url="http://bugs.python.org/issue1483133">bug #1483133</ulink>,
703 some debug versions of Python 2.4
704 (configured and compiled with option <literal>--with-pydebug</literal>)
705 are known to crash the <productname>PostgreSQL</productname> server
706 when using an iterator to return a set result.
707 Unpatched versions of Fedora 4 contain this bug.
708 It does not happen in production versions of Python or on patched
709 versions of Fedora 4.
719 Set-returning functions with <literal>OUT</literal> parameters
720 (using <literal>RETURNS SETOF record</literal>) are also
721 supported. For example:
723 CREATE FUNCTION multiout_simple_setof(n integer, OUT integer, OUT integer) RETURNS SETOF record AS $$
725 $$ LANGUAGE plpythonu;
727 SELECT * FROM multiout_simple_setof(3);
733 <sect1 id="plpython-sharing">
734 <title>Sharing Data</title>
736 The global dictionary <varname>SD</varname> is available to store
737 data between function calls. This variable is private static data.
738 The global dictionary <varname>GD</varname> is public data,
739 available to all Python functions within a session. Use with
740 care.<indexterm><primary>global data</>
741 <secondary>in PL/Python</></indexterm>
745 Each function gets its own execution environment in the
746 Python interpreter, so that global data and function arguments from
747 <function>myfunc</function> are not available to
748 <function>myfunc2</function>. The exception is the data in the
749 <varname>GD</varname> dictionary, as mentioned above.
753 <sect1 id="plpython-do">
754 <title>Anonymous Code Blocks</title>
757 PL/Python also supports anonymous code blocks called with the
758 <xref linkend="sql-do"> statement:
763 $$ LANGUAGE plpythonu;
766 An anonymous code block receives no arguments, and whatever value it
767 might return is discarded. Otherwise it behaves just like a function.
771 <sect1 id="plpython-trigger">
772 <title>Trigger Functions</title>
774 <indexterm zone="plpython-trigger">
775 <primary>trigger</primary>
776 <secondary>in PL/Python</secondary>
780 When a function is used as a trigger, the dictionary
781 <literal>TD</literal> contains trigger-related values:
784 <term><literal>TD["event"]</></term>
787 contains the event as a string:
788 <literal>INSERT</>, <literal>UPDATE</>,
789 <literal>DELETE</>, or <literal>TRUNCATE</>.
795 <term><literal>TD["when"]</></term>
798 contains one of <literal>BEFORE</>, <literal>AFTER</>, or
799 <literal>INSTEAD OF</>.
805 <term><literal>TD["level"]</></term>
808 contains <literal>ROW</> or <literal>STATEMENT</>.
814 <term><literal>TD["new"]</></term>
815 <term><literal>TD["old"]</></term>
818 For a row-level trigger, one or both of these fields contain
819 the respective trigger rows, depending on the trigger event.
825 <term><literal>TD["name"]</></term>
828 contains the trigger name.
834 <term><literal>TD["table_name"]</></term>
837 contains the name of the table on which the trigger occurred.
843 <term><literal>TD["table_schema"]</></term>
846 contains the schema of the table on which the trigger occurred.
852 <term><literal>TD["relid"]</></term>
855 contains the OID of the table on which the trigger occurred.
861 <term><literal>TD["args"]</></term>
864 If the <command>CREATE TRIGGER</> command
865 included arguments, they are available in <literal>TD["args"][0]</> to
866 <literal>TD["args"][<replaceable>n</>-1]</>.
874 If <literal>TD["when"]</literal> is <literal>BEFORE</> or
875 <literal>INSTEAD OF</> and
876 <literal>TD["level"]</literal> is <literal>ROW</>, you can
877 return <literal>None</literal> or <literal>"OK"</literal> from the
878 Python function to indicate the row is unmodified,
879 <literal>"SKIP"</> to abort the event, or if <literal>TD["event"]</>
880 is <command>INSERT</> or <command>UPDATE</> you can return
881 <literal>"MODIFY"</> to indicate you've modified the new row.
882 Otherwise the return value is ignored.
886 <sect1 id="plpython-database">
887 <title>Database Access</title>
890 The PL/Python language module automatically imports a Python module
891 called <literal>plpy</literal>. The functions and constants in
892 this module are available to you in the Python code as
893 <literal>plpy.<replaceable>foo</replaceable></literal>.
897 <title>Database Access Functions</title>
900 The <literal>plpy</literal> module provides several functions to execute
906 <term><literal>plpy.<function>execute</function>(<replaceable>query</replaceable> [, <replaceable>max-rows</replaceable>])</literal></term>
909 Calling <function>plpy.execute</function> with a query string and an
910 optional row limit argument causes that query to be run and the result to
911 be returned in a result object.
915 The result object emulates a list or dictionary object. The result
916 object can be accessed by row number and column name. For example:
918 rv = plpy.execute("SELECT * FROM my_table", 5)
920 returns up to 5 rows from <literal>my_table</literal>. If
921 <literal>my_table</literal> has a column
922 <literal>my_column</literal>, it would be accessed as:
924 foo = rv[i]["my_column"]
926 The number of rows returned can be obtained using the built-in
927 <function>len</function> function.
931 The result object has these additional methods:
934 <term><literal><function>nrows</function>()</literal></term>
937 Returns the number of rows processed by the command. Note that this
938 is not necessarily the same as the number of rows returned. For
939 example, an <command>UPDATE</command> command will set this value but
940 won't return any rows (unless <literal>RETURNING</literal> is used).
946 <term><literal><function>status</function>()</literal></term>
949 The <function>SPI_execute()</function> return value.
955 <term><literal><function>colnames</function>()</literal></term>
956 <term><literal><function>coltypes</function>()</literal></term>
957 <term><literal><function>coltypmods</function>()</literal></term>
960 Return a list of column names, list of column type OIDs, and list of
961 type-specific type modifiers for the columns, respectively.
965 These methods raise an exception when called on a result object from
966 a command that did not produce a result set, e.g.,
967 <command>UPDATE</command> without <literal>RETURNING</literal>, or
968 <command>DROP TABLE</command>. But it is OK to use these methods on
969 a result set containing zero rows.
975 <term><literal><function>__str__</function>()</literal></term>
978 The standard <literal>__str__</literal> method is defined so that it
979 is possible for example to debug query execution results
980 using <literal>plpy.debug(rv)</literal>.
988 The result object can be modified.
992 Note that calling <literal>plpy.execute</literal> will cause the entire
993 result set to be read into memory. Only use that function when you are
994 sure that the result set will be relatively small. If you don't want to
995 risk excessive memory usage when fetching large results,
996 use <literal>plpy.cursor</literal> rather
997 than <literal>plpy.execute</literal>.
1003 <term><literal>plpy.<function>prepare</function>(<replaceable>query</replaceable> [, <replaceable>argtypes</replaceable>])</literal></term>
1004 <term><literal>plpy.<function>execute</function>(<replaceable>plan</replaceable> [, <replaceable>arguments</replaceable> [, <replaceable>max-rows</replaceable>]])</literal></term>
1007 <indexterm><primary>preparing a query</><secondary>in PL/Python</></indexterm>
1008 <function>plpy.prepare</function> prepares the execution plan for a
1009 query. It is called with a query string and a list of parameter types,
1010 if you have parameter references in the query. For example:
1012 plan = plpy.prepare("SELECT last_name FROM my_users WHERE first_name = $1", ["text"])
1014 <literal>text</literal> is the type of the variable you will be passing
1015 for <literal>$1</literal>. The second argument is optional if you don't
1016 want to pass any parameters to the query.
1019 After preparing a statement, you use a variant of the
1020 function <function>plpy.execute</function> to run it:
1022 rv = plpy.execute(plan, ["name"], 5)
1024 Pass the plan as the first argument (instead of the query string), and a
1025 list of values to substitute into the query as the second argument. The
1026 second argument is optional if the query does not expect any parameters.
1027 The third argument is the optional row limit as before.
1031 Query parameters and result row fields are converted between PostgreSQL
1032 and Python data types as described in <xref linkend="plpython-data">.
1036 When you prepare a plan using the PL/Python module it is automatically
1037 saved. Read the SPI documentation (<xref linkend="spi">) for a
1038 description of what this means. In order to make effective use of this
1039 across function calls one needs to use one of the persistent storage
1040 dictionaries <literal>SD</literal> or <literal>GD</literal> (see
1041 <xref linkend="plpython-sharing">). For example:
1043 CREATE FUNCTION usesavedplan() RETURNS trigger AS $$
1047 plan = plpy.prepare("SELECT 1")
1050 $$ LANGUAGE plpythonu;
1057 <term><literal>plpy.<function>cursor</function>(<replaceable>query</replaceable>)</literal></term>
1058 <term><literal>plpy.<function>cursor</function>(<replaceable>plan</replaceable> [, <replaceable>arguments</replaceable>])</literal></term>
1061 The <literal>plpy.cursor</literal> function accepts the same arguments
1062 as <literal>plpy.execute</literal> (except for the row limit) and returns
1063 a cursor object, which allows you to process large result sets in smaller
1064 chunks. As with <literal>plpy.execute</literal>, either a query string
1065 or a plan object along with a list of arguments can be used.
1069 The cursor object provides a <literal>fetch</literal> method that accepts
1070 an integer parameter and returns a result object. Each time you
1071 call <literal>fetch</literal>, the returned object will contain the next
1072 batch of rows, never larger than the parameter value. Once all rows are
1073 exhausted, <literal>fetch</literal> starts returning an empty result
1074 object. Cursor objects also provide an
1075 <ulink url="http://docs.python.org/library/stdtypes.html#iterator-types">iterator
1076 interface</ulink>, yielding one row at a time until all rows are
1077 exhausted. Data fetched that way is not returned as result objects, but
1078 rather as dictionaries, each dictionary corresponding to a single result
1083 An example of two ways of processing data from a large table is:
1085 CREATE FUNCTION count_odd_iterator() RETURNS integer AS $$
1087 for row in plpy.cursor("select num from largetable"):
1091 $$ LANGUAGE plpythonu;
1093 CREATE FUNCTION count_odd_fetch(batch_size integer) RETURNS integer AS $$
1095 cursor = plpy.cursor("select num from largetable")
1097 rows = cursor.fetch(batch_size)
1104 $$ LANGUAGE plpythonu;
1106 CREATE FUNCTION count_odd_prepared() RETURNS integer AS $$
1108 plan = plpy.prepare("select num from largetable where num % $1 <> 0", ["integer"])
1109 rows = list(plpy.cursor(plan, [2]))
1112 $$ LANGUAGE plpythonu;
1117 Cursors are automatically disposed of. But if you want to explicitly
1118 release all resources held by a cursor, use the <literal>close</literal>
1119 method. Once closed, a cursor cannot be fetched from anymore.
1124 Do not confuse objects created by <literal>plpy.cursor</literal> with
1125 DB-API cursors as defined by
1126 the <ulink url="http://www.python.org/dev/peps/pep-0249/">Python
1127 Database API specification</ulink>. They don't have anything in common
1128 except for the name.
1137 <sect2 id="plpython-trapping">
1138 <title>Trapping Errors</title>
1141 Functions accessing the database might encounter errors, which
1142 will cause them to abort and raise an exception. Both
1143 <function>plpy.execute</function> and
1144 <function>plpy.prepare</function> can raise an instance of a subclass of
1145 <literal>plpy.SPIError</literal>, which by default will terminate
1146 the function. This error can be handled just like any other
1147 Python exception, by using the <literal>try/except</literal>
1148 construct. For example:
1150 CREATE FUNCTION try_adding_joe() RETURNS text AS $$
1152 plpy.execute("INSERT INTO users(username) VALUES ('joe')")
1153 except plpy.SPIError:
1154 return "something went wrong"
1157 $$ LANGUAGE plpythonu;
1162 The actual class of the exception being raised corresponds to the
1163 specific condition that caused the error. Refer
1164 to <xref linkend="errcodes-table"> for a list of possible
1165 conditions. The module
1166 <literal>plpy.spiexceptions</literal> defines an exception class
1167 for each <productname>PostgreSQL</productname> condition, deriving
1168 their names from the condition name. For
1169 instance, <literal>division_by_zero</literal>
1170 becomes <literal>DivisionByZero</literal>, <literal>unique_violation</literal>
1171 becomes <literal>UniqueViolation</literal>, <literal>fdw_error</literal>
1172 becomes <literal>FdwError</literal>, and so on. Each of these
1173 exception classes inherits from <literal>SPIError</literal>. This
1174 separation makes it easier to handle specific errors, for
1177 CREATE FUNCTION insert_fraction(numerator int, denominator int) RETURNS text AS $$
1178 from plpy import spiexceptions
1180 plan = plpy.prepare("INSERT INTO fractions (frac) VALUES ($1 / $2)", ["int", "int"])
1181 plpy.execute(plan, [numerator, denominator])
1182 except spiexceptions.DivisionByZero:
1183 return "denominator cannot equal zero"
1184 except spiexceptions.UniqueViolation:
1185 return "already have that fraction"
1186 except plpy.SPIError, e:
1187 return "other error, SQLSTATE %s" % e.sqlstate
1189 return "fraction inserted"
1190 $$ LANGUAGE plpythonu;
1192 Note that because all exceptions from
1193 the <literal>plpy.spiexceptions</literal> module inherit
1194 from <literal>SPIError</literal>, an <literal>except</literal>
1195 clause handling it will catch any database access error.
1199 As an alternative way of handling different error conditions, you
1200 can catch the <literal>SPIError</literal> exception and determine
1201 the specific error condition inside the <literal>except</literal>
1202 block by looking at the <literal>sqlstate</literal> attribute of
1203 the exception object. This attribute is a string value containing
1204 the <quote>SQLSTATE</quote> error code. This approach provides
1205 approximately the same functionality
1210 <sect1 id="plpython-subtransaction">
1211 <title>Explicit Subtransactions</title>
1214 Recovering from errors caused by database access as described in
1215 <xref linkend="plpython-trapping"> can lead to an undesirable
1216 situation where some operations succeed before one of them fails,
1217 and after recovering from that error the data is left in an
1218 inconsistent state. PL/Python offers a solution to this problem in
1219 the form of explicit subtransactions.
1223 <title>Subtransaction Context Managers</title>
1226 Consider a function that implements a transfer between two
1229 CREATE FUNCTION transfer_funds() RETURNS void AS $$
1231 plpy.execute("UPDATE accounts SET balance = balance - 100 WHERE account_name = 'joe'")
1232 plpy.execute("UPDATE accounts SET balance = balance + 100 WHERE account_name = 'mary'")
1233 except plpy.SPIError, e:
1234 result = "error transferring funds: %s" % e.args
1236 result = "funds transferred correctly"
1237 plan = plpy.prepare("INSERT INTO operations (result) VALUES ($1)", ["text"])
1238 plpy.execute(plan, [result])
1239 $$ LANGUAGE plpythonu;
1241 If the second <literal>UPDATE</literal> statement results in an
1242 exception being raised, this function will report the error, but
1243 the result of the first <literal>UPDATE</literal> will
1244 nevertheless be committed. In other words, the funds will be
1245 withdrawn from Joe's account, but will not be transferred to
1250 To avoid such issues, you can wrap your
1251 <literal>plpy.execute</literal> calls in an explicit
1252 subtransaction. The <literal>plpy</literal> module provides a
1253 helper object to manage explicit subtransactions that gets created
1254 with the <literal>plpy.subtransaction()</literal> function.
1255 Objects created by this function implement the
1256 <ulink url="http://docs.python.org/library/stdtypes.html#context-manager-types">
1257 context manager interface</ulink>. Using explicit subtransactions
1258 we can rewrite our function as:
1260 CREATE FUNCTION transfer_funds2() RETURNS void AS $$
1262 with plpy.subtransaction():
1263 plpy.execute("UPDATE accounts SET balance = balance - 100 WHERE account_name = 'joe'")
1264 plpy.execute("UPDATE accounts SET balance = balance + 100 WHERE account_name = 'mary'")
1265 except plpy.SPIError, e:
1266 result = "error transferring funds: %s" % e.args
1268 result = "funds transferred correctly"
1269 plan = plpy.prepare("INSERT INTO operations (result) VALUES ($1)", ["text"])
1270 plpy.execute(plan, [result])
1271 $$ LANGUAGE plpythonu;
1273 Note that the use of <literal>try/catch</literal> is still
1274 required. Otherwise the exception would propagate to the top of
1275 the Python stack and would cause the whole function to abort with
1276 a <productname>PostgreSQL</productname> error, so that the
1277 <literal>operations</literal> table would not have any row
1278 inserted into it. The subtransaction context manager does not
1279 trap errors, it only assures that all database operations executed
1280 inside its scope will be atomically committed or rolled back. A
1281 rollback of the subtransaction block occurs on any kind of
1282 exception exit, not only ones caused by errors originating from
1283 database access. A regular Python exception raised inside an
1284 explicit subtransaction block would also cause the subtransaction
1290 <title>Older Python Versions</title>
1293 Context managers syntax using the <literal>with</literal> keyword
1294 is available by default in Python 2.6. If using PL/Python with an
1295 older Python version, it is still possible to use explicit
1296 subtransactions, although not as transparently. You can call the
1297 subtransaction manager's <literal>__enter__</literal> and
1298 <literal>__exit__</literal> functions using the
1299 <literal>enter</literal> and <literal>exit</literal> convenience
1300 aliases. The example function that transfers funds could be
1303 CREATE FUNCTION transfer_funds_old() RETURNS void AS $$
1305 subxact = plpy.subtransaction()
1308 plpy.execute("UPDATE accounts SET balance = balance - 100 WHERE account_name = 'joe'")
1309 plpy.execute("UPDATE accounts SET balance = balance + 100 WHERE account_name = 'mary'")
1312 subxact.exit(*sys.exc_info())
1315 subxact.exit(None, None, None)
1316 except plpy.SPIError, e:
1317 result = "error transferring funds: %s" % e.args
1319 result = "funds transferred correctly"
1321 plan = plpy.prepare("INSERT INTO operations (result) VALUES ($1)", ["text"])
1322 plpy.execute(plan, [result])
1323 $$ LANGUAGE plpythonu;
1329 Although context managers were implemented in Python 2.5, to use
1330 the <literal>with</literal> syntax in that version you need to
1332 url="http://docs.python.org/release/2.5/ref/future.html">future
1333 statement</ulink>. Because of implementation details, however,
1334 you cannot use future statements in PL/Python functions.
1340 <sect1 id="plpython-util">
1341 <title>Utility Functions</title>
1343 The <literal>plpy</literal> module also provides the functions
1345 <member><literal>plpy.debug(<replaceable>msg, **kwargs</>)</literal></member>
1346 <member><literal>plpy.log(<replaceable>msg, **kwargs</>)</literal></member>
1347 <member><literal>plpy.info(<replaceable>msg, **kwargs</>)</literal></member>
1348 <member><literal>plpy.notice(<replaceable>msg, **kwargs</>)</literal></member>
1349 <member><literal>plpy.warning(<replaceable>msg, **kwargs</>)</literal></member>
1350 <member><literal>plpy.error(<replaceable>msg, **kwargs</>)</literal></member>
1351 <member><literal>plpy.fatal(<replaceable>msg, **kwargs</>)</literal></member>
1353 <indexterm><primary>elog</><secondary>in PL/Python</></indexterm>
1354 <function>plpy.error</function> and <function>plpy.fatal</function>
1355 actually raise a Python exception which, if uncaught, propagates out to
1356 the calling query, causing the current transaction or subtransaction to
1357 be aborted. <literal>raise plpy.Error(<replaceable>msg</>)</literal> and
1358 <literal>raise plpy.Fatal(<replaceable>msg</>)</literal> are
1359 equivalent to calling <literal>plpy.error(<replaceable>msg</>)</literal> and
1360 <literal>plpy.fatal(<replaceable>msg</>)</literal>, respectively but
1361 the <literal>raise</literal> form does not allow passing keyword arguments.
1362 The other functions only generate messages of different priority levels.
1363 Whether messages of a particular priority are reported to the client,
1364 written to the server log, or both is controlled by the
1365 <xref linkend="guc-log-min-messages"> and
1366 <xref linkend="guc-client-min-messages"> configuration
1367 variables. See <xref linkend="runtime-config"> for more information.
1371 The <replaceable>msg</> argument is given as a positional argument. For
1372 backward compatibility, more than one positional argument can be given. In
1373 that case, the string representation of the tuple of positional arguments
1374 becomes the message reported to the client.
1378 The following keyword-only arguments are accepted:
1380 <member><literal>detail</literal></member>
1381 <member><literal>hint</literal></member>
1382 <member><literal>sqlstate</literal></member>
1383 <member><literal>schema_name</literal></member>
1384 <member><literal>table_name</literal></member>
1385 <member><literal>column_name</literal></member>
1386 <member><literal>datatype_name</literal></member>
1387 <member><literal>constraint_name</literal></member>
1389 The string representation of the objects passed as keyword-only arguments
1390 is used to enrich the messages reported to the client. For example:
1393 CREATE FUNCTION raise_custom_exception() RETURNS void AS $$
1394 plpy.error("custom exception message",
1395 detail="some info about exception",
1396 hint="hint for users")
1397 $$ LANGUAGE plpythonu;
1399 =# SELECT raise_custom_exception();
1400 ERROR: plpy.Error: custom exception message
1401 DETAIL: some info about exception
1402 HINT: hint for users
1403 CONTEXT: Traceback (most recent call last):
1404 PL/Python function "raise_custom_exception", line 4, in <module>
1405 hint="hint for users")
1406 PL/Python function "raise_custom_exception"
1411 Another set of utility functions are
1412 <literal>plpy.quote_literal(<replaceable>string</>)</literal>,
1413 <literal>plpy.quote_nullable(<replaceable>string</>)</literal>, and
1414 <literal>plpy.quote_ident(<replaceable>string</>)</literal>. They
1415 are equivalent to the built-in quoting functions described in <xref
1416 linkend="functions-string">. They are useful when constructing
1417 ad-hoc queries. A PL/Python equivalent of dynamic SQL from <xref
1418 linkend="plpgsql-quote-literal-example"> would be:
1420 plpy.execute("UPDATE tbl SET %s = %s WHERE key = %s" % (
1421 plpy.quote_ident(colname),
1422 plpy.quote_nullable(newvalue),
1423 plpy.quote_literal(keyvalue)))
1428 <sect1 id="plpython-envar">
1429 <title>Environment Variables</title>
1432 Some of the environment variables that are accepted by the Python
1433 interpreter can also be used to affect PL/Python behavior. They
1434 would need to be set in the environment of the main PostgreSQL
1435 server process, for example in a start script. The available
1436 environment variables depend on the version of Python; see the
1437 Python documentation for details. At the time of this writing, the
1438 following environment variables have an affect on PL/Python,
1439 assuming an adequate Python version:
1442 <para><envar>PYTHONHOME</envar></para>
1446 <para><envar>PYTHONPATH</envar></para>
1450 <para><envar>PYTHONY2K</envar></para>
1454 <para><envar>PYTHONOPTIMIZE</envar></para>
1458 <para><envar>PYTHONDEBUG</envar></para>
1462 <para><envar>PYTHONVERBOSE</envar></para>
1466 <para><envar>PYTHONCASEOK</envar></para>
1470 <para><envar>PYTHONDONTWRITEBYTECODE</envar></para>
1474 <para><envar>PYTHONIOENCODING</envar></para>
1478 <para><envar>PYTHONUSERBASE</envar></para>
1482 <para><envar>PYTHONHASHSEED</envar></para>
1486 (It appears to be a Python implementation detail beyond the control
1487 of PL/Python that some of the environment variables listed on
1488 the <command>python</command> man page are only effective in a
1489 command-line interpreter and not an embedded Python interpreter.)