--- /dev/null
+From pgsql-hackers-owner+M16329=candle.pha.pa.us=pgman@postgresql.org Thu Dec 6 13:31:28 2001
+Return-path: <pgsql-hackers-owner+M16329=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB6IVRZ13376
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 13:31:27 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB6IVQa26949
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 13:31:26 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB6IRvR61959
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 12:29:06 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16329=candle.pha.pa.us=pgman@postgresql.org)
+Received: from gromit.dotclick.com (ipn9-f8366.net-resource.net [216.204.83.66])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB6IQ2m78192
+ for <pgsql-hackers@postgresql.org>; Thu, 6 Dec 2001 13:26:05 -0500 (EST)
+ (envelope-from markw@mohawksoft.com)
+Received: from mohawksoft.com (IDENT:markw@localhost.localdomain [127.0.0.1])
+ by gromit.dotclick.com (8.9.3/8.9.3) with ESMTP id NAA10076
+ for <pgsql-hackers@postgresql.org>; Thu, 6 Dec 2001 13:28:04 -0500
+Message-ID: <3C0FB8B4.382C7736@mohawksoft.com>
+Date: Thu, 06 Dec 2001 13:28:04 -0500
+From: mlw <markw@mohawksoft.com>
+X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.4.16 i686)
+X-Accept-Language: en
+MIME-Version: 1.0
+To: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: [HACKERS] Remote connections?
+Content-Type: text/plain; charset=us-ascii
+Content-Transfer-Encoding: 7bit
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+I just found out something about Oracle which that looks like something
+that could be doable in PostgreSQL.
+
+What do you all think:
+
+Oracle's version is something like this:
+
+create [public] database link using [...]
+
+select * from sometable@remotelink
+
+
+I was thinking how this could be done with postgreSQL. How hard would it
+be to make something that is similar to a view, but executes a query
+remotely? I envision something like this:
+
+create [public] link name query using [...]
+
+The table link will be similar to a view. It could be used like this:
+
+CREATE LINK test as select * from test WITH 'user=postgres host=remote
+db=data';
+
+SELECT * from test;
+
+or
+
+SELECT * from fubar join test on (fubar.id = test.id) ;
+
+So, what do you think? Impossible, possible, too hard? too easy?
+
+---------------------------(end of broadcast)---------------------------
+TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
+
+From pgsql-hackers-owner+M16331=candle.pha.pa.us=pgman@postgresql.org Thu Dec 6 15:12:28 2001
+Return-path: <pgsql-hackers-owner+M16331=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB6KCQZ19987
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 15:12:27 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB6KCQa13967
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 15:12:26 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB6K6nR65020
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 14:07:54 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16331=candle.pha.pa.us=pgman@postgresql.org)
+Received: from ece.rice.edu (ece.rice.edu [128.42.4.34])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB6K6Im96910
+ for <pgsql-hackers@postgresql.org>; Thu, 6 Dec 2001 15:06:18 -0500 (EST)
+ (envelope-from reedstrm@rice.edu)
+Received: from localhost (localhost [127.0.0.1])
+ by ece.rice.edu (Postfix) with ESMTP
+ id A9E4E68A1D; Thu, 6 Dec 2001 14:06:17 -0600 (CST)
+Received: from wallace.ece.rice.edu (wallace.ece.rice.edu [128.42.12.154])
+ by ece.rice.edu (Postfix) with ESMTP
+ id EA06668A17; Thu, 6 Dec 2001 14:06:16 -0600 (CST)
+Received: from reedstrm by wallace.ece.rice.edu with local (Exim 3.22 #1 (Debian))
+ id 16C4me-0002uX-00; Thu, 06 Dec 2001 14:06:16 -0600
+Date: Thu, 6 Dec 2001 14:06:16 -0600
+From: "Ross J. Reedstrom" <reedstrm@rice.edu>
+To: mlw <markw@mohawksoft.com>
+cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+Message-ID: <20011206140616.C10995@rice.edu>
+Mail-Followup-To: "Ross J. Reedstrom" <reedstrm@ece.rice.edu>,
+ mlw <markw@mohawksoft.com>,
+ "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+References: <3C0FB8B4.382C7736@mohawksoft.com>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+In-Reply-To: <3C0FB8B4.382C7736@mohawksoft.com>
+User-Agent: Mutt/1.3.18i
+X-Virus-Scanned: by AMaViS snapshot-20010714
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+On Thu, Dec 06, 2001 at 01:28:04PM -0500, mlw wrote:
+> I just found out something about Oracle which that looks like something
+> that could be doable in PostgreSQL.
+>
+> What do you all think:
+>
+> Oracle's version is something like this:
+>
+> create [public] database link using [...]
+>
+> select * from sometable@remotelink
+>
+>
+> I was thinking how this could be done with postgreSQL. How hard would it
+> be to make something that is similar to a view, but executes a query
+> remotely? I envision something like this:
+>
+> create [public] link name query using [...]
+>
+> The table link will be similar to a view. It could be used like this:
+>
+> CREATE LINK test as select * from test WITH 'user=postgres host=remote
+> db=data';
+>
+> SELECT * from test;
+>
+> or
+>
+> SELECT * from fubar join test on (fubar.id = test.id) ;
+>
+> So, what do you think? Impossible, possible, too hard? too easy?
+
+Here we come, full circle. This is just about where I came on board.
+Many moons ago, I started looking at Mariposa, in the hopes of forward
+patching it into PostgreSQL, and generalizing the 'remote' part to allow
+exactly the sort of access you described above.
+
+The biggest problem with this is transactional semantics: you need
+two-stage commits to get this right, and we don't hav'em. (Has there
+been an indepth discussion concerning what how hard it would be to do
+that with postgresql?)
+
+The _actual_ biggest problem was my lack of knowledge of the PostgreSQL
+codebase ;-)
+
+Ross
+--
+Ross Reedstrom, Ph.D. reedstrm@rice.edu
+Executive Director phone: 713-348-6166
+Gulf Coast Consortium for Bioinformatics fax: 713-348-6182
+Rice University MS-39
+Houston, TX 77005
+
+---------------------------(end of broadcast)---------------------------
+TIP 4: Don't 'kill -9' the postmaster
+
+From pgsql-hackers-owner+M16332=candle.pha.pa.us=pgman@postgresql.org Thu Dec 6 15:31:27 2001
+Return-path: <pgsql-hackers-owner+M16332=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB6KVQZ21158
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 15:31:26 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB6KVQa22900
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 15:31:26 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB6KRrR65596
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 14:28:55 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16332=candle.pha.pa.us=pgman@postgresql.org)
+Received: from gromit.dotclick.com (ipn9-f8366.net-resource.net [216.204.83.66])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB6KJXm97564
+ for <pgsql-hackers@postgresql.org>; Thu, 6 Dec 2001 15:19:33 -0500 (EST)
+ (envelope-from markw@mohawksoft.com)
+Received: from mohawksoft.com (IDENT:markw@localhost.localdomain [127.0.0.1])
+ by gromit.dotclick.com (8.9.3/8.9.3) with ESMTP id PAA10771;
+ Thu, 6 Dec 2001 15:21:13 -0500
+Message-ID: <3C0FD339.6F663329@mohawksoft.com>
+Date: Thu, 06 Dec 2001 15:21:13 -0500
+From: mlw <markw@mohawksoft.com>
+X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.4.16 i686)
+X-Accept-Language: en
+MIME-Version: 1.0
+To: "Ross J. Reedstrom" <reedstrm@rice.edu>
+cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+References: <3C0FB8B4.382C7736@mohawksoft.com> <20011206140616.C10995@rice.edu>
+Content-Type: text/plain; charset=us-ascii
+Content-Transfer-Encoding: 7bit
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+"Ross J. Reedstrom" wrote:
+>
+> On Thu, Dec 06, 2001 at 01:28:04PM -0500, mlw wrote:
+> > I just found out something about Oracle which that looks like something
+> > that could be doable in PostgreSQL.
+> >
+> > What do you all think:
+> >
+> > Oracle's version is something like this:
+> >
+> > create [public] database link using [...]
+> >
+> > select * from sometable@remotelink
+> >
+> >
+> > I was thinking how this could be done with postgreSQL. How hard would it
+> > be to make something that is similar to a view, but executes a query
+> > remotely? I envision something like this:
+> >
+> > create [public] link name query using [...]
+> >
+> > The table link will be similar to a view. It could be used like this:
+> >
+> > CREATE LINK test as select * from test WITH 'user=postgres host=remote
+> > db=data';
+> >
+> > SELECT * from test;
+> >
+> > or
+> >
+> > SELECT * from fubar join test on (fubar.id = test.id) ;
+> >
+> > So, what do you think? Impossible, possible, too hard? too easy?
+>
+> Here we come, full circle. This is just about where I came on board.
+> Many moons ago, I started looking at Mariposa, in the hopes of forward
+> patching it into PostgreSQL, and generalizing the 'remote' part to allow
+> exactly the sort of access you described above.
+>
+> The biggest problem with this is transactional semantics: you need
+> two-stage commits to get this right, and we don't hav'em. (Has there
+> been an indepth discussion concerning what how hard it would be to do
+> that with postgresql?)
+>
+> The _actual_ biggest problem was my lack of knowledge of the PostgreSQL
+> codebase ;-)
+
+I think we can we can dispense worrying about two stage commits, if we
+assume that remote connections are treated as views with no rules. As
+long as remote tables are "read only" then the implementation is much
+easier.
+
+I too find the internals of PostgreSQL virtually incomprehensible at the
+internal level. If there were a document somewhere which published how a
+function could return multiple tuples, remote views would be a trivial
+undertaking. It could look like:
+
+select * from remote('select *from table', 'user=postgres host=outland
+db=remote');
+
+---------------------------(end of broadcast)---------------------------
+TIP 4: Don't 'kill -9' the postmaster
+
+From pgsql-hackers-owner+M16335=candle.pha.pa.us=pgman@postgresql.org Thu Dec 6 17:11:29 2001
+Return-path: <pgsql-hackers-owner+M16335=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB6MBSZ06676
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 17:11:28 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB6MBSq07059
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 17:11:28 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB6M6sR68489
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 16:08:16 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16335=candle.pha.pa.us=pgman@postgresql.org)
+Received: from mx1.relaypoint.net (ns2.generalbroadband.com [64.32.62.5])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB6M3Im04451
+ for <pgsql-hackers@postgresql.org>; Thu, 6 Dec 2001 17:03:18 -0500 (EST)
+ (envelope-from joseph.conway@home.com)
+Received: from [206.19.64.3] (account jconway@wwc.com HELO home.com)
+ by mx1.relaypoint.net (CommuniGate Pro SMTP 3.4.8)
+ with ESMTP id 1707032; Thu, 06 Dec 2001 14:03:14 -0800
+Message-ID: <3C0FEB2C.70000@home.com>
+Date: Thu, 06 Dec 2001 14:03:24 -0800
+From: Joe Conway <joseph.conway@home.com>
+User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.6+) Gecko/20011126
+X-Accept-Language: en-us
+MIME-Version: 1.0
+To: mlw <markw@mohawksoft.com>
+cc: "Ross J. Reedstrom" <reedstrm@rice.edu>,
+ "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+References: <3C0FB8B4.382C7736@mohawksoft.com> <20011206140616.C10995@rice.edu> <3C0FD339.6F663329@mohawksoft.com>
+Content-Type: text/plain; charset=us-ascii; format=flowed
+Content-Transfer-Encoding: 7bit
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+mlw wrote:
+
+> I too find the internals of PostgreSQL virtually incomprehensible at the
+> internal level. If there were a document somewhere which published how a
+> function could return multiple tuples, remote views would be a trivial
+> undertaking. It could look like:
+>
+> select * from remote('select *from table', 'user=postgres host=outland
+> db=remote');
+>
+
+See contrib/dblink in the 7.2 beta. It was my attempt inspired by
+Oracle's dblink and some code that you (mlw) posted a while back. Based
+on the limitations wrt returning muliple tuples, I wound up with
+something like:
+
+lt_lcat=# select dblink_tok(t1.dblink_p,0) as f1 from (select
+dblink('hostaddr=127.0.0.1 dbname=template1 user=postgres
+password=postgres','select proname from pg_proc') as dblink_p) as t1;
+
+Which, as has been pointed out more than once, is pretty ugly, but at
+least it's a start ;-)
+
+
+Joe
+
+
+---------------------------(end of broadcast)---------------------------
+TIP 4: Don't 'kill -9' the postmaster
+
+From pgsql-hackers-owner+M16336=candle.pha.pa.us=pgman@postgresql.org Thu Dec 6 18:41:31 2001
+Return-path: <pgsql-hackers-owner+M16336=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB6NfPZ12249
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 18:41:25 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB6NfQq03244
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 18:41:26 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB6NbOR70960
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 17:38:19 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16336=candle.pha.pa.us=pgman@postgresql.org)
+Received: from spider.pilosoft.com (p55-222.acedsl.com [160.79.55.222])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB6NIgm07232
+ for <pgsql-hackers@postgresql.org>; Thu, 6 Dec 2001 18:18:43 -0500 (EST)
+ (envelope-from alex@pilosoft.com)
+Received: from localhost (alexmail@localhost)
+ by spider.pilosoft.com (8.9.3/8.9.3) with ESMTP id SAA23999;
+ Thu, 6 Dec 2001 18:23:22 -0500 (EST)
+Date: Thu, 6 Dec 2001 18:23:22 -0500 (EST)
+From: Alex Pilosov <alex@pilosoft.com>
+To: mlw <markw@mohawksoft.com>
+cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+In-Reply-To: <3C0FD339.6F663329@mohawksoft.com>
+Message-ID: <Pine.BSO.4.10.10112061822080.22618-100000@spider.pilosoft.com>
+MIME-Version: 1.0
+Content-Type: TEXT/PLAIN; charset=US-ASCII
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+On Thu, 6 Dec 2001, mlw wrote:
+
+> I too find the internals of PostgreSQL virtually incomprehensible at the
+> internal level. If there were a document somewhere which published how a
+> function could return multiple tuples, remote views would be a trivial
+> undertaking. It could look like:
+>
+> select * from remote('select *from table', 'user=postgres host=outland
+> db=remote');
+This isn't possible yet. I was working on implementation of this, about
+80% done, but never finished. Now I'm out of time to work more on this for
+a while. :(
+
+Let me know if you want my code.
+
+-alex
+
+
+---------------------------(end of broadcast)---------------------------
+TIP 6: Have you searched our list archives?
+
+http://archives.postgresql.org
+
+From pgsql-hackers-owner+M16340=candle.pha.pa.us=pgman@postgresql.org Fri Dec 7 00:32:59 2001
+Return-path: <pgsql-hackers-owner+M16340=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB75WsZ06911
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 00:32:54 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB75Wtq16801
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 00:32:55 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB75SCR80525
+ for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 23:29:17 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16340=candle.pha.pa.us=pgman@postgresql.org)
+Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.mediaone.net [24.218.67.153])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB7593m27913
+ for <pgsql-hackers@postgresql.org>; Fri, 7 Dec 2001 00:09:03 -0500 (EST)
+ (envelope-from markw@mohawksoft.com)
+Received: from mohawksoft.com (localhost [127.0.0.1])
+ by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id fB7561030613;
+ Fri, 7 Dec 2001 00:06:01 -0500
+Message-ID: <3C104E38.DA19C867@mohawksoft.com>
+Date: Fri, 07 Dec 2001 00:06:01 -0500
+From: mlw <markw@mohawksoft.com>
+X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.14ext3 i686)
+X-Accept-Language: en
+MIME-Version: 1.0
+To: Joe Conway <joseph.conway@home.com>
+cc: "Ross J. Reedstrom" <reedstrm@rice.edu>,
+ "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+References: <3C0FB8B4.382C7736@mohawksoft.com> <20011206140616.C10995@rice.edu> <3C0FD339.6F663329@mohawksoft.com> <3C0FEB2C.70000@home.com>
+Content-Type: text/plain; charset=us-ascii
+Content-Transfer-Encoding: 7bit
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+Hey this looks really cool. It looks like something I was thinking about doing.
+I have a couple suggestions that could make it a little better, I hope you will
+not be offended. (If you want my help, I'll chip in!)
+
+Why not use a binary cursor? That way native types can slip through without the
+overhead of conversion.
+
+Right now you get all rows up front, you may be able to increase overall
+performance by fetching only a few rows at a time, rather than get everything
+all at once. (Think on the order of 4 million rows from your remote query!)
+Execute the commit at the end of processing. There are even some asynchronous
+functions you may be able to utilize to reduce the I/O bottleneck. Use the
+synchronous function first, then before you return initiate an asynchronous
+read. Every successive pass through the function, read the newly arrived tuple,
+and initiate the next asynchronous read. (The two machine could be processing
+the query simultaneously, and this could even IMPROVE performance over a single
+system for heavy duty queries.)
+
+Setup a hash table for field names, rather than requiring field numbers. (Keep
+field number for efficiency, of course.)
+
+You could eliminate having to pass the result pointer around by keeping a
+static array in your extension. Use something like Oracle's "contains" notation
+of result number. Where each instantiation of "contains()" and "score()"
+require an id. i.e. 1,2,3,40 etc. Then hash those numbers into an array. I have
+some code that does this for a PostgreSQL extension (it implements contains) on
+my website (pgcontains, under download). It is ugly but works for the most
+part.
+
+Seriously, your stuff looks great. I think it could be the beginning of a
+fairly usable system for my company. Any help you need/want, just let me know.
+
+
+Joe Conway wrote:
+>
+> mlw wrote:
+>
+> > I too find the internals of PostgreSQL virtually incomprehensible at the
+> > internal level. If there were a document somewhere which published how a
+> > function could return multiple tuples, remote views would be a trivial
+> > undertaking. It could look like:
+> >
+> > select * from remote('select *from table', 'user=postgres host=outland
+> > db=remote');
+> >
+>
+> See contrib/dblink in the 7.2 beta. It was my attempt inspired by
+> Oracle's dblink and some code that you (mlw) posted a while back. Based
+> on the limitations wrt returning muliple tuples, I wound up with
+> something like:
+>
+> lt_lcat=# select dblink_tok(t1.dblink_p,0) as f1 from (select
+> dblink('hostaddr=127.0.0.1 dbname=template1 user=postgres
+> password=postgres','select proname from pg_proc') as dblink_p) as t1;
+>
+> Which, as has been pointed out more than once, is pretty ugly, but at
+> least it's a start ;-)
+>
+> Joe
+>
+> ---------------------------(end of broadcast)---------------------------
+> TIP 4: Don't 'kill -9' the postmaster
+
+---------------------------(end of broadcast)---------------------------
+TIP 3: if posting/reading through Usenet, please send an appropriate
+subscribe-nomail command to majordomo@postgresql.org so that your
+message can get through to the mailing list cleanly
+
+From pgsql-hackers-owner+M16344=candle.pha.pa.us=pgman@postgresql.org Fri Dec 7 02:51:51 2001
+Return-path: <pgsql-hackers-owner+M16344=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB77poZ14221
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 02:51:50 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB77pqq08152
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 02:51:52 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB77lwR84105
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 01:49:04 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16344=candle.pha.pa.us=pgman@postgresql.org)
+Received: from ra.sai.msu.su (ra.sai.msu.su [158.250.29.2])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB77bmm32783
+ for <pgsql-hackers@postgresql.org>; Fri, 7 Dec 2001 02:37:49 -0500 (EST)
+ (envelope-from oleg@sai.msu.su)
+Received: from ra (ra [158.250.29.2])
+ by ra.sai.msu.su (8.9.3/8.9.3) with ESMTP id KAA04160;
+ Fri, 7 Dec 2001 10:37:04 +0300 (GMT)
+Date: Fri, 7 Dec 2001 10:37:04 +0300 (GMT)
+From: Oleg Bartunov <oleg@sai.msu.su>
+X-X-Sender: <megera@ra.sai.msu.su>
+To: mlw <markw@mohawksoft.com>
+cc: Joe Conway <joseph.conway@home.com>,
+ "Ross J. Reedstrom" <reedstrm@rice.edu>,
+ "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+In-Reply-To: <3C104E38.DA19C867@mohawksoft.com>
+Message-ID: <Pine.GSO.4.33.0112071035180.20511-100000@ra.sai.msu.su>
+MIME-Version: 1.0
+Content-Type: TEXT/PLAIN; charset=US-ASCII
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+On Fri, 7 Dec 2001, mlw wrote:
+
+>
+> You could eliminate having to pass the result pointer around by keeping a
+> static array in your extension. Use something like Oracle's "contains" notation
+> of result number. Where each instantiation of "contains()" and "score()"
+> require an id. i.e. 1,2,3,40 etc. Then hash those numbers into an array. I have
+> some code that does this for a PostgreSQL extension (it implements contains) on
+> my website (pgcontains, under download). It is ugly but works for the most
+> part.
+
+contrib/intarray does this job very well
+
+>
+> Seriously, your stuff looks great. I think it could be the beginning of a
+> fairly usable system for my company. Any help you need/want, just let me know.
+>
+>
+> Joe Conway wrote:
+> >
+> > mlw wrote:
+> >
+> > > I too find the internals of PostgreSQL virtually incomprehensible at the
+> > > internal level. If there were a document somewhere which published how a
+> > > function could return multiple tuples, remote views would be a trivial
+> > > undertaking. It could look like:
+> > >
+> > > select * from remote('select *from table', 'user=postgres host=outland
+> > > db=remote');
+> > >
+> >
+> > See contrib/dblink in the 7.2 beta. It was my attempt inspired by
+> > Oracle's dblink and some code that you (mlw) posted a while back. Based
+> > on the limitations wrt returning muliple tuples, I wound up with
+> > something like:
+> >
+> > lt_lcat=# select dblink_tok(t1.dblink_p,0) as f1 from (select
+> > dblink('hostaddr=127.0.0.1 dbname=template1 user=postgres
+> > password=postgres','select proname from pg_proc') as dblink_p) as t1;
+> >
+> > Which, as has been pointed out more than once, is pretty ugly, but at
+> > least it's a start ;-)
+> >
+> > Joe
+> >
+> > ---------------------------(end of broadcast)---------------------------
+> > TIP 4: Don't 'kill -9' the postmaster
+>
+> ---------------------------(end of broadcast)---------------------------
+> TIP 3: if posting/reading through Usenet, please send an appropriate
+> subscribe-nomail command to majordomo@postgresql.org so that your
+> message can get through to the mailing list cleanly
+>
+
+ Regards,
+ Oleg
+_____________________________________________________________
+Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
+Sternberg Astronomical Institute, Moscow University (Russia)
+Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
+phone: +007(095)939-16-83, +007(095)939-23-83
+
+
+---------------------------(end of broadcast)---------------------------
+TIP 3: if posting/reading through Usenet, please send an appropriate
+subscribe-nomail command to majordomo@postgresql.org so that your
+message can get through to the mailing list cleanly
+
+From pgsql-hackers-owner+M16412=candle.pha.pa.us=pgman@postgresql.org Mon Dec 10 12:35:01 2001
+Return-path: <pgsql-hackers-owner+M16412=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fBAHZ0Z09772
+ for <pgman@candle.pha.pa.us>; Mon, 10 Dec 2001 12:35:00 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fBAHZ0a11739
+ for <pgman@candle.pha.pa.us>; Mon, 10 Dec 2001 12:35:00 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fBAHRAR20284
+ for <pgman@candle.pha.pa.us>; Mon, 10 Dec 2001 11:32:00 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16412=candle.pha.pa.us=pgman@postgresql.org)
+Received: from snoopy.mohawksoft.com (h0050bf7a618d.ne.mediaone.net [24.218.67.153])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB7DhOm75114
+ for <pgsql-hackers@postgresql.org>; Fri, 7 Dec 2001 08:43:24 -0500 (EST)
+ (envelope-from markw@mohawksoft.com)
+Received: from mohawksoft.com (localhost [127.0.0.1])
+ by snoopy.mohawksoft.com (8.11.6/8.11.6) with ESMTP id fB7De0000437;
+ Fri, 7 Dec 2001 08:40:01 -0500
+Message-ID: <3C10C6B0.865669C1@mohawksoft.com>
+Date: Fri, 07 Dec 2001 08:40:00 -0500
+From: mlw <markw@mohawksoft.com>
+X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.14ext3 i686)
+X-Accept-Language: en
+MIME-Version: 1.0
+To: Oleg Bartunov <oleg@sai.msu.su>
+cc: Joe Conway <joseph.conway@home.com>,
+ "Ross J. Reedstrom" <reedstrm@rice.edu>,
+ "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+References: <Pine.GSO.4.33.0112071035180.20511-100000@ra.sai.msu.su>
+Content-Type: text/plain; charset=us-ascii
+Content-Transfer-Encoding: 7bit
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+The dblink code is a very cool idea.
+
+It got me thinking, what if, just thinking out load here, it was redesigned as
+something a little more grandeous.
+
+Imagine this:
+
+
+select dblink('select * from table', 'table_name', 'db=oracle.test user=chris
+passwd=knight', 1) as t1, dblink('table2_name', 1) as t2
+
+
+Just something to think about.
+
+The first instance of dblink would take 4 parameters: query, table which it
+returns, connect string, and link token.
+
+The second instance of dblink would just take the name of the table which it
+returns and a link token.
+
+The cool bit is the notion that the query string could specify different
+databases or even .DBF libraries.
+
+Just something to think about.
+
+It would REALLY be great if functions could return multiple tuples!
+
+---------------------------(end of broadcast)---------------------------
+TIP 5: Have you checked our extensive FAQ?
+
+http://www.postgresql.org/users-lounge/docs/faq.html
+
+From pgsql-hackers-owner+M16365=candle.pha.pa.us=pgman@postgresql.org Fri Dec 7 12:32:26 2001
+Return-path: <pgsql-hackers-owner+M16365=candle.pha.pa.us=pgman@postgresql.org>
+Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
+ by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB7HWMZ26245
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 12:32:22 -0500 (EST)
+Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
+ by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB7HWLB14472
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 12:32:21 -0500 (EST)
+Received: from postgresql.org (postgresql.org [64.49.215.8])
+ by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB7HSHR01506
+ for <pgman@candle.pha.pa.us>; Fri, 7 Dec 2001 11:29:07 -0600 (CST)
+ (envelope-from pgsql-hackers-owner+M16365=candle.pha.pa.us=pgman@postgresql.org)
+Received: from mx1.relaypoint.net (ns2.generalbroadband.com [64.32.62.5])
+ by postgresql.org (8.11.3/8.11.4) with ESMTP id fB7HQfm90424
+ for <pgsql-hackers@postgresql.org>; Fri, 7 Dec 2001 12:26:42 -0500 (EST)
+ (envelope-from joseph.conway@home.com)
+Received: from [206.19.64.3] (account jconway@wwc.com HELO home.com)
+ by mx1.relaypoint.net (CommuniGate Pro SMTP 3.4.8)
+ with ESMTP id 1722339; Fri, 07 Dec 2001 09:26:46 -0800
+Message-ID: <3C10FBD7.4070602@home.com>
+Date: Fri, 07 Dec 2001 09:26:47 -0800
+From: Joe Conway <joseph.conway@home.com>
+User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.6+) Gecko/20011126
+X-Accept-Language: en-us
+MIME-Version: 1.0
+To: mlw <markw@mohawksoft.com>
+cc: "Ross J. Reedstrom" <reedstrm@rice.edu>,
+ "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
+Subject: Re: [HACKERS] Remote connections?
+References: <3C0FB8B4.382C7736@mohawksoft.com> <20011206140616.C10995@rice.edu> <3C0FD339.6F663329@mohawksoft.com> <3C0FEB2C.70000@home.com> <3C104E38.DA19C867@mohawksoft.com>
+Content-Type: text/plain; charset=us-ascii; format=flowed
+Content-Transfer-Encoding: 7bit
+Precedence: bulk
+Sender: pgsql-hackers-owner@postgresql.org
+Status: OR
+
+mlw wrote:
+
+> Hey this looks really cool. It looks like something I was thinking about doing.
+> I have a couple suggestions that could make it a little better, I hope you will
+> not be offended. (If you want my help, I'll chip in!)
+>
+
+
+Thanks! Suggestions welcomed.
+
+> Why not use a binary cursor? That way native types can slip through without the
+> overhead of conversion.
+>
+
+
+I wasn't sure that would work. Would you create dblink_tok as returning
+opaque then?
+
+
+> Right now you get all rows up front, you may be able to increase overall
+> performance by fetching only a few rows at a time, rather than get everything
+> all at once. (Think on the order of 4 million rows from your remote query!)
+> Execute the commit at the end of processing. There are even some asynchronous
+> functions you may be able to utilize to reduce the I/O bottleneck. Use the
+> synchronous function first, then before you return initiate an asynchronous
+> read. Every successive pass through the function, read the newly arrived tuple,
+> and initiate the next asynchronous read. (The two machine could be processing
+> the query simultaneously, and this could even IMPROVE performance over a single
+> system for heavy duty queries.)
+
+
+Interesting . . . but aren't there some issues with the asynch functions?
+
+>
+> Setup a hash table for field names, rather than requiring field numbers. (Keep
+> field number for efficiency, of course.)
+>
+> You could eliminate having to pass the result pointer around by keeping a
+> static array in your extension. Use something like Oracle's "contains" notation
+> of result number. Where each instantiation of "contains()" and "score()"
+> require an id. i.e. 1,2,3,40 etc. Then hash those numbers into an array. I have
+> some code that does this for a PostgreSQL extension (it implements contains) on
+> my website (pgcontains, under download). It is ugly but works for the most
+> part.
+>
+
+
+I thought about the static array, but I'm not familiar with Oracle
+contains() and score() -- I'm only fluent enough with Oracle to be
+dangerous. Guess I'll have to dig out the books . . .
+
+
+> Seriously, your stuff looks great. I think it could be the beginning of a
+> fairly usable system for my company. Any help you need/want, just let me know.
+>
+
+I am planning to improve dblink during the next release cycle, so I'll
+keep all this in mind (and might take you up on the help offer too!). I
+was hoping we'd have functions returning tuples by now, which would
+improve this extension dramatically. Unfortunately, it sounds like Alex
+won't have time to finish that even for 7.3 :(
+
+Alex, can we get a look at your latest code? Is it any different the
+your last submission to PATCHES?
+
+Joe
+
+
+---------------------------(end of broadcast)---------------------------
+TIP 3: if posting/reading through Usenet, please send an appropriate
+subscribe-nomail command to majordomo@postgresql.org so that your
+message can get through to the mailing list cleanly
+
projects / postgresql / commitdiff