+++ /dev/null
-From pgsql-hackers-owner+M59479@postgresql.org Thu Sep 30 15:55:23 2004
-Return-path: <pgsql-hackers-owner+M59479@postgresql.org>
-Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i8UJtHw26165
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 15:55:19 -0400 (EDT)
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id A4BDD32A219
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 20:55:10 +0100 (BST)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 24195-05 for <pgman@candle.pha.pa.us>;
- Thu, 30 Sep 2004 19:55:08 +0000 (GMT)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by svr1.postgresql.org (Postfix) with ESMTP id 537C532A216
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 20:55:10 +0100 (BST)
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id 333B932A1EF
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Thu, 30 Sep 2004 20:49:20 +0100 (BST)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 21793-04
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Thu, 30 Sep 2004 19:49:09 +0000 (GMT)
-Received: from authenticity.encs.concordia.ca (authenticity-96.encs.concordia.ca [132.205.96.93])
- by svr1.postgresql.org (Postfix) with ESMTP id BEB6A32A156
- for <pgsql-hackers@postgresql.org>; Thu, 30 Sep 2004 20:49:03 +0100 (BST)
-Received: from haida.cs.concordia.ca (IDENT:mokhov@haida.cs.concordia.ca [132.205.64.45])
- by authenticity.encs.concordia.ca (8.12.11/8.12.11) with ESMTP id i8UJn0Xe022202;
- Thu, 30 Sep 2004 15:49:00 -0400
-Date: Thu, 30 Sep 2004 15:49:00 -0400 (EDT)
-From: "Serguei A. Mokhov" <mokhov@cs.concordia.ca>
-To: pgsql-hackers@postgresql.org
-cc: "Serguei A. Mokhov" <mokhov@cs.concordia.ca>
-Subject: [HACKERS] pg_upgrade project: high-level design proposal of in-place upgrade
- facility
-Message-ID: <Pine.LNX.4.58.0409301532390.4685@haida.cs.concordia.ca>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-X-Virus-Scanned: by amavisd-new at hub.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-Virus-Scanned: by amavisd-new at hub.org
-X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on
- candle.pha.pa.us
-X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham
- version=2.61
-Status: OR
-
-Hello dear all,
-
-[Please CC your replies to me as I am on the digest mode]
-
-Here's finally a very high-level design proposal of the pg_upgrade feature
-I was handwaiving a couple of weeks ago. Since, I am almost done with the
-moving, I can allocate some time for this for 8.1/8.2.
-
-If this topic is of interest to you, please read on until the very end
-before flaming or bashing the ideas out. I had designed that thing and
-kept updating (the design) more or less regularly, and also reflected some
-issues from the nearby threads [1] and [2].
-
-This design is very high-level at the moment and is not very detailed. I
-will need to figure out more stuff as I go and design some aspects in
-finer detail. I started to poke around asking for initdb-forcing code
-paths in [3], but got no response so far. But I guess if the general idea
-or, rather, ideas accepted I will insist on more information more
-aggressively :) if I can't figure something out for myself.
-
-[1] http://archives.postgresql.org/pgsql-hackers/2004-09/msg00000.php
-[2] http://archives.postgresql.org/pgsql-hackers/2004-09/msg00382.php
-[3] http://archives.postgresql.org/pgsql-hackers/2004-08/msg01594.php
-
-Comments are very welcome, especially _*CONSTRUCTIVE*_...
-
-Thank you, and now sit back and read...
-
-CONTENTS:
-=========
-
-1. The Need
-1. Utilities and User's View of the pg_upgrade Feature
-2. Storage Management
- - Storage Managers and the smgr API
-3. Source Code Maintenance Aspects
-2. The Upgrade Sequence
-4. Proposed Implementation Plan
- - initdb() API
- - upgrade API
-
-
-1. The Need
------------
-
-It's been a problem for PG for quite awhile now to have a less painful
-upgrade procedure with every new revision of PostgreSQL, so the
-dump/restore sequence is required. That can take a while for a production
-DB, while keeping it offline. The new replication-related solutions, such
-as Slony I, pg_pool, and others can remedy the problem somewhat, but
-require to roughly double the storage requirements of a given database
-while replicating from the older server to a newer one.
-
-The proposed implementation of an in-server pg_upgrade facility attempts
-to address both issues at the same time -- a possibility to keep the
-server running and upgrading lazily w/o doubling the storage requirements
-(there will be some extra disk space taken, but far from doubling the
-size). The in-process upgrade will not take much of down time and won't
-require that much memory/disk/network resources as replication solutions
-do.
-
-
-Prerequisites
--------------
-
-Ideally, the (maybe not so anymore) ambitious goal is to simply be able to
-"drop in" the new binaries of the new server and kick off on the older
-version of data files. I think is this feasible now a lot more than before
-since we have those things available, which should ease up the
-implementation:
-
- - bgwriter
- - pg_autovacuum (the one to be integrated into the backend in 8.1)
- - smgr API for pluggable storage managers
- - initdb in C
- - ...
-
-initdb in C, bgwriter and pg_autovacuum, and pluggable storage manager
-have made the possibility of creation of the Upgrade Subsystem for
-PostgreSQL to be something more reasonable, complete, feasible, and sane
-to a point.
-
-
-Utilities and the User's (DBA) View of the Feature
---------------------------------------------------
-
-Two instances exist:
-
- pg_upgrade (in C)
-
- A standalone utility to upgrade the binary on-disk format from one
- version to another when the database is offline.
- We should always have this as an option.
- pg_upgrade will accept sub/super set of pg_dump(all)/pg_restore
- options that do not require a connection. I haven't
- thought through this in detail yet.
-
- pg_autoupgrade
-
- a postgres subprocess, modeled after bgwriter and pg_autovacuum
- daemons. This will work when the database system is running
- on old data directory, and lazily converting relations to the new
- format.
-
-pg_autoupgrade daemon can be triggered by the following events in addition
-to the lazy upgrade process:
-
- "SQL" level: UPGRADE <ALL | relation_name [, relation_name]> [NOW | time]
-
-While the database won't be offline running over older database files,
-SELECT/read-only queries would be allowed using older storage managers*.
-Any write operation on old data will act using write-invalidate approach
-that will force the upgrade the affected relations to the new format to be
-scheduled after the relation-in-progress.
-
-(* See the "Storage Management" section.)
-
-Availability of the relations while upgrade is in progress is likely to be
-the same as in VACUUM FULL for that relation, i.e. the entire relation is
-locked until the upgrade is complete. Maybe we could optimize that by
-locking only particular pages of relations, I have to figure that out.
-
-The upgrade of indices can be done using REINDEX, which seems far less
-complicated than trying to convert its on-disk representation. This has
-to be done after the relation is converted. Alternatively, the index
-upgrade can simply be done by "CREATE INDEX" after the upgrade of
-relations.
-
-The relations to be upgraded are ordered according to some priority, e.g.
-system relations being first, then user-owned relations. System relations
-upgrade is forced upon the postmaster startup, and then user relations are
-processed lazily.
-
-So, in a sense, pg_autoupgrade will act like a proxy choosing appropriate
-storage manager (like a driver) between the new server and the old data
-file upgrading them on-demand. For that purpose we might need to add a
-pg_upgradeproxy to intercept backend requests and use appropriate storage
-manager. There will be one proxy process per backend.
-
-
-Storage Management
-==================
-
-Somebody has made a possibility to plug a different storage manager in
-postgres and we even had two of them at some point . for the magnetic disk
-and the main memory. The main memory one is gone, but the smgr API is
-still there. Some were dubious why we would ever need another third-party
-storage manager, but here I propose to "plug in" storage managers from the
-older Postgres versions itself! Here is where the pluggable storage
-manager API would be handy once fully resurrected. Instead of trying to
-plug some third party storage managers it will primarily be used by the
-storage managers of different versions of Postgres.
-
-We can take the storage manager code from the past maintenance releases,
-namely 6.5.3, 7.0.3, 7.1.3, 7.2.5, 7.3.7, 7.4.5, and 8.0, and arrange them
-in appropriate fashion and have them implement the API properly. Anyone
-can contribute a storage manager as they see fit, there's no need to get
-them all at once. As a trial implementation I will try to do the last
-three or four maybe.
-
-
-Where to put relations being upgraded?
---------------------------------------
-
-At the beginning of the upgrade process if pg detects the old version of
-data files, it moves them under $PGDATA/<ver>, and keeps the old relations
-there until upgraded. The relations to be upgraded will be kept in the
-pg_upgrade_catalog. Once all relations upgraded, the <ver> directory is
-removed and the auto and proxy processes are shut down. The contents of
-the pg_upgrade_catalog emptied. The only issue remains is how to deal
-with tablespaces (or LOCATION in 7.* releases) elsewhere .- this can
-probably be addressed in the similar fashion, but having a
-/my/tablespace/<ver> directory.
-
-Source Code Maintenance
-=======================
-
-Now, after the above some of you may get scared on the amount of similar
-code to possibly maintain in all those storage managers, but in reality
-they would require as much maintenance as the corresponding releases do
-get back-patched in that code area, and some are not being maintained for
-quite some time already. Plus, I should be around to maintain it, should
-this become realized.
-
-Release-time Maintenance
-------------------------
-
-For maintenance of pg_upgrade itself, one will have to fork out a new
-storage manager from the previous stable release and "register" it within
-the system. Alternatively, the new storage manager can be forked when the
-new release cycle begins. Additionally, a pg_upgrade version has to be
-added implementing the API steps outlined in the pg_upgrade API section.
-
-
-Implementation Steps
-====================
-
-To materialize the above idea, I'd proceed as follows:
-
-*) Provide the initdb() API (quick)
-
-*) Resurrect the pluggable storage manager API to be usable for the
- purpose.
-
-*) Document it
-
-*) Implement pg_upgrade API for 8.0 and 7.4.5.
-
-*) Extract 8.0 and 7.4.5 storage managers and have them implement the API
- as a proof of concept. Massage the API as needed.
-
-*) Document the process of adding new storage managers and pg_upgrade
- drivers.
-
-*) Extract other versions storage managers.
-
-
-pg_upgrade sequence
--------------------
-
-pg_upgrade API for the steps below to update for the next release.
-
-What to do with WAL?? Maybe upgrade can simply be done using WAL replay
-with old WAL manager? Not, fully, because not everything is in WAL, but
-some WAL recovery maybe needed in case the server was not shutdown cleanly
-before the upgrade.
-
-pg_upgrade will proceed as follows:
-
-- move PGDATA to PGDATA/<major pg version>
-- move tablespaces likewise
-- optional recovery from WAL in case old server was not shutdown properly
--? Shall I upgrade PITR logs of 8.x??? So one can recover to a
- point-in-time in the upgraded database?
-- CLUSTER all old data
-- ANALYZE all old data
-- initdb() new system catalogs
-- Merge in modifications from old system catalogs
-- upgrade schemas/users
- -- variations
-- upgrade user relations
-
-Upgrade API:
-------------
-
-First draft, to be refined multiple times, but to convey the ideas behind:
-
-moveData()
- movePGData()
- moveTablespaces() 8.0+
- moveDbLocation() < 8.0
-
-preliminaryRecovery()
- - WAL??
- - PITR 8.0+??
-
-preliminaryCleanup()
- CLUSTER -- recover some dead space
- ANALYZE -- gives us stats
-
-upgradeSystemInfo()
- initdb()
- mergeOldCatalogs()
- mergeOldTemplates()
-
-upgradeUsers()
-
-upgradeSchemas()
- - > 7.2, else NULL
-
-upgradeUserRelations()
- upgradeIndices()
- DROP/CREATE
-
-upgradeInit()
-{
-
-}
-
-The main body in pseudocode:
-
-upgradeLoop()
-{
- moveData();
- preliminaryRecovery();
- preliminaryCleanup();
- upgradeSystemInfo();
- upgradeUsers();
- upgradeSchemas();
- upgradeUserRelations();
-}
-
-Something along these lines the API would be:
-
-typedef struct t_upgrade
-{
- bool (*moveData) (void);
- bool (*preliminaryRecovery) (void); /* may be NULL */
- bool (*preliminaryCleanup) (void); /* may be NULL */
- bool (*upgradeSystemInfo) (void); /* may be NULL */
- bool (*upgradeUsers) (void); /* may be NULL */
- bool (*upgradeSchemas) (void); /* may be NULL */
- bool (*upgradeUserRelations) (void); /* may be NULL */
-} t_upgrade;
-
-
-The above sequence is executed by either pg_upgrade utility uninterrupted
-or by the pg_autoupgrade daemon. In the former the upgrade priority is
-simply by OID, in the latter also, but can be overridden by the user using
-the UPGRADE command to schedule relations upgrade, write operation can
-also change such schedule, with user's selected choice to be first. The
-more write requests a relation receives while in the upgrade queue, its
-priority increases; thus, the relation with most hits is on top. In case
-of tie, OID is the decision mark.
-
-Some issues to look into:
-
-- catalog merger
-- a crash in the middle of upgrade
-- PITR logs for 8.x+
-- ...
-
-Flames and Handwaiving
-----------------------
-
-Okay, flame is on, but before you flame, mind you, this is a very initial
-version of the design. Some of the ideas may seem far fetched, the
-contents may seem messy, but I believe it's now more doable than ever and
-I am willing to put effort in it for the next release or two and then
-maintain it afterwards. It's not going to be done in one shot maybe, but
-incrementally, using input, feedback, and hints from you, guys.
-
-Thank you for reading till this far :-) I.d like to hear from you if any
-of this made sense to you.
-
-Truly yours,
-
---
-Serguei A. Mokhov | /~\ The ASCII
-Computer Science Department | \ / Ribbon Campaign
-Concordia University | X Against HTML
-Montreal, Quebec, Canada | / \ Email!
-
----------------------------(end of broadcast)---------------------------
-TIP 5: Have you checked our extensive FAQ?
-
- http://www.postgresql.org/docs/faqs/FAQ.html
-
-From pgsql-hackers-owner+M59486@postgresql.org Thu Sep 30 16:39:54 2004
-Return-path: <pgsql-hackers-owner+M59486@postgresql.org>
-Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i8UKdrw02740
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 16:39:53 -0400 (EDT)
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id EF25F329E3B
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 21:39:48 +0100 (BST)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 38456-02 for <pgman@candle.pha.pa.us>;
- Thu, 30 Sep 2004 20:39:46 +0000 (GMT)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by svr1.postgresql.org (Postfix) with ESMTP id 88673329C6B
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 21:39:48 +0100 (BST)
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id AA522329DAE
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Thu, 30 Sep 2004 21:37:43 +0100 (BST)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 38130-02
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Thu, 30 Sep 2004 20:37:39 +0000 (GMT)
-Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130])
- by svr1.postgresql.org (Postfix) with ESMTP id 846E9329C63
- for <pgsql-hackers@postgresql.org>; Thu, 30 Sep 2004 21:37:39 +0100 (BST)
-Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
- by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id i8UKa3jk025254;
- Thu, 30 Sep 2004 16:36:03 -0400 (EDT)
-To: "Serguei A. Mokhov" <mokhov@cs.concordia.ca>
-cc: pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] pg_upgrade project: high-level design proposal of in-place upgrade facility
-In-Reply-To: <Pine.LNX.4.58.0409301532390.4685@haida.cs.concordia.ca>
-References: <Pine.LNX.4.58.0409301532390.4685@haida.cs.concordia.ca>
-Comments: In-reply-to "Serguei A. Mokhov" <mokhov@cs.concordia.ca>
- message dated "Thu, 30 Sep 2004 15:49:00 -0400"
-Date: Thu, 30 Sep 2004 16:36:02 -0400
-Message-ID: <25253.1096576562@sss.pgh.pa.us>
-From: Tom Lane <tgl@sss.pgh.pa.us>
-X-Virus-Scanned: by amavisd-new at hub.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-Virus-Scanned: by amavisd-new at hub.org
-X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on
- candle.pha.pa.us
-X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham
- version=2.61
-Status: OR
-
-"Serguei A. Mokhov" <mokhov@cs.concordia.ca> writes:
-> Comments are very welcome, especially _*CONSTRUCTIVE*_...
-
-This is fundamentally wrong, because you are assigning the storage
-manager functionality that it does not have. In particular, the
-storage manager knows nothing of the contents or format of the files
-it is managing, and so you can't realistically expect to use the smgr
-switch as a way to support access to tables with different internal
-formats. The places that change in interesting ways across versions
-are usually far above the smgr switch.
-
-I don't believe in the idea of incremental "lazy" upgrades very much
-either. It certainly will not work on the system catalogs --- you have
-to convert those in a big-bang fashion, because how are you going to
-find the other stuff otherwise? And the real problem with it IMHO is
-that if something goes wrong partway through the process, you're in
-deep trouble because you have no way to back out. You can't just revert
-to the old version because it won't understand your data, and your old
-backups that are compatible with it are now out of date. If there are
-going to be any problems, you really need to find out about them
-immediately while your old backups are still current, not in a "lazy"
-fashion.
-
-The design we'd pretty much all bought into six months ago involved
-being able to do in-place upgrades when the format/contents of user
-relations and indexes is not changing. All you'd have to do is dump
-and restore the schema data (system catalogs) which is a reasonably
-short process even on a large DB, so the big-bang nature of the
-conversion isn't a problem. Admittedly this will not work for every
-single upgrade, but we had agreed that we could schedule upgrades
-so that the majority of releases do not change user data. Historically
-that's been mostly true anyway, even without any deliberate attempt to
-group user-data-changing features together.
-
-I think the last major discussion about it started here:
-http://archives.postgresql.org/pgsql-hackers/2003-12/msg00379.php
-(I got distracted by other stuff and never did the promised work,
-but I still think the approach is sound.)
-
- regards, tom lane
-
----------------------------(end of broadcast)---------------------------
-TIP 2: you can get off all lists at once with the unregister command
- (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
-
-From pgsql-hackers-owner+M59497@postgresql.org Thu Sep 30 17:44:44 2004
-Return-path: <pgsql-hackers-owner+M59497@postgresql.org>
-Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id i8ULihw11377
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 17:44:43 -0400 (EDT)
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id D0A6B329E2E
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 22:44:38 +0100 (BST)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 55636-04 for <pgman@candle.pha.pa.us>;
- Thu, 30 Sep 2004 21:44:36 +0000 (GMT)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by svr1.postgresql.org (Postfix) with ESMTP id 6ED67329DFC
- for <pgman@candle.pha.pa.us>; Thu, 30 Sep 2004 22:44:38 +0100 (BST)
-X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id 040D2329E2E
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Thu, 30 Sep 2004 22:42:24 +0100 (BST)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 55767-04
- for <pgsql-hackers-postgresql.org@localhost.postgresql.org>;
- Thu, 30 Sep 2004 21:42:18 +0000 (GMT)
-Received: from authenticity.encs.concordia.ca (authenticity-96.encs.concordia.ca [132.205.96.93])
- by svr1.postgresql.org (Postfix) with ESMTP id E3A4D329DFC
- for <pgsql-hackers@postgresql.org>; Thu, 30 Sep 2004 22:42:19 +0100 (BST)
-Received: from haida.cs.concordia.ca (IDENT:mokhov@haida.cs.concordia.ca [132.205.64.45])
- by authenticity.encs.concordia.ca (8.12.11/8.12.11) with ESMTP id i8ULgJrP001049;
- Thu, 30 Sep 2004 17:42:19 -0400
-Date: Thu, 30 Sep 2004 17:42:19 -0400 (EDT)
-From: "Serguei A. Mokhov" <mokhov@cs.concordia.ca>
-To: Tom Lane <tgl@sss.pgh.pa.us>
-cc: pgsql-hackers@postgresql.org
-Subject: Re: [HACKERS] pg_upgrade project: high-level design proposal of
-In-Reply-To: <25253.1096576562@sss.pgh.pa.us>
-Message-ID: <Pine.LNX.4.58.0409301725550.4685@haida.cs.concordia.ca>
-References: <Pine.LNX.4.58.0409301532390.4685@haida.cs.concordia.ca>
- <25253.1096576562@sss.pgh.pa.us>
-MIME-Version: 1.0
-Content-Type: TEXT/PLAIN; charset=US-ASCII
-X-Virus-Scanned: by amavisd-new at hub.org
-X-Mailing-List: pgsql-hackers
-Precedence: bulk
-Sender: pgsql-hackers-owner@postgresql.org
-X-Virus-Scanned: by amavisd-new at hub.org
-X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on
- candle.pha.pa.us
-X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham
- version=2.61
-Status: OR
-
-On Thu, 30 Sep 2004, Tom Lane wrote:
-
-> Date: Thu, 30 Sep 2004 16:36:02 -0400
->
-> "Serguei A. Mokhov" <mokhov@cs.concordia.ca> writes:
-> > Comments are very welcome, especially _*CONSTRUCTIVE*_...
->
-> This is fundamentally wrong, because you are assigning the storage
-> manager functionality that it does not have. In particular, the
-
-Maybe, that's why I was asking of all init-db forcing paths, so I can go
-level above smgr to upgrade stuff, let say in access/ and other parts. I
-did ask that before and never got a reply. So the concept of "Storage
-Managers" may and will go well beyond the smgt API. That's the design
-refinement stage.
-
-> I don't believe in the idea of incremental "lazy" upgrades very much
-> either. It certainly will not work on the system catalogs --- you have
-> to convert those in a big-bang fashion,
-
-I never proposed to do that to system catalogs, on the contrary, I said
-the system catalogs are to be upgraded upon the postmaster startup. only
-user relations are upgraded lazily:
-
-> The relations to be upgraded are ordered according to some priority,
-> e.g. system relations being first, then user-owned relations. System
-> relations upgrade is forced upon the postmaster startup, and then user
-> relations are processed lazily.
-
-So looks like we don't disagree here :)
-
-> The design we'd pretty much all bought into six months ago involved
-> being able to do in-place upgrades when the format/contents of user
-> relations and indexes is not changing. All you'd have to do is dump and
-> restore the schema data (system catalogs) which is a reasonably short
-> process even on a large DB, so the big-bang nature of the conversion
-> isn't a problem. Admittedly this will not work for every single
-> upgrade, but we had agreed that we could schedule upgrades so that the
-> majority of releases do not change user data. Historically that's been
-> mostly true anyway, even without any deliberate attempt to group
-> user-data-changing features together.
-
-Annoyingly enough, they still do change.
-
-> I think the last major discussion about it started here:
-> http://archives.postgresql.org/pgsql-hackers/2003-12/msg00379.php
-> (I got distracted by other stuff and never did the promised work,
-> but I still think the approach is sound.)
-
-I'll go over that discussion and maybe will combine useful ideas together.
-I'll open a pgfoundry project to develop it there and then will submit for
-evaluation UNLESS you reserved it for yourself, Tom, to fullfill the
-promise... If anybody objects the pgfoundry idea to test the concepts,
-I'll apply for a project there.
-
-Thank you for the comments!
-
-> regards, tom lane
->
-
---
-Serguei A. Mokhov | /~\ The ASCII
-Computer Science Department | \ / Ribbon Campaign
-Concordia University | X Against HTML
-Montreal, Quebec, Canada | / \ Email!
-
----------------------------(end of broadcast)---------------------------
-TIP 7: don't forget to increase your free space map settings
-
projects / postgresql / commitdiff