Fix bit-rot in pg_upgrade's test.sh, and improve documentation.

Doing a cross-version upgrade test with test.sh evidently hasn't been
tested since circa 9.2, because the script lacked case branches for
old-version servers newer than 9.1.  Future-proof that a bit, and
clean up breakage induced by our recent drop of V0 function call
protocol (namely that oldstyle_length() isn't in the regression
suite anymore).

(This isn't enough to make the test work perfectly cleanly across
versions, but at least it finishes and provides dump files that
you can diff manually.  One issue I didn't touch is that we might
want to execute the "reindex_hash.sql" file in the new DB before
dumping it, so that the hash indexes don't vanish from the dump.)

Improve the TESTING doc file: put the tl;dr version at the top not
the bottom, and bring its explanation of how to run a cross-version
test up to speed, since the installcheck target isn't there and won't
be resurrected.  Improve the comment in the Makefile about why not.

In passing, teach .gitignore and "make clean" about a couple more
junk output files.

Discussion: https://postgr.es/m/14058.1496892482@sss.pgh.pa.us

diff --git a/src/bin/pg_upgrade/.gitignore b/src/bin/pg_upgrade/.gitignore
index d24ec60184fe7ac69c5c0591a858134f76448fb1..6fb644de7a969dbac00d7b6d89af487e0853f11a 100644 (file)
--- a/src/bin/pg_upgrade/.gitignore
+++ b/src/bin/pg_upgrade/.gitignore
@@ -4,5 +4,7 @@
 /delete_old_cluster.sh
 /analyze_new_cluster.bat
 /delete_old_cluster.bat
+/reindex_hash.sql
+/loadable_libraries.txt
 /log/
 /tmp_check/

diff --git a/src/bin/pg_upgrade/Makefile b/src/bin/pg_upgrade/Makefile
index 88232887081dd06dca0edaf586e8ce70dee0c72b..d252e08d37a0c6c10a28ec1a129c4e64e4f507a5 100644 (file)
--- a/src/bin/pg_upgrade/Makefile
+++ b/src/bin/pg_upgrade/Makefile
@@ -32,12 +32,12 @@ uninstall:
 clean distclean maintainer-clean:
        rm -f pg_upgrade$(X) $(OBJS)
        rm -rf analyze_new_cluster.sh delete_old_cluster.sh log/ tmp_check/ \
+              loadable_libraries.txt reindex_hash.sql \
               pg_upgrade_dump_globals.sql \
               pg_upgrade_dump_*.custom pg_upgrade_*.log
 
 check: test.sh all
        MAKE=$(MAKE) bindir=$(bindir) libdir=$(libdir) EXTRA_REGRESS_OPTS="$(EXTRA_REGRESS_OPTS)" $(SHELL) $< --install
 
-# disabled because it upsets the build farm
-#installcheck: test.sh
-#      MAKE=$(MAKE) bindir=$(bindir) libdir=$(libdir) $(SHELL) $<
+# installcheck is not supported because there's no meaningful way to test
+# pg_upgrade against a single already-running server

diff --git a/src/bin/pg_upgrade/TESTING b/src/bin/pg_upgrade/TESTING
index 4ecfc5798e01d3ad841333ccd117ebf144869ec5..6831f679f6683620fc70560a4d38620d154006f2 100644 (file)
--- a/src/bin/pg_upgrade/TESTING
+++ b/src/bin/pg_upgrade/TESTING
@@ -1,3 +1,30 @@
+THE SHORT VERSION
+-----------------
+
+On non-Windows machines, you can execute the testing process
+described below by running
+       make check
+in this directory.  This will run the shell script test.sh, performing
+an upgrade from the version in this source tree to a new instance of
+the same version.
+
+To test an upgrade from a different version, you must have a built
+source tree for the old version as well as this version, and you
+must have done "make install" for both versions.  Then do:
+
+export oldsrc=...somewhere/postgresql  (old version's source tree)
+export oldbindir=...otherversion/bin   (old version's installed bin dir)
+export bindir=...thisversion/bin       (this version's installed bin dir)
+export libdir=...thisversion/lib       (this version's installed lib dir)
+sh test.sh
+
+In this case, you will have to manually eyeball the resulting dump
+diff for version-specific differences, as explained below.
+
+
+DETAILS
+-------
+
 The most effective way to test pg_upgrade, aside from testing on user
 data, is by upgrading the PostgreSQL regression database.
 
@@ -7,7 +34,7 @@ specific to each major version of Postgres.
 
 Here are the steps needed to create a regression database dump file:
 
-1)  Create and populate the regression database in the old cluster
+1)  Create and populate the regression database in the old cluster.
     This database can be created by running 'make installcheck' from
     src/test/regression.
 
@@ -60,22 +87,3 @@ steps:
 
 7)  Diff the regression database dump file with the regression dump
     file loaded into the old server.
-
-The shell script test.sh in this directory performs more or less this
-procedure.  You can invoke it by running
-
-    make check
-
-or by running
-
-    make installcheck
-
-if "make install" (or "make install-world") were done beforehand.
-When invoked without arguments, it will run an upgrade from the
-version in this source tree to a new instance of the same version.  To
-test an upgrade from a different version, invoke it like this:
-
-    make installcheck oldbindir=...otherversion/bin oldsrc=...somewhere/postgresql
-
-In this case, you will have to manually eyeball the resulting dump
-diff for version-specific differences, as explained above.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index 841da034b09766615f7023906c9b4e561ed3a2ca..f4556341f324368b99ea0723498b8174aee0aa4f 100644 (file)
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -170,18 +170,32 @@ createdb "$dbname2" || createdb_status=$?
 createdb "$dbname3" || createdb_status=$?
 
 if "$MAKE" -C "$oldsrc" installcheck; then
-       pg_dumpall --no-sync -f "$temp_root"/dump1.sql || pg_dumpall1_status=$?
+       oldpgversion=`psql -X -A -t -d regression -c "SHOW server_version_num"`
+
+       # before dumping, get rid of objects not existing in later versions
        if [ "$newsrc" != "$oldsrc" ]; then
-               oldpgversion=`psql -X -A -t -d regression -c "SHOW server_version_num"`
                fix_sql=""
                case $oldpgversion in
                        804??)
-                               fix_sql="UPDATE pg_proc SET probin = replace(probin::text, '$oldsrc', '$newsrc')::bytea WHERE probin LIKE '$oldsrc%'; DROP FUNCTION public.myfunc(integer);"
+                               fix_sql="DROP FUNCTION public.myfunc(integer); DROP FUNCTION public.oldstyle_length(integer, text);"
                                ;;
-                       900??)
-                               fix_sql="SET bytea_output TO escape; UPDATE pg_proc SET probin = replace(probin::text, '$oldsrc', '$newsrc')::bytea WHERE probin LIKE '$oldsrc%';"
+                       *)
+                               fix_sql="DROP FUNCTION public.oldstyle_length(integer, text);"
+                               ;;
+               esac
+               psql -X -d regression -c "$fix_sql;" || psql_fix_sql_status=$?
+       fi
+
+       pg_dumpall --no-sync -f "$temp_root"/dump1.sql || pg_dumpall1_status=$?
+
+       if [ "$newsrc" != "$oldsrc" ]; then
+               # update references to old source tree's regress.so etc
+               fix_sql=""
+               case $oldpgversion in
+                       804??)
+                               fix_sql="UPDATE pg_proc SET probin = replace(probin::text, '$oldsrc', '$newsrc')::bytea WHERE probin LIKE '$oldsrc%';"
                                ;;
-                       901??)
+                       *)
                                fix_sql="UPDATE pg_proc SET probin = replace(probin, '$oldsrc', '$newsrc') WHERE probin LIKE '$oldsrc%';"
                                ;;
                esac