From: Tom Lane Date: Thu, 8 Jun 2017 17:48:27 +0000 (-0400) Subject: Fix bit-rot in pg_upgrade's test.sh, and improve documentation. X-Git-Tag: REL_10_BETA2~203 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=5bab1985dfc25eecf4b098145789955c0b246160;p=postgresql Fix bit-rot in pg_upgrade's test.sh, and improve documentation. Doing a cross-version upgrade test with test.sh evidently hasn't been tested since circa 9.2, because the script lacked case branches for old-version servers newer than 9.1. Future-proof that a bit, and clean up breakage induced by our recent drop of V0 function call protocol (namely that oldstyle_length() isn't in the regression suite anymore). (This isn't enough to make the test work perfectly cleanly across versions, but at least it finishes and provides dump files that you can diff manually. One issue I didn't touch is that we might want to execute the "reindex_hash.sql" file in the new DB before dumping it, so that the hash indexes don't vanish from the dump.) Improve the TESTING doc file: put the tl;dr version at the top not the bottom, and bring its explanation of how to run a cross-version test up to speed, since the installcheck target isn't there and won't be resurrected. Improve the comment in the Makefile about why not. In passing, teach .gitignore and "make clean" about a couple more junk output files. Discussion: https://postgr.es/m/14058.1496892482@sss.pgh.pa.us --- diff --git a/src/bin/pg_upgrade/.gitignore b/src/bin/pg_upgrade/.gitignore index d24ec60184..6fb644de7a 100644 --- a/src/bin/pg_upgrade/.gitignore +++ b/src/bin/pg_upgrade/.gitignore @@ -4,5 +4,7 @@ /delete_old_cluster.sh /analyze_new_cluster.bat /delete_old_cluster.bat +/reindex_hash.sql +/loadable_libraries.txt /log/ /tmp_check/ diff --git a/src/bin/pg_upgrade/Makefile b/src/bin/pg_upgrade/Makefile index 8823288708..d252e08d37 100644 --- a/src/bin/pg_upgrade/Makefile +++ b/src/bin/pg_upgrade/Makefile @@ -32,12 +32,12 @@ uninstall: clean distclean maintainer-clean: rm -f pg_upgrade$(X) $(OBJS) rm -rf analyze_new_cluster.sh delete_old_cluster.sh log/ tmp_check/ \ + loadable_libraries.txt reindex_hash.sql \ pg_upgrade_dump_globals.sql \ pg_upgrade_dump_*.custom pg_upgrade_*.log check: test.sh all MAKE=$(MAKE) bindir=$(bindir) libdir=$(libdir) EXTRA_REGRESS_OPTS="$(EXTRA_REGRESS_OPTS)" $(SHELL) $< --install -# disabled because it upsets the build farm -#installcheck: test.sh -# MAKE=$(MAKE) bindir=$(bindir) libdir=$(libdir) $(SHELL) $< +# installcheck is not supported because there's no meaningful way to test +# pg_upgrade against a single already-running server diff --git a/src/bin/pg_upgrade/TESTING b/src/bin/pg_upgrade/TESTING index 4ecfc5798e..6831f679f6 100644 --- a/src/bin/pg_upgrade/TESTING +++ b/src/bin/pg_upgrade/TESTING @@ -1,3 +1,30 @@ +THE SHORT VERSION +----------------- + +On non-Windows machines, you can execute the testing process +described below by running + make check +in this directory. This will run the shell script test.sh, performing +an upgrade from the version in this source tree to a new instance of +the same version. + +To test an upgrade from a different version, you must have a built +source tree for the old version as well as this version, and you +must have done "make install" for both versions. Then do: + +export oldsrc=...somewhere/postgresql (old version's source tree) +export oldbindir=...otherversion/bin (old version's installed bin dir) +export bindir=...thisversion/bin (this version's installed bin dir) +export libdir=...thisversion/lib (this version's installed lib dir) +sh test.sh + +In this case, you will have to manually eyeball the resulting dump +diff for version-specific differences, as explained below. + + +DETAILS +------- + The most effective way to test pg_upgrade, aside from testing on user data, is by upgrading the PostgreSQL regression database. @@ -7,7 +34,7 @@ specific to each major version of Postgres. Here are the steps needed to create a regression database dump file: -1) Create and populate the regression database in the old cluster +1) Create and populate the regression database in the old cluster. This database can be created by running 'make installcheck' from src/test/regression. @@ -60,22 +87,3 @@ steps: 7) Diff the regression database dump file with the regression dump file loaded into the old server. - -The shell script test.sh in this directory performs more or less this -procedure. You can invoke it by running - - make check - -or by running - - make installcheck - -if "make install" (or "make install-world") were done beforehand. -When invoked without arguments, it will run an upgrade from the -version in this source tree to a new instance of the same version. To -test an upgrade from a different version, invoke it like this: - - make installcheck oldbindir=...otherversion/bin oldsrc=...somewhere/postgresql - -In this case, you will have to manually eyeball the resulting dump -diff for version-specific differences, as explained above. diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh index 841da034b0..f4556341f3 100644 --- a/src/bin/pg_upgrade/test.sh +++ b/src/bin/pg_upgrade/test.sh @@ -170,18 +170,32 @@ createdb "$dbname2" || createdb_status=$? createdb "$dbname3" || createdb_status=$? if "$MAKE" -C "$oldsrc" installcheck; then - pg_dumpall --no-sync -f "$temp_root"/dump1.sql || pg_dumpall1_status=$? + oldpgversion=`psql -X -A -t -d regression -c "SHOW server_version_num"` + + # before dumping, get rid of objects not existing in later versions if [ "$newsrc" != "$oldsrc" ]; then - oldpgversion=`psql -X -A -t -d regression -c "SHOW server_version_num"` fix_sql="" case $oldpgversion in 804??) - fix_sql="UPDATE pg_proc SET probin = replace(probin::text, '$oldsrc', '$newsrc')::bytea WHERE probin LIKE '$oldsrc%'; DROP FUNCTION public.myfunc(integer);" + fix_sql="DROP FUNCTION public.myfunc(integer); DROP FUNCTION public.oldstyle_length(integer, text);" ;; - 900??) - fix_sql="SET bytea_output TO escape; UPDATE pg_proc SET probin = replace(probin::text, '$oldsrc', '$newsrc')::bytea WHERE probin LIKE '$oldsrc%';" + *) + fix_sql="DROP FUNCTION public.oldstyle_length(integer, text);" + ;; + esac + psql -X -d regression -c "$fix_sql;" || psql_fix_sql_status=$? + fi + + pg_dumpall --no-sync -f "$temp_root"/dump1.sql || pg_dumpall1_status=$? + + if [ "$newsrc" != "$oldsrc" ]; then + # update references to old source tree's regress.so etc + fix_sql="" + case $oldpgversion in + 804??) + fix_sql="UPDATE pg_proc SET probin = replace(probin::text, '$oldsrc', '$newsrc')::bytea WHERE probin LIKE '$oldsrc%';" ;; - 901??) + *) fix_sql="UPDATE pg_proc SET probin = replace(probin, '$oldsrc', '$newsrc') WHERE probin LIKE '$oldsrc%';" ;; esac