-<!-- $PostgreSQL: pgsql/doc/src/sgml/backup.sgml,v 2.82 2006/08/06 03:53:43 tgl Exp $ -->
+<!-- $PostgreSQL: pgsql/doc/src/sgml/backup.sgml,v 2.83 2006/08/17 23:04:02 tgl Exp $ -->
<chapter id="backup">
<title>Backup and Restore</title>
the archiving will be done. Instead, <productname>PostgreSQL</> lets
the administrator specify a shell command to be executed to copy a
completed segment file to wherever it needs to go. The command could be
- as simple as a <application>cp</>, or it could invoke a complex shell
+ as simple as a <literal>cp</>, or it could invoke a complex shell
script — it's all up to you.
</para>
it is working as you intend.
</para>
- <para>
- If you are concerned about being able to recover right up to the
- current instant, you may want to take additional steps to ensure that
- the current, partially-filled WAL segment is also copied someplace.
- This is particularly important if your server generates only little WAL
- traffic (or has slack periods where it does so), since it could take a
- long time before a WAL segment file is completely filled and ready to
- archive. One possible way to handle this is to set up a
- <application>cron</> job that periodically (once a minute, perhaps)
- identifies the current WAL segment file and saves it someplace safe.
- Then the combination of the archived WAL segments and the saved current
- segment will be enough to ensure you can always restore to within a
- minute of current time. This behavior is not presently built into
- <productname>PostgreSQL</> because we did not want to complicate the
- definition of the <xref linkend="guc-archive-command"> by requiring it
- to keep track of successively archived, but different, copies of the
- same WAL file. The <xref linkend="guc-archive-command"> is only
- invoked on completed WAL segments. Except in the case of retrying a
- failure, it will be called only once for any given file name.
- </para>
-
- <para>
- Another way to limit your exposure to data loss is to call
- <function>pg_switch_xlog()</> periodically, such as once a minute.
- This function forces the current WAL segment file to be completed
- and made available to the archiving command. This approach does
- not work well for extremely short update intervals, however, since
- copying a new 16MB segment file every few seconds is expensive.
- </para>
-
<para>
In writing your archive command, you should assume that the file names to
be archived may be up to 64 characters long and may contain any
<xref linkend="runtime-config-file-locations"> for how to relocate the
configuration files.
</para>
+
+ <para>
+ The archive command is only invoked on completed WAL segments. Hence,
+ if your server generates only little WAL traffic (or has slack periods
+ where it does so), there could be a long delay between the completion
+ of a transaction and its safe recording in archive storage. To put
+ a limit on how old unarchived data can be, you can set
+ <xref linkend="guc-archive-timeout"> to force the server to switch
+ to a new WAL segment file at least that often. Note that archived
+ files that are ended early due to a forced switch are still the same
+ length as completely full files. It is therefore unwise to set a very
+ short <varname>archive_timeout</> — it will bloat your archive
+ storage. <varname>archive_timeout</> settings of a minute or so are
+ usually reasonable.
+ </para>
+
+ <para>
+ Also, you can force a segment switch manually with
+ <function>pg_switch_xlog()</>,
+ if you want to ensure that a just-finished transaction is archived
+ immediately. Other utility functions related to WAL management are
+ listed in <xref linkend="functions-admin-backup-table">.
+ </para>
</sect2>
<sect2 id="backup-base-backup">
-<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.74 2006/08/15 18:26:58 tgl Exp $ -->
+<!-- $PostgreSQL: pgsql/doc/src/sgml/config.sgml,v 1.75 2006/08/17 23:04:03 tgl Exp $ -->
<chapter Id="runtime-config">
<title>Server Configuration</title>
</listitem>
</varlistentry>
+ <varlistentry id="guc-archive-timeout" xreflabel="archive_timeout">
+ <term><varname>archive_timeout</varname> (<type>integer</type>)</term>
+ <indexterm>
+ <primary><varname>archive_timeout</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The <xref linkend="guc-archive-command"> is only invoked on completed
+ WAL segments. Hence,
+ if your server generates only little WAL traffic (or has slack periods
+ where it does so), there could be a long delay between the completion
+ of a transaction and its safe recording in archive storage. To put
+ a limit on how old unarchived data can be, you can set
+ <varname>archive_timeout</> to force the server to switch
+ to a new WAL segment file periodically. When this parameter is
+ greater than zero, the server will switch to a new segment file
+ whenever this many seconds elapse since the last segment file switch.
+ Note that archived
+ files that are ended early due to a forced switch are still the same
+ length as completely full files. It is therefore unwise to set a very
+ short <varname>archive_timeout</> — it will bloat your archive
+ storage. <varname>archive_timeout</> settings of a minute or so are
+ usually reasonable.
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
</sect1>
-<!-- $PostgreSQL: pgsql/doc/src/sgml/func.sgml,v 1.329 2006/08/06 03:53:43 tgl Exp $ -->
+<!-- $PostgreSQL: pgsql/doc/src/sgml/func.sgml,v 1.330 2006/08/17 23:04:03 tgl Exp $ -->
<chapter id="functions">
<title>Functions and Operators</title>
<indexterm zone="functions-admin">
<primary>pg_current_xlog_location</primary>
</indexterm>
+ <indexterm zone="functions-admin">
+ <primary>pg_current_xlog_insert_location</primary>
+ </indexterm>
<indexterm zone="functions-admin">
<primary>pg_xlogfile_name_offset</primary>
</indexterm>
<literal><function>pg_current_xlog_location</function>()</literal>
</entry>
<entry><type>text</type></entry>
- <entry>Get current xlog location</entry>
+ <entry>Get current xlog write location</entry>
</row>
<row>
<entry>
- <literal><function>pg_xlogfile_name_offset</function>(<parameter>location</> <type>text</>)</literal>
+ <literal><function>pg_current_xlog_insert_location</function>()</literal>
</entry>
<entry><type>text</type></entry>
+ <entry>Get current xlog insert location</entry>
+ </row>
+ <row>
+ <entry>
+ <literal><function>pg_xlogfile_name_offset</function>(<parameter>location</> <type>text</>)</literal>
+ </entry>
+ <entry><type>text</>, <type>integer</></entry>
<entry>Convert xlog location string to filename and decimal byte offset within file</entry>
</row>
<row>
</para>
<para>
- <function>pg_current_xlog_location</> displays the current xlog insertion
- point in the same format used by the above functions. This is a
- read-only operation and does not require superuser permissions.
+ <function>pg_current_xlog_location</> displays the current xlog write
+ location in the same format used by the above functions. Similarly
+ <function>pg_current_xlog_insert_location</> displays the current xlog
+ insertion point. The insertion point is the <quote>logical</> end of xlog
+ at any instant, while the write location is the end of what has actually
+ been written out from the server's internal buffers. The write location
+ is the end of what can be examined from outside the server, and is usually
+ what you want if you are interested in archiving partially-complete xlog
+ files. The insertion point is made available primarily for server
+ debugging purposes. These are both read-only operations and do not
+ require superuser permissions.
</para>
<para>
corresponding xlog filename and byte offset from the results of any of the
above functions. For example:
<programlisting>
-postgres=# select pg_xlogfile_name_offset(pg_stop_backup());
- pg_xlogfile_name_offset
-----------------------------------
- 00000001000000000000000D 4039624
+postgres=# select * from pg_xlogfile_name_offset(pg_stop_backup());
+ file_name | file_offset
+--------------------------+-------------
+ 00000001000000000000000D | 4039624
(1 row)
</programlisting>
Similarly, <function>pg_xlogfile_name</> extracts just the xlog filename.
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * $PostgreSQL: pgsql/src/backend/access/transam/xlog.c,v 1.247 2006/08/07 16:57:56 tgl Exp $
+ * $PostgreSQL: pgsql/src/backend/access/transam/xlog.c,v 1.248 2006/08/17 23:04:05 tgl Exp $
*
*-------------------------------------------------------------------------
*/
#include <sys/time.h>
#include "access/clog.h"
+#include "access/heapam.h"
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/transam.h"
#include "access/xlogutils.h"
#include "catalog/catversion.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_type.h"
+#include "funcapi.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgwriter.h"
/* User-settable parameters */
int CheckPointSegments = 3;
int XLOGbuffers = 8;
+int XLogArchiveTimeout = 0;
char *XLogArchiveCommand = NULL;
char *XLOG_sync_method = NULL;
const char XLOG_sync_method_default[] = DEFAULT_SYNC_METHOD_STR;
*/
typedef struct XLogCtlWrite
{
- XLogwrtResult LogwrtResult; /* current value of LogwrtResult */
- int curridx; /* cache index of next block to write */
+ XLogwrtResult LogwrtResult; /* current value of LogwrtResult */
+ int curridx; /* cache index of next block to write */
+ time_t lastSegSwitchTime; /* time of last xlog segment switch */
} XLogCtlWrite;
/*
* switch.
*
* This is also the right place to notify the Archiver that the
- * segment is ready to copy to archival storage.
+ * segment is ready to copy to archival storage, and to update
+ * the timer for archive_timeout.
*/
if (finishing_seg || (xlog_switch && last_iteration))
{
if (XLogArchivingActive())
XLogArchiveNotifySeg(openLogId, openLogSeg);
+
+ Write->lastSegSwitchTime = time(NULL);
}
}
ControlFile->time = time(NULL);
UpdateControlFile();
+ /* start the archive_timeout timer running */
+ XLogCtl->Write.lastSegSwitchTime = ControlFile->time;
+
/* Start up the commit log and related stuff, too */
StartupCLOG();
StartupSUBTRANS(oldestActiveXID);
return RedoRecPtr;
}
+/*
+ * Get the time of the last xlog segment switch
+ */
+time_t
+GetLastSegSwitchTime(void)
+{
+ time_t result;
+
+ /* Need WALWriteLock, but shared lock is sufficient */
+ LWLockAcquire(WALWriteLock, LW_SHARED);
+ result = XLogCtl->Write.lastSegSwitchTime;
+ LWLockRelease(WALWriteLock);
+
+ return result;
+}
+
/*
* GetRecentNextXid - get the nextXid value saved by the most recent checkpoint
*
* or the end+1 address of the prior segment if we did not need to
* write a switch record because we are already at segment start.
*/
-static XLogRecPtr
+XLogRecPtr
RequestXLogSwitch(void)
{
XLogRecPtr RecPtr;
}
/*
- * Report the current WAL location (same format as pg_start_backup etc)
+ * Report the current WAL write location (same format as pg_start_backup etc)
+ *
+ * This is useful for determining how much of WAL is visible to an external
+ * archiving process. Note that the data before this point is written out
+ * to the kernel, but is not necessarily synced to disk.
*/
Datum
pg_current_xlog_location(PG_FUNCTION_ARGS)
+{
+ text *result;
+ char location[MAXFNAMELEN];
+
+ /* Make sure we have an up-to-date local LogwrtResult */
+ {
+ /* use volatile pointer to prevent code rearrangement */
+ volatile XLogCtlData *xlogctl = XLogCtl;
+
+ SpinLockAcquire(&xlogctl->info_lck);
+ LogwrtResult = xlogctl->LogwrtResult;
+ SpinLockRelease(&xlogctl->info_lck);
+ }
+
+ snprintf(location, sizeof(location), "%X/%X",
+ LogwrtResult.Write.xlogid, LogwrtResult.Write.xrecoff);
+
+ result = DatumGetTextP(DirectFunctionCall1(textin,
+ CStringGetDatum(location)));
+ PG_RETURN_TEXT_P(result);
+}
+
+/*
+ * Report the current WAL insert location (same format as pg_start_backup etc)
+ *
+ * This function is mostly for debugging purposes.
+ */
+Datum
+pg_current_xlog_insert_location(PG_FUNCTION_ARGS)
{
text *result;
XLogCtlInsert *Insert = &XLogCtl->Insert;
pg_xlogfile_name_offset(PG_FUNCTION_ARGS)
{
text *location = PG_GETARG_TEXT_P(0);
- text *result;
char *locationstr;
unsigned int uxlogid;
unsigned int uxrecoff;
uint32 xrecoff;
XLogRecPtr locationpoint;
char xlogfilename[MAXFNAMELEN];
+ Datum values[2];
+ bool isnull[2];
+ TupleDesc resultTupleDesc;
+ HeapTuple resultHeapTuple;
+ Datum result;
+ /*
+ * Read input and parse
+ */
locationstr = DatumGetCString(DirectFunctionCall1(textout,
PointerGetDatum(location)));
locationpoint.xlogid = uxlogid;
locationpoint.xrecoff = uxrecoff;
+ /*
+ * Construct a tuple descriptor for the result row. This must match
+ * this function's pg_proc entry!
+ */
+ resultTupleDesc = CreateTemplateTupleDesc(2, false);
+ TupleDescInitEntry(resultTupleDesc, (AttrNumber) 1, "file_name",
+ TEXTOID, -1, 0);
+ TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
+ INT4OID, -1, 0);
+
+ resultTupleDesc = BlessTupleDesc(resultTupleDesc);
+
+ /*
+ * xlogfilename
+ */
XLByteToPrevSeg(locationpoint, xlogid, xlogseg);
XLogFileName(xlogfilename, ThisTimeLineID, xlogid, xlogseg);
+ values[0] = DirectFunctionCall1(textin,
+ CStringGetDatum(xlogfilename));
+ isnull[0] = false;
+
+ /*
+ * offset
+ */
xrecoff = locationpoint.xrecoff - xlogseg * XLogSegSize;
- snprintf(xlogfilename + strlen(xlogfilename),
- sizeof(xlogfilename) - strlen(xlogfilename),
- " %u",
- (unsigned int) xrecoff);
- result = DatumGetTextP(DirectFunctionCall1(textin,
- CStringGetDatum(xlogfilename)));
- PG_RETURN_TEXT_P(result);
+ values[1] = UInt32GetDatum(xrecoff);
+ isnull[1] = false;
+
+ /*
+ * Tuple jam: Having first prepared your Datums, then squash together
+ */
+ resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
+
+ result = HeapTupleGetDatum(resultHeapTuple);
+
+ PG_RETURN_DATUM(result);
}
/*
*
*
* IDENTIFICATION
- * $PostgreSQL: pgsql/src/backend/postmaster/bgwriter.c,v 1.26 2006/07/14 14:52:22 momjian Exp $
+ * $PostgreSQL: pgsql/src/backend/postmaster/bgwriter.c,v 1.27 2006/08/17 23:04:06 tgl Exp $
*
*-------------------------------------------------------------------------
*/
#include <signal.h>
#include <time.h>
+#include "access/xlog_internal.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "postmaster/bgwriter.h"
static bool ckpt_active = false;
static time_t last_checkpoint_time;
+static time_t last_xlog_switch_time;
static void bg_quickdie(SIGNAL_ARGS);
#endif
/*
- * Initialize so that first time-driven checkpoint happens at the correct
+ * Initialize so that first time-driven event happens at the correct
* time.
*/
- last_checkpoint_time = time(NULL);
+ last_checkpoint_time = last_xlog_switch_time = time(NULL);
/*
* Create a resource owner to keep track of our resources (currently
else
BgBufferSync();
+ /*
+ * Check for archive_timeout, if so, switch xlog files. First
+ * we do a quick check using possibly-stale local state.
+ */
+ if (XLogArchiveTimeout > 0 &&
+ (int) (now - last_xlog_switch_time) >= XLogArchiveTimeout)
+ {
+ /*
+ * Update local state ... note that last_xlog_switch_time is
+ * the last time a switch was performed *or requested*.
+ */
+ time_t last_time = GetLastSegSwitchTime();
+
+ last_xlog_switch_time = Max(last_xlog_switch_time, last_time);
+
+ /* if we did a checkpoint, 'now' might be stale too */
+ if (do_checkpoint)
+ now = time(NULL);
+
+ /* Now we can do the real check */
+ if ((int) (now - last_xlog_switch_time) >= XLogArchiveTimeout)
+ {
+ XLogRecPtr switchpoint;
+
+ /* OK, it's time to switch */
+ switchpoint = RequestXLogSwitch();
+
+ /*
+ * If the returned pointer points exactly to a segment
+ * boundary, assume nothing happened.
+ */
+ if ((switchpoint.xrecoff % XLogSegSize) != 0)
+ ereport(DEBUG1,
+ (errmsg("xlog switch forced (archive_timeout=%d)",
+ XLogArchiveTimeout)));
+ /*
+ * Update state in any case, so we don't retry constantly
+ * when the system is idle.
+ */
+ last_xlog_switch_time = now;
+ }
+ }
+
/*
* Nap for the configured time, or sleep for 10 seconds if there is no
* bgwriter activity configured.
if ((bgwriter_all_percent > 0.0 && bgwriter_all_maxpages > 0) ||
(bgwriter_lru_percent > 0.0 && bgwriter_lru_maxpages > 0))
udelay = BgWriterDelay * 1000L;
+ else if (XLogArchiveTimeout > 0)
+ udelay = 1000000L; /* One second */
else
- udelay = 10000000L;
- while (udelay > 1000000L)
+ udelay = 10000000L; /* Ten seconds */
+
+ while (udelay > 999999L)
{
if (got_SIGHUP || checkpoint_requested || shutdown_requested)
break;
AbsorbFsyncRequests();
udelay -= 1000000L;
}
+
if (!(got_SIGHUP || checkpoint_requested || shutdown_requested))
pg_usleep(udelay);
}
* Written by Peter Eisentraut <peter_e@gmx.net>.
*
* IDENTIFICATION
- * $PostgreSQL: pgsql/src/backend/utils/misc/guc.c,v 1.342 2006/08/15 18:26:59 tgl Exp $
+ * $PostgreSQL: pgsql/src/backend/utils/misc/guc.c,v 1.343 2006/08/17 23:04:06 tgl Exp $
*
*--------------------------------------------------------------------
*/
{
{"ignore_system_indexes", PGC_BACKEND, DEVELOPER_OPTIONS,
- gettext_noop("Disabled reading from system indexes."),
+ gettext_noop("Disables reading from system indexes."),
gettext_noop("It does not prevent updating the indexes, so it is safe "
"to use. The worst consequence is slowness."),
GUC_NOT_IN_SAMPLE
static struct config_int ConfigureNamesInt[] =
{
+ {
+ {"archive_timeout", PGC_SIGHUP, WAL_SETTINGS,
+ gettext_noop("Forces a switch to the next xlog file if a "
+ "new file has not been started within N seconds."),
+ NULL,
+ GUC_UNIT_S
+ },
+ &XLogArchiveTimeout,
+ 0, 0, INT_MAX, NULL, NULL
+ },
{
{"post_auth_delay", PGC_BACKEND, DEVELOPER_OPTIONS,
gettext_noop("Waits N seconds on connection startup after authentication."),
# - Archiving -
-#archive_command = '' # command to use to archive a logfile
- # segment
+#archive_command = '' # command to use to archive a logfile segment
+#archive_timeout = 0 # force a logfile segment switch after this
+ # many seconds; 0 is off
#---------------------------------------------------------------------------
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * $PostgreSQL: pgsql/src/include/access/xlog.h,v 1.72 2006/07/13 16:49:19 momjian Exp $
+ * $PostgreSQL: pgsql/src/include/access/xlog.h,v 1.73 2006/08/17 23:04:08 tgl Exp $
*/
#ifndef XLOG_H
#define XLOG_H
extern int CheckPointSegments;
extern int XLOGbuffers;
extern char *XLogArchiveCommand;
+extern int XLogArchiveTimeout;
extern char *XLOG_sync_method;
extern const char XLOG_sync_method_default[];
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * $PostgreSQL: pgsql/src/include/access/xlog_internal.h,v 1.15 2006/08/07 16:57:57 tgl Exp $
+ * $PostgreSQL: pgsql/src/include/access/xlog_internal.h,v 1.16 2006/08/17 23:04:08 tgl Exp $
*/
#ifndef XLOG_INTERNAL_H
#define XLOG_INTERNAL_H
+#include <time.h>
+
#include "access/xlog.h"
#include "fmgr.h"
#include "storage/block.h"
extern const RmgrData RmgrTable[];
+/*
+ * Exported to support xlog switching from bgwriter
+ */
+extern time_t GetLastSegSwitchTime(void);
+extern XLogRecPtr RequestXLogSwitch(void);
+
/*
* These aren't in xlog.h because I'd rather not include fmgr.h there.
*/
extern Datum pg_stop_backup(PG_FUNCTION_ARGS);
extern Datum pg_switch_xlog(PG_FUNCTION_ARGS);
extern Datum pg_current_xlog_location(PG_FUNCTION_ARGS);
+extern Datum pg_current_xlog_insert_location(PG_FUNCTION_ARGS);
extern Datum pg_xlogfile_name_offset(PG_FUNCTION_ARGS);
extern Datum pg_xlogfile_name(PG_FUNCTION_ARGS);
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * $PostgreSQL: pgsql/src/include/catalog/catversion.h,v 1.349 2006/08/12 02:52:06 tgl Exp $
+ * $PostgreSQL: pgsql/src/include/catalog/catversion.h,v 1.350 2006/08/17 23:04:08 tgl Exp $
*
*-------------------------------------------------------------------------
*/
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 200608101
+#define CATALOG_VERSION_NO 200608171
#endif
* Portions Copyright (c) 1996-2006, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * $PostgreSQL: pgsql/src/include/catalog/pg_proc.h,v 1.420 2006/08/06 03:53:44 tgl Exp $
+ * $PostgreSQL: pgsql/src/include/catalog/pg_proc.h,v 1.421 2006/08/17 23:04:10 tgl Exp $
*
* NOTES
* The script catalog/genbki.sh reads this file and generates .bki
DATA(insert OID = 2848 ( pg_switch_xlog PGNSP PGUID 12 f f t f v 0 25 "" _null_ _null_ _null_ pg_switch_xlog - _null_ ));
DESCR("Switch to new xlog file");
DATA(insert OID = 2849 ( pg_current_xlog_location PGNSP PGUID 12 f f t f v 0 25 "" _null_ _null_ _null_ pg_current_xlog_location - _null_ ));
-DESCR("current xlog location");
-DATA(insert OID = 2850 ( pg_xlogfile_name_offset PGNSP PGUID 12 f f t f i 1 25 "25" _null_ _null_ _null_ pg_xlogfile_name_offset - _null_ ));
+DESCR("current xlog write location");
+DATA(insert OID = 2852 ( pg_current_xlog_insert_location PGNSP PGUID 12 f f t f v 0 25 "" _null_ _null_ _null_ pg_current_xlog_insert_location - _null_ ));
+DESCR("current xlog insert location");
+DATA(insert OID = 2850 ( pg_xlogfile_name_offset PGNSP PGUID 12 f f t f i 1 2249 "25" "{25,25,23}" "{i,o,o}" "{wal_location,file_name,file_offset}" pg_xlogfile_name_offset - _null_ ));
DESCR("xlog filename and byte offset, given an xlog location");
DATA(insert OID = 2851 ( pg_xlogfile_name PGNSP PGUID 12 f f t f i 1 25 "25" _null_ _null_ _null_ pg_xlogfile_name - _null_ ));
DESCR("xlog filename, given an xlog location");
projects / postgresql / commitdiff