From: Heikki Linnakangas Date: Sun, 28 Jun 2015 21:09:10 +0000 (+0300) Subject: Also trigger restartpoints based on max_wal_size on standby. X-Git-Tag: REL9_5_ALPHA1~7 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=d661532e27b34e9c89d0700c6ce246731e70072c;p=postgresql Also trigger restartpoints based on max_wal_size on standby. When archive recovery and restartpoints were initially introduced, checkpoint_segments was ignored on the grounds that the files restored from archive don't consume any space in the recovery server. That was changed in later releases, but even then it was arguably a feature rather than a bug, as performing restartpoints as often as checkpoints during normal operation might be excessive, but you might nevertheless not want to waste a lot of space for pre-allocated WAL by setting checkpoint_segments to a high value. But now that we have separate min_wal_size and max_wal_size settings, you can bound WAL usage with max_wal_size, and still avoid consuming excessive space usage by setting min_wal_size to a lower value, so that argument is moot. There are still some issues with actually limiting the space usage to max_wal_size: restartpoints in recovery can only start after seeing the checkpoint record, while a checkpoint starts flushing buffers as soon as the redo-pointer is set. Restartpoint is paced to happen at the same leisurily speed, determined by checkpoint_completion_target, as checkpoints, but because they are started later, max_wal_size can be exceeded by upto one checkpoint cycle's worth of WAL, depending on checkpoint_completion_target. But that seems better than not trying at all, and max_wal_size is a soft limit anyway. The documentation already claimed that max_wal_size is obeyed in recovery, so this just fixes the behaviour to match the docs. However, add some weasel-words there to mention that max_wal_size may well be exceeded by some amount in recovery. --- diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml index f4083c3fe5..e3941c9391 100644 --- a/doc/src/sgml/wal.sgml +++ b/doc/src/sgml/wal.sgml @@ -590,7 +590,11 @@ A restartpoint is triggered when a checkpoint record is reached if at least checkpoint_timeout seconds have passed since the last restartpoint, or if WAL size is about to exceed - max_wal_size. + max_wal_size. However, because of limitations on when a + restartpoint can be performed, max_wal_size is often exceeded + during recovery, by up to one checkpoint cycle's worth of WAL. + (max_wal_size is never a hard limit anyway, so you should + always leave plenty of headroom to avoid running out of disk space.) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 7830b47c8d..0def47d6ed 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -10943,7 +10943,7 @@ XLogPageRead(XLogReaderState *xlogreader, XLogRecPtr targetPagePtr, int reqLen, * Request a restartpoint if we've replayed too much xlog since the * last one. */ - if (StandbyModeRequested && bgwriterLaunched) + if (bgwriterLaunched) { if (XLogCheckpointNeeded(readSegNo)) { diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c index 0dce6a8ffa..3b3a09ef88 100644 --- a/src/backend/postmaster/checkpointer.c +++ b/src/backend/postmaster/checkpointer.c @@ -475,10 +475,12 @@ CheckpointerMain(void) /* * Initialize checkpointer-private variables used during - * checkpoint + * checkpoint. */ ckpt_active = true; - if (!do_restartpoint) + if (do_restartpoint) + ckpt_start_recptr = GetXLogReplayRecPtr(NULL); + else ckpt_start_recptr = GetInsertRecPtr(); ckpt_start_time = now; ckpt_cached_elapsed = 0; @@ -720,7 +722,7 @@ CheckpointWriteDelay(int flags, double progress) /* * IsCheckpointOnSchedule -- are we on schedule to finish this checkpoint - * in time? + * (or restartpoint) in time? * * Compares the current progress against the time/segments elapsed since last * checkpoint, and returns true if the progress we've made this far is greater @@ -757,17 +759,27 @@ IsCheckpointOnSchedule(double progress) * compares against RedoRecptr, so this is not completely accurate. * However, it's good enough for our purposes, we're only calculating an * estimate anyway. + * + * During recovery, we compare last replayed WAL record's location with + * the location computed before calling CreateRestartPoint. That maintains + * the same pacing as we have during checkpoints in normal operation, but + * we might exceed max_wal_size by a fair amount. That's because there can + * be a large gap between a checkpoint's redo-pointer and the checkpoint + * record itself, and we only start the restartpoint after we've seen the + * checkpoint record. (The gap is typically up to CheckPointSegments * + * checkpoint_completion_target where checkpoint_completion_target is the + * value that was in effect when the WAL was generated). */ - if (!RecoveryInProgress()) - { + if (RecoveryInProgress()) + recptr = GetXLogReplayRecPtr(NULL); + else recptr = GetInsertRecPtr(); - elapsed_xlogs = (((double) (recptr - ckpt_start_recptr)) / XLogSegSize) / CheckPointSegments; + elapsed_xlogs = (((double) (recptr - ckpt_start_recptr)) / XLogSegSize) / CheckPointSegments; - if (progress < elapsed_xlogs) - { - ckpt_cached_elapsed = elapsed_xlogs; - return false; - } + if (progress < elapsed_xlogs) + { + ckpt_cached_elapsed = elapsed_xlogs; + return false; } /*