From: Paul Wilkins <paulwilkins@google.com>
Date: Thu, 6 Sep 2018 14:56:01 +0000 (+0100)
Subject: Fix rate control bug with recode all.
X-Git-Tag: v1.8.0~337^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=bb58dfade710c94fc78b50a50f76925863873f38;p=libvpx

Fix rate control bug with recode all.

This patch fixes a rate control bug that can manifest if the recode
loop is activated for all frame types. Specifically things go wrong when the
recode loop is used on an overlay frame that has a rate target of 0 bits.

The patch prevents adjustment of the active worst quality and repeat recode
loops for overlay frames.

The bug showed up during artificial experiments on re-distribution of bits in
ARF groups but does not activate in any current encode profile, as even best
best quality does not currently allow recodes for all frames.

Change-Id: I80872093d9ebd3350106230c42c3928e56ecb754
---

diff --git a/vp9/encoder/vp9_encoder.c b/vp9/encoder/vp9_encoder.c
index 1fd4a036a..ed8467443 100644
--- a/vp9/encoder/vp9_encoder.c
+++ b/vp9/encoder/vp9_encoder.c
@@ -4342,8 +4342,9 @@ static void encode_with_recode_loop(VP9_COMP *cpi, size_t *size,
           // Special case if the projected size is > the max allowed.
           if ((q == q_high) &&
               ((rc->projected_frame_size >= rc->max_frame_bandwidth) ||
-               (rc->projected_frame_size >=
-                big_rate_miss_high_threshold(cpi)))) {
+               (!rc->is_src_frame_alt_ref &&
+                (rc->projected_frame_size >=
+                 big_rate_miss_high_threshold(cpi))))) {
             int max_rate = VPXMAX(1, VPXMIN(rc->max_frame_bandwidth,
                                             big_rate_miss_high_threshold(cpi)));
             double q_val_high;
@@ -4448,7 +4449,7 @@ static void encode_with_recode_loop(VP9_COMP *cpi, size_t *size,
 #endif
     // Have we been forced to adapt Q outside the expected range by an extreme
     // rate miss. If so adjust the active maxQ for the subsequent frames.
-    if (q > cpi->twopass.active_worst_quality) {
+    if (!rc->is_src_frame_alt_ref && (q > cpi->twopass.active_worst_quality)) {
       cpi->twopass.active_worst_quality = q;
     } else if (oxcf->vbr_corpus_complexity && q == q_low &&
                rc->projected_frame_size < rc->this_frame_target) {