From: Hui Su <huisu@google.com>
Date: Thu, 27 Sep 2018 17:12:55 +0000 (-0700)
Subject: Fix a loophole in nonrd_pick_partition()
X-Git-Tag: v1.8.0~281^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=2176d88fc343c8ffa6f29fafc3cffb5ec043f1d2;p=libvpx

Fix a loophole in nonrd_pick_partition()

In some rare cases, all possible paritions may be skipped during RD
search. The patch makes the encoder do rectangular partition search if
both partition-none and partition-split are not allowed.

Tested on the rtc and ytlivehr testsets with speed 5 and 7, no coding
stats changes were observed.

Change-Id: I8b6d8b62b6d2431be8e73317d113311c98f631d5
---

diff --git a/vp9/encoder/vp9_encodeframe.c b/vp9/encoder/vp9_encodeframe.c
index f1527f930..811674d84 100644
--- a/vp9/encoder/vp9_encodeframe.c
+++ b/vp9/encoder/vp9_encodeframe.c
@@ -4404,6 +4404,8 @@ static void nonrd_pick_partition(VP9_COMP *cpi, ThreadData *td,
     partition_vert_allowed &= force_vert_split;
   }
 
+  if (!partition_none_allowed && !do_split) do_rect = 1;
+
   ctx->pred_pixel_ready =
       !(partition_vert_allowed || partition_horz_allowed || do_split);