From: Jingning Han <jingning@google.com>
Date: Tue, 6 May 2014 23:26:30 +0000 (-0700)
Subject: Tune rate-distortion modeling to account for frame light change
X-Git-Tag: v1.4.0~1639^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=74f98e6f45dcd63bc1f43cad970ee080d90e65ba;p=libvpx

Tune rate-distortion modeling to account for frame light change

When the variance is far less than sse, the block is considered to
be under light change. All the energy is compacted into DC coeff
and can be coded at low cost. In such situation, switch the rate-
distortion modeling from sse+var based back to variance based.

Note that this is a temporary solution to handle the rare situations
where the scene light changes.

Change-Id: I1ee0fe2b9eda6b5fac40152e1841bf23f4d229fd
---

diff --git a/vp9/encoder/vp9_pickmode.c b/vp9/encoder/vp9_pickmode.c
index 56eb9440c..7d2cab42b 100644
--- a/vp9/encoder/vp9_pickmode.c
+++ b/vp9/encoder/vp9_pickmode.c
@@ -194,7 +194,13 @@ static void model_rd_for_sb_y(VP9_COMP *cpi, BLOCK_SIZE bsize,
   int var = cpi->fn_ptr[bsize].vf(p->src.buf, p->src.stride,
                                   pd->dst.buf, pd->dst.stride, &sse);
 
-  vp9_model_rd_from_var_lapndz(sse + var, 1 << num_pels_log2_lookup[bsize],
+  // TODO(jingning) This is a temporary solution to account for frames with
+  // light changes. Need to customize the rate-distortion modeling for non-RD
+  // mode decision.
+  if ((sse >> 3) > var)
+    sse = var;
+
+  vp9_model_rd_from_var_lapndz(var + sse, 1 << num_pels_log2_lookup[bsize],
                                pd->dequant[1] >> 3, &rate, &dist);
   *out_rate_sum = rate;
   *out_dist_sum = dist << 3;