From: Deb Mukherjee <debargha@google.com>
Date: Fri, 3 Oct 2014 17:22:21 +0000 (-0700)
Subject: Prevent negative cost for highbitdepth
X-Git-Tag: v1.4.0~681^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=431cdc33eee902bbc1c4272a8551e3734fe4a16f;p=libvpx

Prevent negative cost for highbitdepth

Adds proper scaling for highbitdepth in a rdopt cost.

Change-Id: I066694799a7f491b830945ef1c66eb202071c355
---

diff --git a/vp9/encoder/vp9_rdopt.c b/vp9/encoder/vp9_rdopt.c
index b14ff0822..0ae0d5757 100644
--- a/vp9/encoder/vp9_rdopt.c
+++ b/vp9/encoder/vp9_rdopt.c
@@ -491,7 +491,9 @@ static void block_rd_txfm(int plane, int block, BLOCK_SIZE plane_bsize,
       if (x->plane[plane].eobs[block]) {
         int64_t dc_correct = coeff[0] * coeff[0] -
             (coeff[0] - dqcoeff[0]) * (coeff[0] - dqcoeff[0]);
-
+#if CONFIG_VP9_HIGHBITDEPTH
+        dc_correct >>= ((xd->bd - 8) * 2);
+#endif
         if (tx_size != TX_32X32)
           dc_correct >>= 2;