From: Yaowu Xu <yaowu@google.com>
Date: Fri, 14 Oct 2011 16:58:35 +0000 (-0700)
Subject: added a last stage rounding for 8x8 inverse dct
X-Git-Tag: v1.3.0~1217^2~380^2~181^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=a66c945c5973246561b3ff710886018f14e6679c;p=libvpx

added a last stage rounding for 8x8 inverse dct

Prior to the added rounding, tests on randomly generated data showed
that forward-inverse transform round trip errors are about 3.02/block
for input range [-10,10] and 2.68/block for input range [-256, 255].
The added rounding reduced the errors to 0.031/block for input range
[-10,10] and 0.037/block for input range [-256, 255].

Maximum round trip error on for any pixel position is 1.

The average errors are calculated based on 100,000 blocks of randomly
with the specified ranges.

Paul mentioned in discussion that the change was not clear on why we
need change the rounding, so Patch 2 intends to make the rationale
obvious in code, it merged the two separate shifts into one, and the
two separate rounding factors into one. Patch 1 and 2 have same
numerical test results.

Change-Id: Ic5e2f5463de17253084d8b2398c4a210194b20de
---

diff --git a/vp8/common/idctllm.c b/vp8/common/idctllm.c
index 567aa1dcc..a0042ae6d 100644
--- a/vp8/common/idctllm.c
+++ b/vp8/common/idctllm.c
@@ -369,7 +369,7 @@ static void idctcol (int *blk)
       ((blk[8 * 0] + 32) >> 6);
     return;
   }
-  x0 = (blk[8 * 0] << 8) + 8192;
+  x0 = (blk[8 * 0] << 8) + 16384;
 
   /* first stage */
   x8 = W7 * (x4 + x5) + 4;
@@ -399,14 +399,14 @@ static void idctcol (int *blk)
   x4 = (181 * (x4 - x5) + 128) >> 8;
 
   /* fourth stage */
-  blk[8 * 0] = (x7 + x1) >> 14;
-  blk[8 * 1] = (x3 + x2) >> 14;
-  blk[8 * 2] = (x0 + x4) >> 14;
-  blk[8 * 3] = (x8 + x6) >> 14;
-  blk[8 * 4] = (x8 - x6) >> 14;
-  blk[8 * 5] = (x0 - x4) >> 14;
-  blk[8 * 6] = (x3 - x2) >> 14;
-  blk[8 * 7] = (x7 - x1) >> 14;
+  blk[8 * 0] = (x7 + x1 ) >> 15;
+  blk[8 * 1] = (x3 + x2 ) >> 15;
+  blk[8 * 2] = (x0 + x4 ) >> 15;
+  blk[8 * 3] = (x8 + x6 ) >> 15;
+  blk[8 * 4] = (x8 - x6 ) >> 15;
+  blk[8 * 5] = (x0 - x4 ) >> 15;
+  blk[8 * 6] = (x3 - x2 ) >> 15;
+  blk[8 * 7] = (x7 - x1 ) >> 15;
 }
 
 #define TX_DIM 8
@@ -434,7 +434,7 @@ void vp8_short_idct8x8_c(short *coefs, short *block, int pitch)
     {
         for (j = 0; j < TX_DIM; j++)
         {
-             block[i*shortpitch+j]  = X[i * TX_DIM + j]>>1;
+             block[i*shortpitch+j]  = X[i * TX_DIM + j];
         }
     }
 }