From: Debargha Mukherjee <debargha@google.com>
Date: Mon, 8 Feb 2016 23:59:17 +0000 (-0800)
Subject: Explicitly set tx_type for sub8x8 blocks
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=d46c1f2349f5f7d76a66f367a2f041dae8653c06;p=libvpx

Explicitly set tx_type for sub8x8 blocks

Fixes an issue where the tx_type was not set correctly for
sub8x8 inter and intra blocks. In the current syntax, for
sub8x8 blocks, there is still a single tx_type that is
transmitted. Ideally, this should be searched for the best
rd performance, albeit at the expense of encode speed.
For now, we just set it to DCT_DCT. Previously it was left
incorrectly as what was used for the previous non sub8x8
block.

derflr: BDRATE -0.277%

Change-Id: If76ba903bfbfd4d374cf1ac7d1daee50e92f0edd
---

diff --git a/vp10/encoder/rdopt.c b/vp10/encoder/rdopt.c
index af360242f..3117a08aa 100644
--- a/vp10/encoder/rdopt.c
+++ b/vp10/encoder/rdopt.c
@@ -1702,6 +1702,10 @@ static int64_t rd_pick_intra_sub_8x8_y_mode(VP10_COMP *cpi, MACROBLOCK *mb,
   mic->mbmi.intra_filter = INTRA_FILTER_LINEAR;
 #endif  // CONFIG_EXT_INTRA
 
+  // TODO(any): Add search of the tx_type to improve rd performance at the
+  // expense of speed.
+  mic->mbmi.tx_type = DCT_DCT;
+
   // Pick modes for each sub-block (of size 4x4, 4x8, or 8x4) in an 8x8 block.
   for (idy = 0; idy < 2; idy += num_4x4_blocks_high) {
     for (idx = 0; idx < 2; idx += num_4x4_blocks_wide) {
@@ -7365,6 +7369,10 @@ void vp10_rd_pick_inter_mode_sub8x8(struct VP10_COMP *cpi,
       for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
         filter_cache[i] = INT64_MAX;
 
+      // TODO(any): Add search of the tx_type to improve rd performance at the
+      // expense of speed.
+      mbmi->tx_type = DCT_DCT;
+
       if (cm->interp_filter != BILINEAR) {
         tmp_best_filter = EIGHTTAP;
         if (x->source_variance < sf->disable_filter_search_var_thresh) {