From: Jingning Han <jingning@google.com>
Date: Thu, 22 Oct 2015 19:06:32 +0000 (-0700)
Subject: Reset tx_type in recursive transform block partitioning
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=d3e5545fa60628c744ec8a3488516a2c5f7a7ffa;p=libvpx

Reset tx_type in recursive transform block partitioning

Temporarily reset the transform type in the inter modes when
recursive transform block partitioning is used. This resolves an
enc/dec mismatch issue in nextgenv2 codebase when both var-tx and
ext-tx experiments are turned on.

Change-Id: I2543f0a567243da95b237752d46964b07b669ad9
---

diff --git a/vp10/encoder/rdopt.c b/vp10/encoder/rdopt.c
index dfec9b4e8..43eeb0610 100644
--- a/vp10/encoder/rdopt.c
+++ b/vp10/encoder/rdopt.c
@@ -1442,6 +1442,10 @@ static void inter_block_yrd(const VP10_COMP *cpi, MACROBLOCK *x,
   *sse = 0;
   *skippable = 1;
 
+#if CONFIG_EXT_TX
+  xd->mi[0]->mbmi.tx_type = DCT_DCT;
+#endif
+
   if (is_cost_valid) {
     const struct macroblockd_plane *const pd = &xd->plane[0];
     const BLOCK_SIZE plane_bsize = get_plane_block_size(bsize, pd);