From 4f95a7fe3ef8e786dbb6c9c1dc3240da24538f92 Mon Sep 17 00:00:00 2001 From: Yaowu Xu Date: Wed, 9 Apr 2014 11:37:27 -0700 Subject: [PATCH] Fix the setting of mode_skip_mask In mode selection loop, once mode_index pass mode_skip_start, all modes with a different reference frame from current best mode are masked out using mode_skip_mask. However, the setting of mode_skip_mask may use an invalid mode if there is no mode tested yet. This commit fixes the issue by making sure a mode has been tested and selected. Otherwise, no mode will be masked out because of their reference frame. Change-Id: Ib0009e8a96836a65cf5347440fff8a2e1a67f29f --- vp9/encoder/vp9_rdopt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vp9/encoder/vp9_rdopt.c b/vp9/encoder/vp9_rdopt.c index a3e513277..1066f6a17 100644 --- a/vp9/encoder/vp9_rdopt.c +++ b/vp9/encoder/vp9_rdopt.c @@ -3304,7 +3304,7 @@ int64_t vp9_rd_pick_inter_mode_sb(VP9_COMP *cpi, MACROBLOCK *x, // Look at the reference frame of the best mode so far and set the // skip mask to look at a subset of the remaining modes. - if (mode_index == mode_skip_start) { + if (mode_index == mode_skip_start && best_rd < INT64_MAX) { switch (vp9_mode_order[best_mode_index].ref_frame[0]) { case INTRA_FRAME: break; -- 2.50.1