Fix regression in r736

r736 added intra RD refinement to B-frames; however, it is possible for subme=7 to be used without b-rdo.
This means intra RD isn't run, and therefore it is possible for intra chroma analysis to not have been run, since update_cache was never called for an intra block, and chroma ME is not required even at subme=7.
r801, which removed a memset, made this worse because previously the chroma prediction mode was at least initialized to zero; now it was not initialized at all.
Therefore, --no-chroma-me, --subme 7, and no --b-rdo had the potential to crash.
This change restricts intra RD refinement to only be run when --b-rdo is enabled (sensible to begin with), thus preventing a crash in this case.

diff --git a/encoder/analyse.c b/encoder/analyse.c
index 0d3831a8b5da5ab2ea7144abd9854fb5252b7fab..17efe03289afc4cd17d28efbd5a0c56a7f8581b0 100644 (file)
--- a/encoder/analyse.c
+++ b/encoder/analyse.c
@@ -2580,7 +2580,7 @@ void x264_macroblock_analyse( x264_t *h )
             h->mb.i_type = i_type;
             h->mb.i_partition = i_partition;
 
-            if( h->mb.i_subpel_refine >= 7 && IS_INTRA( i_type ) )
+            if( analysis.b_mbrd && h->mb.i_subpel_refine >= 7 && IS_INTRA( i_type ) )
                 x264_intra_rd_refine( h, &analysis );
             else if( h->param.analyse.b_bidir_me )
                 refine_bidir( h, &analysis );