From f86756985d42ac4a14866534c588061ede860b7b Mon Sep 17 00:00:00 2001 From: Alexey Samsonov Date: Mon, 25 Jan 2016 16:05:25 -0800 Subject: [PATCH] Fix float-cast-overflow in x264_ratecontrol_end function According to the C standard, it is undefined behavior to cast a negative floating point number to an unsigned integer. Float-cast-overflow in general is known to produce different results on different architectures. Building x264 code with Clang and -fsanitize=float-cast-overflow (http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#availablle-checks) and running it on some real-life examples occasionally produces errors of the form: encoder/ratecontrol.c:1892: runtime error: value -5011.14 is outside the range of representable values of type 'unsigned short' Fix these errors by explicitly coding the de-facto x86 behavior: casting float to uint16_t through int16_t. --- encoder/ratecontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/encoder/ratecontrol.c b/encoder/ratecontrol.c index d8d06b35..0c6913a7 100644 --- a/encoder/ratecontrol.c +++ b/encoder/ratecontrol.c @@ -1889,7 +1889,7 @@ int x264_ratecontrol_end( x264_t *h, int bits, int *filler ) uint8_t i_type = h->sh.i_type; /* Values are stored as big-endian FIX8.8 */ for( int i = 0; i < h->mb.i_mb_count; i++ ) - rc->mbtree.qp_buffer[0][i] = endian_fix16( h->fenc->f_qp_offset[i]*256.0 ); + rc->mbtree.qp_buffer[0][i] = endian_fix16( (int16_t)(h->fenc->f_qp_offset[i]*256.0) ); if( fwrite( &i_type, 1, 1, rc->p_mbtree_stat_file_out ) < 1 ) goto fail; if( fwrite( rc->mbtree.qp_buffer[0], sizeof(uint16_t), h->mb.i_mb_count, rc->p_mbtree_stat_file_out ) < h->mb.i_mb_count ) -- 2.40.0