]> granicus.if.org Git - libvpx/commitdiff
vp9_quantize_sse2: quiet clang-7 integer sanitizer warning
authorJames Zern <jzern@google.com>
Tue, 10 Sep 2019 22:48:04 +0000 (15:48 -0700)
committerJames Zern <jzern@google.com>
Tue, 10 Sep 2019 22:50:22 +0000 (15:50 -0700)
nzflag is used as a boolean, it doesn't need to be a sized type, int is
enough (and _mm_movemask_epi8 returns one)

fixes:
vp9_quantize_sse2.c:136:16: implicit conversion from type
'int' of value 65535 (32-bit, signed) to type 'int16_t' (aka 'short')
changed the value to -1 (16-bit, signed)

BUG=webm:1649

Change-Id: I0e3f5278af49d84760f3dfb607f28099cf02f21d

vp9/encoder/x86/vp9_quantize_sse2.c

index 885220a71292360b7a47354448bced7fdf0fe334..e3d803b8f076c7dc820554b8a1c5ee77e6250796 100644 (file)
@@ -25,7 +25,7 @@ void vp9_quantize_fp_sse2(const tran_low_t *coeff_ptr, intptr_t n_coeffs,
                           const int16_t *iscan) {
   __m128i zero;
   __m128i thr;
-  int16_t nzflag;
+  int nzflag;
   __m128i eob;
   __m128i round, quant, dequant;