From: Hien Ho <hienho@google.com>
Date: Wed, 19 Jun 2019 23:41:22 +0000 (-0700)
Subject: vp8/encoder/boolhuff: fix integer sanitizer warnings
X-Git-Tag: v1.8.2~146^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=7cb611d221c7161c9c3af1d3496c64651ec5c003;p=libvpx

vp8/encoder/boolhuff: fix integer sanitizer warnings

from sanitizer run:
runtime error: implicit conversion from type 'unsigned int' of value 256
(32-bit, unsigned) to type 'unsigned char' changed the value to
 0 (8-bit, unsigned)

BUG=webm:1615

Change-Id: I9321bbd58a305419bc8669ecd7594adc47e8b116
---

diff --git a/vp8/encoder/boolhuff.h b/vp8/encoder/boolhuff.h
index 8ac0a2cc4..8cc61bdd4 100644
--- a/vp8/encoder/boolhuff.h
+++ b/vp8/encoder/boolhuff.h
@@ -92,7 +92,7 @@ static void vp8_encode_bool(BOOL_CODER *bc, int bit, int probability) {
     }
 
     validate_buffer(bc->buffer + bc->pos, 1, bc->buffer_end, bc->error);
-    bc->buffer[bc->pos++] = (lowvalue >> (24 - offset));
+    bc->buffer[bc->pos++] = (lowvalue >> (24 - offset) & 0xff);
 
     lowvalue <<= offset;
     shift = count;