]> granicus.if.org Git - libvpx/commit
{vp8,vp9}_set_roi_map: fix validation with INT_MIN
authorJames Zern <jzern@google.com>
Fri, 1 Oct 2021 20:46:02 +0000 (13:46 -0700)
committerJerome Jiang <jianj@google.com>
Mon, 4 Oct 2021 18:46:48 +0000 (11:46 -0700)
commit2ea1b908d87b29bcc6214efd3073b92392d495ff
treec6d7f5abd004cf1dc4c75cda00adae4c4912547c
parent16837ae1680bbc73381570cc783439b0ea121ba6
{vp8,vp9}_set_roi_map: fix validation with INT_MIN

previously ranges were checked with abs() whose behavior is undefined
with INT_MIN. this fixes a crash when the original value is returned and
it later used as and offset into a table.

Bug: webm:1742
Change-Id: I345970b75c46699587a4fbc4a059e59277f4c2c8
test/encode_api_test.cc
vp8/encoder/onyx_if.c
vp9/encoder/vp9_encoder.c