]> granicus.if.org Git - libvpx/commit
rd_pick_inter_mode: prevent signed integer overflow
authorJames Zern <jzern@google.com>
Tue, 6 Nov 2012 01:14:32 +0000 (17:14 -0800)
committerJames Zern <jzern@google.com>
Tue, 6 Nov 2012 01:14:32 +0000 (17:14 -0800)
commite47d9f1d072e99c8c952e8ffa278ff7aef2ef1b7
tree254d3d94bcb602d6207f5e703731cd5a48c5c8a7
parentf2541f8a4a1a6503c43de0ea8954f4b7ea2d2499
rd_pick_inter_mode: prevent signed integer overflow

calculate the txfm_cache difference first as both values may be
INT64_MAX with the intent that they cancel each other out.

Change-Id: I214d072458e1b24f60289974e6302af1aff7b66c
vp9/encoder/rdopt.c