]> granicus.if.org Git - libvpx/commit
Fix overflow issue in SSSE3 32x32 quantization
authorJingning Han <jingning@google.com>
Mon, 26 Aug 2013 23:12:16 +0000 (16:12 -0700)
committerJingning Han <jingning@google.com>
Thu, 29 Aug 2013 18:00:54 +0000 (11:00 -0700)
commitabff678866f9f74c58616705b01d460efb994fe2
treea3768fa018a615bc39b644663d70a663da4a866f
parent166dc85bed481decb5fdd572a3300d8c6f83887e
Fix overflow issue in SSSE3 32x32 quantization

The 32x32 quantization process can potentially have the intermediate
stacks over 16-bit range, thereby causing enc/dec mismatch. This commit
fixes this overflow issue in the SSSE3 implementation, as well as the
prototype, of 32x32 quantization.

This fixes issue 607 from webm@googlecode.

Change-Id: I85635e6ca236b90c3dcfc40d449215c7b9caa806
vp9/common/vp9_rtcd_defs.sh
vp9/encoder/vp9_quantize.c
vp9/encoder/x86/vp9_quantize_ssse3.asm