]> granicus.if.org Git - libvpx/commit
quantize 32x32: fix dqcoeff
authorJohann <johann.koenig@duck.com>
Wed, 21 Nov 2018 18:30:45 +0000 (13:30 -0500)
committerJohann <johann.koenig@duck.com>
Wed, 28 Nov 2018 16:30:37 +0000 (11:30 -0500)
commitd566160f32580e5ef8fccbce8dde8495756a0620
treef38b10bc839ba540085ff5a37501fc898d0e69b6
parent0eeb7975128997b3f978d3a6c81d6e25b4d4a8c9
quantize 32x32: fix dqcoeff

Calculate the high bits of dqcoeff and store them appropriately in high
bit depth builds.

Low bit depth builds still do not pass. C truncates the results after
division. X86 only supports packing with saturation at this step.

BUG=webm:1448

Change-Id: Ic80def575136c7ca37edf18d21e26925b475da98
test/vp9_quantize_test.cc
vpx_dsp/vpx_dsp.mk
vpx_dsp/x86/quantize_avx.c
vpx_dsp/x86/quantize_sse2.h
vpx_dsp/x86/quantize_ssse3.c
vpx_dsp/x86/quantize_ssse3.h [new file with mode: 0644]