From: James Zern <jzern@google.com>
Date: Thu, 21 Dec 2017 21:51:40 +0000 (-0500)
Subject: vp9_quantize_ssse3_x86_64: fix out of bounds write
X-Git-Tag: v1.7.0~8^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=84a7263d4c124919718aca2a7eef1a218216917b;p=libvpx

vp9_quantize_ssse3_x86_64: fix out of bounds write

eob is a pointer to a uint16_t. previously the code would store 64-bits
causing a crash or test failure with the right stack layout.

Change-Id: Ibd653baf323db114f2444951b9d8b00c596bf15a
---

diff --git a/vp9/encoder/x86/vp9_quantize_ssse3_x86_64.asm b/vp9/encoder/x86/vp9_quantize_ssse3_x86_64.asm
index 1d421f6ec..5703aa3bb 100644
--- a/vp9/encoder/x86/vp9_quantize_ssse3_x86_64.asm
+++ b/vp9/encoder/x86/vp9_quantize_ssse3_x86_64.asm
@@ -169,7 +169,7 @@ cglobal quantize_%1, 0, %2, 15, coeff, ncoeff, skip, round, quant, \
   pshuflw                         m7, m8, 0x1
   pmaxsw                          m8, m7
   pextrw                          r6, m8, 0
-  mov                           [r2], r6
+  mov                           [r2], r6w
   RET
 %endmacro