Fix various issues reported by the UB sanitizers
Most of these involved left shifting a negative number, which is
technically undefined (although every modern compiler I'm aware of
will implement this by treating the signed integer as a 2's complement
unsigned integer-- the LEFT_SHIFT() macro just makes this behavior
explicit in order to shut up ubsan.) This also fixes a couple of
non-issues in the entropy codecs, whereby the sanitizer reported an
out-of-bounds index in the 4th argument of jpeg_make_d_derived_tbl().
In those cases, the index was actually out of bounds (caused by a
malformed JPEG image), but jpeg_make_d_derived_tbl() would have caught
the error and aborted prior to actually using the invalid address. Here
again, the fix was to make our intentions explicit so as to shut up
ubsan.
James Cowgill [Thu, 17 Sep 2015 04:05:46 +0000 (23:05 -0500)]
Fix MIPS DSPr2 4:2:0 upsample bug w/ small images
The DSPr2 code was errantly comparing the residual (t9, width & 0xF)
with the end pointer (t4, out + width) instead of the width directly
(a1). This would give the wrong results with any image whose output
width was less than 16. The other small changes (ulw to lw and removal
of the nop) are just some easy optimizations around this code.
This issue caused a buffer overrun and subsequent segfault on images
whose scaled output height was 1 pixel and whose scaled output width was
< 16 pixels. Note that the "plain" (non-fancy and non-merged) upsample
routine, which was affected by this bug, is normally not used except
when decompressing a non-YCbCr JPEG image, but it is also used when
decompressing a single-row image (because the other upsampling
algorithms require at least two rows.)
Fix x86-64 ABI conformance issue in SIMD code
(descriptions cribbed by DRC from discussion in #20)
In the x86-64 ABI, the high (unused) DWORD of a 32-bit argument's
register is undefined, so it was incorrect to use a 64-bit mov
instruction to transfer a JDIMENSION argument in the 64-bit SSE2 SIMD
functions. The code worked thus far only because the existing compiler
optimizers weren't smart enough to do anything else with the register in
question, so the upper 32 bits happened to be all zeroes-- for the past
6 years, on every x86-64 compiler previously known to mankind.
The bleeding-edge Clang/LLVM compiler has a smarter optimizer, and
under certain circumstances, it will attempt to load-combine adjacent
32-bit integers from one of the libjpeg structures into a single 64-bit
integer and pass that 64-bit integer as a 32-bit argument to one of the
SIMD functions (which is allowed by the ABI, since the upper 32 bits of
the 32-bit argument's register are undefined.) This caused the
libjpeg-turbo regression tests to crash.
Also enhance the documentation of JDIMENSION to explain that its size
is significant to the implementation of the SIMD code.
Closes #20. Refer also to http://crbug.com/532214.
Add file that explains the libjpeg-turbo licenses
Previously this information was found in a page on libjpeg-turbo.org,
but there was still some confusion, because README-turbo.txt wasn't
clear as to which license applied to what.
DRC [Sat, 29 Aug 2015 23:05:43 +0000 (18:05 -0500)]
Fix negative shift with IFAST FDCT and qual=100
With certain images, compressing using quality=100 and the fast integer
forward DCT will cause the divisor passed to compute_reciprocal() to be
1. In those cases, the library already disables the SIMD quantization
algorithm to avoid 16-bit overflow. However, compute_reciprocal()
doesn't properly handle the divisor==1 case, so we need to use special
values in that case so that the C quantization algorithm will behave
like an identity function.
DRC [Thu, 13 Aug 2015 16:09:05 +0000 (11:09 -0500)]
Fix cjpeg segfault when Windows BMP width/height<0
rdbmp.c used the ambiguous INT32 datatype, which is sometimes typedef'ed
to long. Windows bitmap headers use 32-bit signed integers for the
width and height, because height can sometimes be negative (this
indicates a top-down bitmap.) If biWidth or biHeight was negative and
INT32 was a 64-bit long, then biWidth and biHeight were read as a
positive integer > INT32_MAX, which failed the test in line 385:
if (biWidth <= 0 || biHeight <= 0)
ERREXIT(cinfo, JERR_BMP_EMPTY);
This commit refactors rdbmp.c so that it uses the datatypes specified by
Microsoft for the Windows BMP header.
This closes #9 and also provides a better solution for mozilla/mozjpeg#153.
DRC [Tue, 4 Aug 2015 04:59:12 +0000 (23:59 -0500)]
Update x86[-64] assembler recommendations
NASM 2.11.08 has a bug that prevents it from properly assembling a
macho64 version of libjpeg-turbo (the resulting binary generates corrupt
images.) 2.11.09 works properly. YASM also works properly and has been
a supported alternative since libjpeg-turbo 1.2.
DRC [Tue, 4 Aug 2015 04:58:40 +0000 (23:58 -0500)]
Update x86[-64] assembler recommendations
NASM 2.11.08 has a bug that prevents it from properly assembling a
macho64 version of libjpeg-turbo (the resulting binary generates corrupt
images.) 2.11.09 works properly. YASM also works properly and has been
a supported alternative since libjpeg-turbo 1.2.
DRC [Tue, 4 Aug 2015 04:56:09 +0000 (23:56 -0500)]
Update x86[-64] assembler recommendations
NASM 2.11.08 has a bug that prevents it from properly assembling a
macho64 version of libjpeg-turbo (the resulting binary generates corrupt
images.) 2.11.09 works properly. YASM also works properly and has been
a supported alternative since libjpeg-turbo 1.2.
Fix rare bug: right shift by a negative # of bits
Under very rare circumstances, decompressing specific corrupt JPEG
images would create a situation whereby GET_BITS(1) was invoked
from within HUFF_DECODE_FAST() when bits_left=0. This produced a right
shift by a negative number of bits, which is undefined in C.
Fix rare bug: right shift by a negative # of bits
Under very rare circumstances, decompressing specific corrupt JPEG
images would create a situation whereby GET_BITS(1) was invoked
from within HUFF_DECODE_FAST() when bits_left=0. This produced a right
shift by a negative number of bits, which is undefined in C.
Fix rare bug: right shift by a negative # of bits
Under very rare circumstances, decompressing specific corrupt JPEG
images would create a situation whereby GET_BITS(1) was invoked
from within HUFF_DECODE_FAST() when bits_left=0. This produced a right
shift by a negative number of bits, which is undefined in C.
Add additional protections against defining INT32 if another header has already defined it (code borrowed from libjpeg v8.) This isn't necessary when using the libjpeg-turbo build system on Windows, because the CMake generated jconfig.h defines INT32 and then defines XMD_H to trick jmorecfg.h into not redefining it. However, some projects build libjpeg-turbo using their own build systems.
DRC [Mon, 8 Jun 2015 18:31:34 +0000 (18:31 +0000)]
Now that the TurboJPEG API is reporting libjpeg warnings as errors, an "Invalid SOS parameters for sequential JPEG" warning surfaced in tjDecodeYUV*(). This was caused by the Se member of jpeg_decompress_struct being set to 0 (it is normally set to a non-zero value when the start-of-scan markers are read, but there are no SOS markers in this case, because we're not actually decompressing a JPEG file.)
DRC [Mon, 8 Jun 2015 17:41:34 +0000 (17:41 +0000)]
Fix a segfault that occured in the MIPS DSPr2 fancy upsampling routine when downsampled_width==3. Because the DSPr2 code unrolls the loop for the middle columns (refer to jdsample.c), it has the effect of performing two column iterations, and that only works properly if the number of columns (minus the first and last) is >= 2. For the specific case of downsampled_width==3, this patch skips to the second iteration of the unrolled column loop.
DRC [Mon, 1 Jun 2015 19:22:41 +0000 (19:22 +0000)]
If a warning (such as "Premature end of JPEG file") is triggered in the underlying libjpeg API, make sure that the TurboJPEG API function returns -1. Unlike errors, however, libjpeg warnings do not make the TurboJPEG functions abort.
DRC [Sun, 17 May 2015 15:56:18 +0000 (15:56 +0000)]
Back out r1555 and r1548. Using setenv() didn't fix the iOS simulator issue. It just replaced an undefined _putenv$UNIX2003 symbol with an undefined _setenv$UNIX2003 symbol. The correct solution seems to be to use -D_NONSTD_SOURCE when generating our official builds.
DRC [Sat, 16 May 2015 04:18:21 +0000 (04:18 +0000)]
Fix the Windows build. I remember now why I used putenv() originally-- because Windows doesn't have setenv(). We could use _putenv_s(), but older versions of MinGW don't have that either. Fortunately, since all of the environment values we're setting in turbojpeg.c are static, we can just map setenv() to putenv() using a macro. NOTE: we still have to use _putenv_s() in turbojpeg-jni.c, but at least people who may need to build with an older version of MinGW can still do so by disabling the Java build.
DRC [Fri, 15 May 2015 19:09:44 +0000 (19:09 +0000)]
__WORDSIZE doesn't seem to be available on platforms other than Mac or Linux, and best practices are for user-level code not to rely on it anyhow, since it's meant to be an internal macro. Fortunately, autoconf already has a way of determining the word size at configure time, so it can be passed into the compiler. This should work on any platform and has been tested on all of the Un*x platforms we support (Linux, Mac, FreeBSD, Solaris.)
DRC [Fri, 15 May 2015 18:23:59 +0000 (18:23 +0000)]
Unless you define _ANSI_SOURCE, then putenv() on Mac is renamed to putenv$UNIX2003(), and this causes problems when trying to link an i386 iOS application (for the simulator) against the TurboJPEG static library. It's easiest to just use setenv() instead.
DRC [Wed, 6 May 2015 22:41:12 +0000 (22:41 +0000)]
Fix a bug in the 64-bit Huffman encoder that Google discovered when encoding some very specific (and proprietary) aerial images using quality=98, an optimized Huffman table, and the ISLOW DCT. These images were causing the Huffman bit buffer to overflow, because the code for encoding the DC coefficient was using the equivalent of the 32-bit version of EMIT_BITS(). Thus, when 64-bit code was used, the DC coefficient code was not properly checking how many bits were in the buffer before attempting to add more bits to it. This issue appears to have existed in all versions of libjpeg-turbo.
DRC [Thu, 19 Mar 2015 19:27:40 +0000 (19:27 +0000)]
Allow the executables and libraries outside of the sharedlib/ directory to be linked against msvcr*.dll instead of libcmt*.lib. This is reported to be necessary when building libjpeg-turbo for use with C#.
DRC [Mon, 23 Feb 2015 19:19:40 +0000 (19:19 +0000)]
Surround the usage of getenv() in the TurboJPEG API with #ifndef NO_GETENV so that developers can add -DNO_GETENV to the C flags when building for platforms that don't have getenv(). Currently this is known to be necessary when building for Windows Phone.
DRC [Mon, 23 Feb 2015 19:06:44 +0000 (19:06 +0000)]
If libjpeg-turbo is configured with a non-default prefix, such as /usr, then use the docdir variable defined by autoconf 2.60 and later, if available. This will, for instance, install the documentation under /usr/share/doc/libjpeg-turbo by default if prefix=/usr, unless docdir is overridden. When using earlier versions of autoconf, docdir is set to ${datadir}/doc, as it always has been.
DRC [Mon, 23 Feb 2015 19:03:29 +0000 (19:03 +0000)]
Enable silent build rules for the NASM objects, if the source is configured with automake 1.11 or later. NOTE: the build still spits out "error: ignoring unknown tag NASM" for each object, but unfortunately, if we remove "--tag NASM" from the command line, the build breaks under older versions of automake (it aborts with "unable to infer tagged configuration.")
DRC [Tue, 27 Jan 2015 21:00:22 +0000 (21:00 +0000)]
Oops. Need to set the alpha channel when using TYPE_4BYTE_ABGR*. This has no bearing on the actual tests, but it prevents the PNG pre-encode reference images for those tests from being blank.
DRC [Tue, 27 Jan 2015 20:59:16 +0000 (20:59 +0000)]
Oops. Need to set the alpha channel when using TYPE_4BYTE_ABGR*. This has no bearing on the actual tests, but it prevents the PNG pre-encode reference images for those tests from being blank.
DRC [Wed, 21 Jan 2015 17:42:28 +0000 (17:42 +0000)]
Oops. The MIPS SIMD implementations of h2v1 and h2v2 upsampling were not checking for DSPr2 support, so running 'djpeg -nosmooth' on a non-DSPr2-enabled platform caused an "illegal instruction" error.
DRC [Tue, 20 Jan 2015 10:33:32 +0000 (10:33 +0000)]
Introduce fast paths to speed up NULL color conversion somewhat, particularly when using 64-bit code; on the decompression side, the "slow path" also now use an approach similar to that of the compression side (with the component loop outside of the column loop rather than inside.) This is faster when using 32-bit code.
DRC [Fri, 16 Jan 2015 06:53:36 +0000 (06:53 +0000)]
Make the floating point regression tests optional. It has been known for quite some time that these tests do not always generate the same results unless there is full SIMD coverage of the floating point algorithms in libjpeg-turbo. Further research reveals that there are basically three expected results: the results from our SSE SIMD extensions (which are slightly more accurate than the C code), results from the C code when running on a 32-bit FPU (or when using SSE instructions on an x86-64 CPU, which is the default with GCC), and results from the C code when running on a 64-bit FPU (which presumably uses double-precision arithmetic by default.) There is basically no way to determine which type of math will be used prior to run time, so it's best to just let the developers specify which result they expect on their particular system.
DRC [Fri, 16 Jan 2015 06:45:54 +0000 (06:45 +0000)]
In the process of developing the AltiVec extensions, it was discovered that the normal regression tests aren't sufficient to test the behavior of the library with very small image sizes and when compressing from/decompressing to a subregion of a larger image buffer. Thus, an additional regression test was added that takes advantage of the tiled compression/decompression feature in tjbench. This is being back-ported to the 1.4.x branch primarily to verify that there are no lingering issues in the existing SIMD extensions.
DRC [Fri, 16 Jan 2015 06:37:03 +0000 (06:37 +0000)]
Add separate pseudo-targets for the TurboJPEG and libjpeg regression tests, for those times when you just don't want to sit through 11 iterations of TJUnitTest to find out that your algorithm is broken.
DRC [Fri, 16 Jan 2015 06:29:52 +0000 (06:29 +0000)]
Add the ability to benchmark YCCK JPEG compression/decompression. This is particularly useful since that is the only way to test the performance of the "plain" upsampling routines, which are accelerated on some platforms.
DRC [Fri, 19 Dec 2014 15:36:39 +0000 (15:36 +0000)]
Modify the ARM64 assembly file so that it uses only syntax that the clang assembler in XCode 5.x can understand. These changes should all be cosmetic in nature-- they do not change the meaning or readability of the code nor the ability to build it for Linux. Actually, the code is now more in compliance with the ARM64 programming manual. In addition to these changes, there were a couple of instructions that clang simply doesn't support, so gas-preprocessor.pl was modified so that it now converts those into equivalent instructions that clang can handle.
DRC [Fri, 19 Dec 2014 10:44:09 +0000 (10:44 +0000)]
Add iOS architectures to the shared libraries generated by the Mac/iOS packaging system. I have no idea how useful this is for "standard" iOS application development, but it is useful in a jailbreak environment, and iOS 8 supposedly allows shared libs in "official" apps as well.
DRC [Tue, 25 Nov 2014 09:48:15 +0000 (09:48 +0000)]
Restore the JPP() and JMETHOD() macros. Even though libjpeg-turbo doesn't use them anymore, other software apparently does:
https://bugzilla.redhat.com/show_bug.cgi?id=1164815
https://bugs.kde.org/show_bug.cgi?id=340944
https://bugzilla.mozilla.org/show_bug.cgi?id=1093615
DRC [Sat, 22 Nov 2014 22:24:41 +0000 (22:24 +0000)]
Fix Huffman local buffer overrun discovered by Debian developers when attempting to transform a junk image using ImageMagick:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=768369
DRC [Sat, 22 Nov 2014 22:09:30 +0000 (22:09 +0000)]
Fix Huffman local buffer overrun discovered by Debian developers when attempting to transform a junk image using ImageMagick:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=768369
DRC [Fri, 21 Nov 2014 15:33:19 +0000 (15:33 +0000)]
Sometimes the sampling factors in grayscale images can be > 1 (for instance, if compressing using 'cjpeg -sample 2x2 -grayscale'.) Technically, sampling factors have no meaning with grayscale JPEGs, and the libjpeg decompressor ignores them in that case. Thus, the TurboJPEG decompressor should ignore them as well.
DRC [Wed, 19 Nov 2014 00:55:28 +0000 (00:55 +0000)]
Sometimes the sampling factors in grayscale images can be > 1 (for instance, if compressing using 'cjpeg -sample 2x2 -grayscale'.) Technically, sampling factors have no meaning with grayscale JPEGs, and the libjpeg decompressor ignores them in that case. Thus, the TurboJPEG decompressor should ignore them as well.
projects / libjpeg-turbo / log