DRC [Sun, 1 May 2016 16:42:15 +0000 (11:42 -0500)]
Fix CMake fallback BUILD var on non-U.S. machines
If wmic.exe wasn't available, then CMakeLists.txt would call
"cmd /C date /T" and parse the result in order to set the BUILD
variable. However, the parser assumed that the date was in MM/DD/YYYY
format, which is not generally the case unless the user's locale is U.S.
English with the default region/language settings for that locale.
This commit modifies CMakeLists.txt such that it uses the
string(TIMESTAMP) function available in CMake 2.8.11 and later to set
the BUILD variable, thus eliminating the need to use wmic.exe or any
other platform-specific hack.
This commit also modifies the build instructions to remove any reference
to CMake 2.6 (which hasn't been supported by our build system since
libjpeg-turbo 1.3.x.)
At one time, it was possible to use CMake to build under Cygwin, but
that hasn't worked since 1.4.1 (due to the Huffman codec changes that
now require SIZEOF_SIZE_T to be defined for non-WIN32 platforms) and may
have even been broken before that. Originally, we used the "date"
command under MSYS in order to obtain the default build number, but that
was rendered unnecessary by 5e3bb3e9 (v1.3 beta.) 9fe22dac (1.4 beta)
further modified CMakeLists.txt so that the "date" command was only used
on Cygwin, but for unexplained reasons, that commit also applied the
(now vestigial) code to all non-WIN32 platforms. This prevented
CMakeLists.txt from displaying an error if someone attempted to use the
CMake build system on Un*x platforms, and that may have been behind the
flurry of pull requests and issues-- including #21, #29, #37, #58, #73--
complaining that the CMake build system didn't work on Un*x platforms
(although it was not until #73 that this bug came to light.)
This commit removes all vestiges of Un*x support from the CMake build
system and makes it clear that CMake cannot be used to build
libjpeg-turbo on non-WIN32 platforms. It is our position that CMake
will not be supported on non-WIN32 platforms until/unless the autotools
build system is removed, and this will not happen without broad support
from the community (including major O/S vendors.) If you are in favor
of migrating the entire build system to CMake, then please make your
voice heard by commenting on #56.
Even though tjDecompressToYUV2() is mostly just a wrapper for
tjDecompressToYUVPlanes(), tjDecompressToYUV2() still calls
jpeg_read_header(), so it needs to properly set up the libjpeg error
handler prior to making this call. Otherwise, under very esoteric (and
arguably incorrect) use cases, a program could call tjDecompressToYUV2()
without first checking the JPEG header using tjDecompressHeader3(), and
if the header was corrupt, then the libjpeg API would invoke
my_error_exit(). my_error_exit() would in turn call longjmp() on the
previous value of myerr->setjmp_buffer, which was probably set in a
previous TurboJPEG function, such as tjInitDecompress(). Thus, when a
libjpeg error was triggered within the body of tjDecompressToYUV2(), the
PC would jump to the error handler of the previous TurboJPEG function,
and this usually caused stack corruption in the calling program (because
the signature and return type of the previous TurboJPEG function
probably wasn't the same as that of tjDecompressToYUV2().)
DRC [Thu, 31 Mar 2016 14:49:49 +0000 (09:49 -0500)]
cjpeg: Fix buf overrun caused by bad bin PPM input
This extends the fix in 6709e4a0cfa44d4f54ee8ad05753d4aa9260cb91 to
include binary PPM/PGM files, thus preventing a malformed binary
PPM/PGM input file from triggering an overrun of the rescale array and
potentially crashing cjpeg.
Note that this issue affected only cjpeg and not the underlying
libjpeg-turbo libraries, and thus it did not represent a security
threat.
DRC [Wed, 16 Mar 2016 12:10:35 +0000 (07:10 -0500)]
Add version/build info to global string table
This is a common practice in other infrastructure libraries, such as
OpenSSL and libpng, because it makes it easy to examine an application
binary and determine which version of the library the application was
linked against.
DRC [Sun, 6 Mar 2016 14:15:04 +0000 (08:15 -0600)]
Ensure that default Huffman tables are initialized
This prevents a malformed motion-JPEG frame (MJPEG frames lack Huffman
tables) from causing the "fast path" of the Huffman decoder to read
uninitialized memory. Essentially, this is doing the same thing for
MJPEG frames as 43d8cf4d4572fa50a37cccadbe71b9bee37de55d did for regular
images.
DRC [Fri, 5 Feb 2016 00:34:38 +0000 (18:34 -0600)]
Prevent overread when decoding malformed JPEG
The accelerated Huffman decoder was previously invoked if there were
> 128 bytes in the input buffer. However, it is possible to construct a
JPEG image with Huffman blocks > 430 bytes in length
(http://stackoverflow.com/questions/2734678/jpeg-calculating-max-size).
While such images are pathological and could never be created by a
JPEG compressor, it is conceivable that an attacker could use such an
artifially-constructed image to trigger an input buffer overrun in the
libjpeg-turbo decompressor and thus gain access to some of the data on
the calling program's heap.
This patch simply increases the minimum buffer size for the accelerated
Huffman decoder to 512 bytes, which should (hopefully) accommodate any
possible input.
This addresses a major issue (LJT-01-005) identified in a security audit
by Cure53.
DRC [Thu, 4 Feb 2016 16:59:21 +0000 (10:59 -0600)]
Guard against wrap-around in alloc functions
Because of the exposed nature of the libjpeg API, alloc_small() and
alloc_large() can potentially be called by external code. If an
application were to call either of those functions with
sizeofobject > SIZE_MAX - ALIGN_SIZE - 1, then the math in
round_up_pow2() would wrap around to zero, causing that function to
return a small value. That value would likely not exceed
MAX_ALLOC_CHUNK, so the subsequent size checks in alloc_small() and
alloc_large() would not catch the error.
A similar problem could occur in 32-bit builds if alloc_sarray() were
called with
samplesperrow > SIZE_MAX - (2 * ALIGN_SIZE / sizeof(JSAMPLE)) - 1
This patch simply ensures that the size argument to the alloc_*()
functions will never exceed MAX_ALLOC_CHUNK (1 billion). If it did,
then subsequent size checks would eventually catch that error, so we
are instead catching the error before round_up_pow2() is called.
This addresses a minor concern (LJT-01-001) expressed in a security
audit by Cure53.
DRC [Thu, 4 Feb 2016 15:20:41 +0000 (09:20 -0600)]
TurboJPEG: Avoid dangling pointers
This addresses a minor concern (LJT-01-002) expressed in a security
audit by Cure53. _tjInitCompress() and _tjInitDecompress() call
(respectively) jpeg_mem_dest_tj() and jpeg_mem_src_tj() with a pointer
to a dummy buffer, in order to set up the destination/source manager.
The dummy buffer should never be used, but it's still better to make it
static so that the pointer in the destination/source manager always
points to a valid region of memory.
DRC [Mon, 1 Feb 2016 17:28:55 +0000 (11:28 -0600)]
Update Android build instr. for ARMv8, PIE, etc.
* Include information on how to do a 64-bit ARMv8 build with the latest
NDK
* Suggest -fPIE and -pie as default CFLAGS (required for android-16 and
later.
* Remove -fstrict-aliasing flag (-Wall already includes it)
DRC [Sat, 20 Jun 2015 16:27:51 +0000 (16:27 +0000)]
Update build instructions for new autoconf, GitHub
The Linux build machine has been upgraded to autoconf 2.69, automake
1.15, m4 1.4.17, and libtool 2.4.6, so it is no longer necessary to
recommend running autoreconf prior to building the source, if one is
building from an official source tarball (as opposed to from a git
sandbox.) Also, there is no SVN repository anymore (oops.)
DRC [Thu, 7 Jan 2016 01:17:54 +0000 (19:17 -0600)]
Regression: Allow co-install of 32-bit/64-bit RPMs
Fix a regression introduced in 1.4.1 that prevented 32-bit and 64-bit
libjpeg-turbo RPMs from being installed simultaneously on recent Red
Hat/Fedora distributions. This was due to the addition of the
SIZEOF_SIZE_T macro in jconfig.h, which allows the Huffman codec to
determine the word size at compile time. Since that macro differs
between 32-bit and 64-bit builds, this caused a conflict between the
i386 and x86_64 RPMs (any differing files, other than executables, are
not allowed when 32-bit and 64-bit RPMs are installed simultaneously.)
Since the macro is used only internally, it has been moved into
jconfigint.h.
Fix various issues reported by the UB sanitizers
Most of these involved left shifting a negative number, which is
technically undefined (although every modern compiler I'm aware of
will implement this by treating the signed integer as a 2's complement
unsigned integer-- the LEFT_SHIFT() macro just makes this behavior
explicit in order to shut up ubsan.) This also fixes a couple of
non-issues in the entropy codecs, whereby the sanitizer reported an
out-of-bounds index in the 4th argument of jpeg_make_d_derived_tbl().
In those cases, the index was actually out of bounds (caused by a
malformed JPEG image), but jpeg_make_d_derived_tbl() would have caught
the error and aborted prior to actually using the invalid address. Here
again, the fix was to make our intentions explicit so as to shut up
ubsan.
James Cowgill [Thu, 17 Sep 2015 04:05:46 +0000 (23:05 -0500)]
Fix MIPS DSPr2 4:2:0 upsample bug w/ small images
The DSPr2 code was errantly comparing the residual (t9, width & 0xF)
with the end pointer (t4, out + width) instead of the width directly
(a1). This would give the wrong results with any image whose output
width was less than 16. The other small changes (ulw to lw and removal
of the nop) are just some easy optimizations around this code.
This issue caused a buffer overrun and subsequent segfault on images
whose scaled output height was 1 pixel and whose scaled output width was
< 16 pixels. Note that the "plain" (non-fancy and non-merged) upsample
routine, which was affected by this bug, is normally not used except
when decompressing a non-YCbCr JPEG image, but it is also used when
decompressing a single-row image (because the other upsampling
algorithms require at least two rows.)
Fix x86-64 ABI conformance issue in SIMD code
(descriptions cribbed by DRC from discussion in #20)
In the x86-64 ABI, the high (unused) DWORD of a 32-bit argument's
register is undefined, so it was incorrect to use a 64-bit mov
instruction to transfer a JDIMENSION argument in the 64-bit SSE2 SIMD
functions. The code worked thus far only because the existing compiler
optimizers weren't smart enough to do anything else with the register in
question, so the upper 32 bits happened to be all zeroes-- for the past
6 years, on every x86-64 compiler previously known to mankind.
The bleeding-edge Clang/LLVM compiler has a smarter optimizer, and
under certain circumstances, it will attempt to load-combine adjacent
32-bit integers from one of the libjpeg structures into a single 64-bit
integer and pass that 64-bit integer as a 32-bit argument to one of the
SIMD functions (which is allowed by the ABI, since the upper 32 bits of
the 32-bit argument's register are undefined.) This caused the
libjpeg-turbo regression tests to crash.
Also enhance the documentation of JDIMENSION to explain that its size
is significant to the implementation of the SIMD code.
Closes #20. Refer also to http://crbug.com/532214.
Add file that explains the libjpeg-turbo licenses
Previously this information was found in a page on libjpeg-turbo.org,
but there was still some confusion, because README-turbo.txt wasn't
clear as to which license applied to what.
DRC [Sat, 29 Aug 2015 23:05:43 +0000 (18:05 -0500)]
Fix negative shift with IFAST FDCT and qual=100
With certain images, compressing using quality=100 and the fast integer
forward DCT will cause the divisor passed to compute_reciprocal() to be
1. In those cases, the library already disables the SIMD quantization
algorithm to avoid 16-bit overflow. However, compute_reciprocal()
doesn't properly handle the divisor==1 case, so we need to use special
values in that case so that the C quantization algorithm will behave
like an identity function.
DRC [Thu, 13 Aug 2015 16:09:05 +0000 (11:09 -0500)]
Fix cjpeg segfault when Windows BMP width/height<0
rdbmp.c used the ambiguous INT32 datatype, which is sometimes typedef'ed
to long. Windows bitmap headers use 32-bit signed integers for the
width and height, because height can sometimes be negative (this
indicates a top-down bitmap.) If biWidth or biHeight was negative and
INT32 was a 64-bit long, then biWidth and biHeight were read as a
positive integer > INT32_MAX, which failed the test in line 385:
if (biWidth <= 0 || biHeight <= 0)
ERREXIT(cinfo, JERR_BMP_EMPTY);
This commit refactors rdbmp.c so that it uses the datatypes specified by
Microsoft for the Windows BMP header.
This closes #9 and also provides a better solution for mozilla/mozjpeg#153.
DRC [Tue, 4 Aug 2015 04:59:12 +0000 (23:59 -0500)]
Update x86[-64] assembler recommendations
NASM 2.11.08 has a bug that prevents it from properly assembling a
macho64 version of libjpeg-turbo (the resulting binary generates corrupt
images.) 2.11.09 works properly. YASM also works properly and has been
a supported alternative since libjpeg-turbo 1.2.
DRC [Tue, 4 Aug 2015 04:58:40 +0000 (23:58 -0500)]
Update x86[-64] assembler recommendations
NASM 2.11.08 has a bug that prevents it from properly assembling a
macho64 version of libjpeg-turbo (the resulting binary generates corrupt
images.) 2.11.09 works properly. YASM also works properly and has been
a supported alternative since libjpeg-turbo 1.2.
DRC [Tue, 4 Aug 2015 04:56:09 +0000 (23:56 -0500)]
Update x86[-64] assembler recommendations
NASM 2.11.08 has a bug that prevents it from properly assembling a
macho64 version of libjpeg-turbo (the resulting binary generates corrupt
images.) 2.11.09 works properly. YASM also works properly and has been
a supported alternative since libjpeg-turbo 1.2.
Fix rare bug: right shift by a negative # of bits
Under very rare circumstances, decompressing specific corrupt JPEG
images would create a situation whereby GET_BITS(1) was invoked
from within HUFF_DECODE_FAST() when bits_left=0. This produced a right
shift by a negative number of bits, which is undefined in C.
Fix rare bug: right shift by a negative # of bits
Under very rare circumstances, decompressing specific corrupt JPEG
images would create a situation whereby GET_BITS(1) was invoked
from within HUFF_DECODE_FAST() when bits_left=0. This produced a right
shift by a negative number of bits, which is undefined in C.
Fix rare bug: right shift by a negative # of bits
Under very rare circumstances, decompressing specific corrupt JPEG
images would create a situation whereby GET_BITS(1) was invoked
from within HUFF_DECODE_FAST() when bits_left=0. This produced a right
shift by a negative number of bits, which is undefined in C.
Add additional protections against defining INT32 if another header has already defined it (code borrowed from libjpeg v8.) This isn't necessary when using the libjpeg-turbo build system on Windows, because the CMake generated jconfig.h defines INT32 and then defines XMD_H to trick jmorecfg.h into not redefining it. However, some projects build libjpeg-turbo using their own build systems.
DRC [Mon, 8 Jun 2015 18:31:34 +0000 (18:31 +0000)]
Now that the TurboJPEG API is reporting libjpeg warnings as errors, an "Invalid SOS parameters for sequential JPEG" warning surfaced in tjDecodeYUV*(). This was caused by the Se member of jpeg_decompress_struct being set to 0 (it is normally set to a non-zero value when the start-of-scan markers are read, but there are no SOS markers in this case, because we're not actually decompressing a JPEG file.)
DRC [Mon, 8 Jun 2015 17:41:34 +0000 (17:41 +0000)]
Fix a segfault that occured in the MIPS DSPr2 fancy upsampling routine when downsampled_width==3. Because the DSPr2 code unrolls the loop for the middle columns (refer to jdsample.c), it has the effect of performing two column iterations, and that only works properly if the number of columns (minus the first and last) is >= 2. For the specific case of downsampled_width==3, this patch skips to the second iteration of the unrolled column loop.
DRC [Mon, 1 Jun 2015 19:22:41 +0000 (19:22 +0000)]
If a warning (such as "Premature end of JPEG file") is triggered in the underlying libjpeg API, make sure that the TurboJPEG API function returns -1. Unlike errors, however, libjpeg warnings do not make the TurboJPEG functions abort.
DRC [Sun, 17 May 2015 15:56:18 +0000 (15:56 +0000)]
Back out r1555 and r1548. Using setenv() didn't fix the iOS simulator issue. It just replaced an undefined _putenv$UNIX2003 symbol with an undefined _setenv$UNIX2003 symbol. The correct solution seems to be to use -D_NONSTD_SOURCE when generating our official builds.
DRC [Sat, 16 May 2015 04:18:21 +0000 (04:18 +0000)]
Fix the Windows build. I remember now why I used putenv() originally-- because Windows doesn't have setenv(). We could use _putenv_s(), but older versions of MinGW don't have that either. Fortunately, since all of the environment values we're setting in turbojpeg.c are static, we can just map setenv() to putenv() using a macro. NOTE: we still have to use _putenv_s() in turbojpeg-jni.c, but at least people who may need to build with an older version of MinGW can still do so by disabling the Java build.
DRC [Fri, 15 May 2015 19:09:44 +0000 (19:09 +0000)]
__WORDSIZE doesn't seem to be available on platforms other than Mac or Linux, and best practices are for user-level code not to rely on it anyhow, since it's meant to be an internal macro. Fortunately, autoconf already has a way of determining the word size at configure time, so it can be passed into the compiler. This should work on any platform and has been tested on all of the Un*x platforms we support (Linux, Mac, FreeBSD, Solaris.)
DRC [Fri, 15 May 2015 18:23:59 +0000 (18:23 +0000)]
Unless you define _ANSI_SOURCE, then putenv() on Mac is renamed to putenv$UNIX2003(), and this causes problems when trying to link an i386 iOS application (for the simulator) against the TurboJPEG static library. It's easiest to just use setenv() instead.
DRC [Wed, 6 May 2015 22:41:12 +0000 (22:41 +0000)]
Fix a bug in the 64-bit Huffman encoder that Google discovered when encoding some very specific (and proprietary) aerial images using quality=98, an optimized Huffman table, and the ISLOW DCT. These images were causing the Huffman bit buffer to overflow, because the code for encoding the DC coefficient was using the equivalent of the 32-bit version of EMIT_BITS(). Thus, when 64-bit code was used, the DC coefficient code was not properly checking how many bits were in the buffer before attempting to add more bits to it. This issue appears to have existed in all versions of libjpeg-turbo.
DRC [Thu, 19 Mar 2015 19:27:40 +0000 (19:27 +0000)]
Allow the executables and libraries outside of the sharedlib/ directory to be linked against msvcr*.dll instead of libcmt*.lib. This is reported to be necessary when building libjpeg-turbo for use with C#.
DRC [Mon, 23 Feb 2015 19:19:40 +0000 (19:19 +0000)]
Surround the usage of getenv() in the TurboJPEG API with #ifndef NO_GETENV so that developers can add -DNO_GETENV to the C flags when building for platforms that don't have getenv(). Currently this is known to be necessary when building for Windows Phone.
DRC [Mon, 23 Feb 2015 19:06:44 +0000 (19:06 +0000)]
If libjpeg-turbo is configured with a non-default prefix, such as /usr, then use the docdir variable defined by autoconf 2.60 and later, if available. This will, for instance, install the documentation under /usr/share/doc/libjpeg-turbo by default if prefix=/usr, unless docdir is overridden. When using earlier versions of autoconf, docdir is set to ${datadir}/doc, as it always has been.
DRC [Mon, 23 Feb 2015 19:03:29 +0000 (19:03 +0000)]
Enable silent build rules for the NASM objects, if the source is configured with automake 1.11 or later. NOTE: the build still spits out "error: ignoring unknown tag NASM" for each object, but unfortunately, if we remove "--tag NASM" from the command line, the build breaks under older versions of automake (it aborts with "unable to infer tagged configuration.")
DRC [Tue, 27 Jan 2015 21:00:22 +0000 (21:00 +0000)]
Oops. Need to set the alpha channel when using TYPE_4BYTE_ABGR*. This has no bearing on the actual tests, but it prevents the PNG pre-encode reference images for those tests from being blank.
DRC [Tue, 27 Jan 2015 20:59:16 +0000 (20:59 +0000)]
Oops. Need to set the alpha channel when using TYPE_4BYTE_ABGR*. This has no bearing on the actual tests, but it prevents the PNG pre-encode reference images for those tests from being blank.
DRC [Wed, 21 Jan 2015 17:42:28 +0000 (17:42 +0000)]
Oops. The MIPS SIMD implementations of h2v1 and h2v2 upsampling were not checking for DSPr2 support, so running 'djpeg -nosmooth' on a non-DSPr2-enabled platform caused an "illegal instruction" error.
DRC [Tue, 20 Jan 2015 10:33:32 +0000 (10:33 +0000)]
Introduce fast paths to speed up NULL color conversion somewhat, particularly when using 64-bit code; on the decompression side, the "slow path" also now use an approach similar to that of the compression side (with the component loop outside of the column loop rather than inside.) This is faster when using 32-bit code.
DRC [Fri, 16 Jan 2015 06:53:36 +0000 (06:53 +0000)]
Make the floating point regression tests optional. It has been known for quite some time that these tests do not always generate the same results unless there is full SIMD coverage of the floating point algorithms in libjpeg-turbo. Further research reveals that there are basically three expected results: the results from our SSE SIMD extensions (which are slightly more accurate than the C code), results from the C code when running on a 32-bit FPU (or when using SSE instructions on an x86-64 CPU, which is the default with GCC), and results from the C code when running on a 64-bit FPU (which presumably uses double-precision arithmetic by default.) There is basically no way to determine which type of math will be used prior to run time, so it's best to just let the developers specify which result they expect on their particular system.
DRC [Fri, 16 Jan 2015 06:45:54 +0000 (06:45 +0000)]
In the process of developing the AltiVec extensions, it was discovered that the normal regression tests aren't sufficient to test the behavior of the library with very small image sizes and when compressing from/decompressing to a subregion of a larger image buffer. Thus, an additional regression test was added that takes advantage of the tiled compression/decompression feature in tjbench. This is being back-ported to the 1.4.x branch primarily to verify that there are no lingering issues in the existing SIMD extensions.
DRC [Fri, 16 Jan 2015 06:37:03 +0000 (06:37 +0000)]
Add separate pseudo-targets for the TurboJPEG and libjpeg regression tests, for those times when you just don't want to sit through 11 iterations of TJUnitTest to find out that your algorithm is broken.
DRC [Fri, 16 Jan 2015 06:29:52 +0000 (06:29 +0000)]
Add the ability to benchmark YCCK JPEG compression/decompression. This is particularly useful since that is the only way to test the performance of the "plain" upsampling routines, which are accelerated on some platforms.
projects / libjpeg-turbo / log