DRC [Thu, 14 Feb 2019 16:01:16 +0000 (10:01 -0600)]
appveyor.yml: Cache NASM binary package
... since www.nasm.us seems to be down frequently. This doesn't help us
at the moment, but hopefully once the site is back up this will prevent
future build failures.
When simd/arm/jsimd.c is compiled with __ARM_NEON__ defined (which will
be the case if -mfpu=neon is passed to the compiler), the
parse_proc_cpuinfo() and check_feature() functions and the bufsize
variable are unused and thus need to be #ifdef'ed out in order to avoid
compiler warnings. Note that the bufsize variable was already #ifdef'ed
out on Linux but not on Android due to lack of parentheses (&& takes
precedence over ||.)
%{_libdir}/pkgconfig is a directory and should thus be prefixed by
%{dir} (oops.) This issue caused the debuginfo build under RHEL 8
(which is apparently now enabled by default-- regardless of whether the
RPM actually contains debug info, but that's another matter) to fail
with:
DRC [Tue, 12 Feb 2019 21:37:21 +0000 (15:37 -0600)]
rpm.spec.in: Update deprecated [Build]Prereq tags
AFAICT, Requires and BuildRequires subsumed the functionality of Prereq
and BuildPrereq in RPM 4.0, and none of the platforms we support with
libjpeg-turbo 2.0.x has RPM < 4.4.
DRC [Fri, 25 Jan 2019 22:46:02 +0000 (16:46 -0600)]
Build: Fix install of static build w/ VStudio IDE
Unfortunately, this hack is necessary because:
- install(TARGETS, ...) doesn't support the RENAME option.
- We can't modify OUTPUT_NAME for the "-static" targets without breaking
the regression tests.
- ${CMAKE_CFG_INTDIR} doesn't seem to work properly in an install()
command.
Including the license templates was confusing to some, since it made
it appear as if the copyright year and author were unspecified for the
libjpeg-turbo source. Thus, rather than include the zlib License
template, link to that template on opensource.org. For the Modified BSD
License, include a roll-up of copyright years and authors, since the
terms of that license require the text of it to be included in product
documentation for binary distributions without accompanying source code.
DRC [Mon, 21 Jan 2019 19:57:55 +0000 (13:57 -0600)]
TurboJPEG: Decompress 4:4:4 JPEGs with unusual SFs
Normally, 4:4:4 JPEGs have horizontal x vertical luminance & chrominance
sampling factors of 1x1. However, it is technically legal to create
4:4:4 JPEGs with sampling factors of 2x1, 1x2, 3x1, or 1x3, since the
sums of the products of those sampling factors are still <= 10. The
libjpeg API correctly decodes such images, so the TurboJPEG API should
as well.
DRC [Wed, 21 Nov 2018 04:52:36 +0000 (22:52 -0600)]
Travis: Fetch GPG key from libjpeg-turbo.org
I give up on the public keyserver. It inexplicably just fails
sometimes. I was trying to use it out of an abundance of caution
(<cough> paranoia <cough>), but it seems like most open source projects
just serve up their public keys from their project web sites. The
private and public pre-release keys are still stored on separate sites,
the private key is still strongly encrypted by Travis, and we use a
separate key for pre-releases anyhow, so even if it's compromised, we
can quickly and easily deploy a new one.
DRC [Fri, 2 Nov 2018 16:55:29 +0000 (11:55 -0500)]
Travis: Don't check key server SSL certificate
... when downloading the RPM signing key. Apparently the key server
URL sometimes redirects to an https URL, which may explain why fetching
the RPM signing keys failed frequently when we used to run wget inside
of the CentOS 5 Docker container.
I have a hunch that this is related to the CentOS 5 Docker container, so
this commit causes Travis to download the RPM signing key outside of
the container and share it with the container.
DRC [Fri, 26 Oct 2018 13:55:22 +0000 (08:55 -0500)]
Android: Fix "using JNI after critical get" error
We shouldn't be making JNI calls between GetPrimitiveArrayCritical() and
ReleasePrimitiveArrayCritical(). Apparently Android is stricter about
this than desktop Java.
Fix JPEG spec references per ISO/ITU-T suggestions
- When referring to specific clauses, annexes, tables, and figures, a
"timed reference" (a reference that includes the year) must be used in
order to avoid confusion.
- "CCITT" = "ITU-T"
- Replace ambiguous "JPEG spec" with the specific document number.
... in which one or more of the color indices is out of range for the
number of palette entries.
Fix partly borrowed from jpeg-9c. This commit also adopts Guido's
JERR_PPM_OUTOFRANGE enum value in lieu of our project-specific
JERR_PPM_TOOLARGE enum value.
The sentence:
"Indeed, one of the original reasons for developing this free software
was to help force convergence on common, interoperable format standards
for JPEG files."
might be seen to imply that JPEG 2000 and JPEG XR are not interoperable
with themselves, although it is certainly the case that those formats
are not interoperable with each other, nor with
ITU T.81 | ISO/IEC 10918. They are also certainly not as common as
ITU T.81 | ISO/IEC 10918, and (as an example) popular web browsers will
not display JPEG 2000 files.
The sentence in question was originally referring to proprietary,
non-standard formats and was meant to provide historical context.
libjpeg was originally released prior to the adoption of JFIF as an
official standard, so it encouraged adoption of JFIF as a de facto
standard by providing, under a business-friendly free software license,
a library for reading and writing images in that format.
DRC [Sun, 24 Jun 2018 22:31:15 +0000 (17:31 -0500)]
Fix infinite loop in partial image decompression
... caused by using certain specific combinations of
jpeg_skip_scanlines() and jpeg_read_scanlines() calls with progressive,
vertically-subsampled JPEG images.
DRC [Wed, 13 Jun 2018 01:27:00 +0000 (20:27 -0500)]
rdbmp.c: Fix FPE triggered by malformed BMP
In rdbmp.c, it is necessary to guard against 32-bit overflow/wraparound
when allocating the row buffer, because since BMP files have 32-bit
width and height fields, the value of biWidth can be up to 4294967295.
Specifically, high values of biWidth could cause the samplesperrow
argument in alloc_sarray() to wrap around to 0, triggering a division by
zero error at line 460 in jmemmgr.c, or to wrap around to a small
number, likely triggering a buffer overflow.
This fix is not documented in the change log for this branch, because
the bug was exposed using the tjLoadImage() function in the 2.0.x
branch. However, it is posited that the issue could be triggered using
TJBench in this branch.
Arguably it doesn't make much sense for non-chroma components to be
subsampled (which is why this type of image was overlooked in cd7c3e6672cce3779450c6dd10d0d70b0c2278b2-- I didn't realize it was a
thing), but certain Adobe applications apparently generate these images.
This commit merges the following paragraph from the latest libjpeg
release:
https://github.com/libjpeg-turbo/ijg/blob/jpeg-9c/README#L222-L229
which takes into account the fact that JFIF is now an official ISO/ITU-T
standard. I also included the ISO/IEC document number for the JFIF spec
(jpeg-9c included only the ITU-T rec number.)
This commit also heavily wordsmiths the "FILE FORMAT WARS" section.
In jpeg-7 and later, this section has become somewhat impolitic,
referring to JPEG 2000 and JPEG XR as "faulty technologies" and
"momentary mistakes." The original intent of this section, which was
introduced in jpeg-5 and refined in jpeg-6
(https://github.com/libjpeg-turbo/ijg/blob/jpeg-5/README#L317-L338,
https://github.com/libjpeg-turbo/ijg/blob/jpeg-6b/README#L335-L367)
was to highlight the problem of JPEG file format divergence that existed
in the 1990s prior to the adoption of JFIF as an official ISO/ITU-T
standard. That problem is fortunately no longer a problem, thanks in
part to the existence of libjpeg. I have attempted to preserve Tom's
intent of using this section to describe which file formats the code is
compatible with and why it isn't compatible with some file formats
bearing the name "JPEG." Such modifications always put our project in a
very awkward position, because we are not the IJG and do not claim to
be, but it is still necessary for us to modify the IJG README file from
time to time to eliminate obsolete information while attempting to
remain as neutral as possible.
DRC [Sat, 31 Mar 2018 15:36:06 +0000 (10:36 -0500)]
Build: Use Colin Plumb's public domain MD5 code
... instead of the RSA code, the license for which contains an
advertising clause. It is strongly believed that the RSA advertising
clause is innocuous, because:
- A clarification from RSA
(http://www.ietf.org/ietf-ftp/IPR/RSA-MD-all), published in 2000,
stated:
"Implementations of these message-digest algorithms, including
implementations derived from the reference C code in RFC-1319,
RFC-1320, and RFC-1321, may be made, used, and sold without license
from RSA for any purpose."
Referring to the opinion from Fedora's legal team
(https://fedoraproject.org/wiki/Licensing:FAQ?rd=Licensing/FAQ#What_about_the_RSA_license_on_their_MD5_implementation.3F_Isn.27t_that_GPL-incompatible.3F),
this means that md5.c and md5.h, which were derived from the original
RFC 1321 reference code (http://www.faqs.org/rfcs/rfc1321.html), can
be used without the RSA license.
- In the context of libjpeg-turbo, RSA's MD5 code was used only in the
build/test system. It was not part of the libjpeg-turbo binary
distribution, and thus the only "material mentioning or referencing"
the MD5 code was the libjpeg-turbo source code, which-- by virtue of
including RSA's original copyright headers-- properly attributed the
code as required under the RSA license.
However, in light of the open source community's tendency to have
knee-jerk reactions to stuff like this, it would've been necessary to
include the above explanation in our source tree in order to head off
potential FUD, and a simple fix is always better than a complex
explanation.
This commit also assigns the 3-clause BSD license to my modifications of
the MD5 code. This license is the same one used by md5cmp and other
parts of the build system.
DRC [Fri, 23 Mar 2018 20:31:20 +0000 (15:31 -0500)]
AppVeyor: Use 1.5.x buildscripts branch
The buildscripts repository is maintained such that its branch structure
matches that of the libjpeg-turbo repository, so we need to clone that
repository using the same branch name as the base branch of the build.
This allows us to continue using the autotools-based official build
scripts that were used for 1.5.x
DRC [Fri, 23 Mar 2018 18:57:59 +0000 (13:57 -0500)]
Fix continuous integration of pull requests
- Travis doesn't set the $encrypted_* variables for PRs, so disable GPG
signing when building a PR (artifacts aren't deployed for PRs anyhow,
and even if they were, I wouldn't want them to be signed, as they may
contain unvetted code.)
- Take advantage of the new -d option in buildljt, which allows for
building from an existing Git clone directory. This eliminates the need
to rename and restore .git/shallow, allows the official build scripts to
work properly when building PRs, and prevents 'git clone' being invoked
twice in CI builds.
DRC [Sat, 17 Mar 2018 05:06:10 +0000 (00:06 -0500)]
Make all get/putenv() calls compile-time optional
* Modify the SIMD dispatchers so they guard their usage of getenv() with
the existing NO_GETENV preprocessor definition.
* Introduce a new NO_PUTENV preprocessor definition to guard the
usage of putenv() in the TurboJPEG API library.
This at least puts Windows Store compatibility within the realm of
possibility, although further steps are required.
mayeut [Sat, 24 Feb 2018 13:50:56 +0000 (14:50 +0100)]
Fix Win64 ABI conformance when using xmm8-xmm11
Referring to https://docs.microsoft.com/en-US/cpp/build/stack-usage:
"All memory beyond the current address of RSP is considered volatile:
The OS, or a debugger, may overwrite this memory during a user debug
session, or an interrupt handler. Thus, RSP must always be set before
attempting to read or write values to a stack frame."
Basically, if-- under extremely rare circumstances-- a context swap were
to occur between saving the values of xmm8-xmm11 and setting the new
value of rsp, the O/S might not preserve that area of the stack. In
general, libjpeg-turbo should not be using xmm8-xmm11 before or after
the call to jsimd_huff_encode_one_block_sse2(), so this is probably a
non-issue, but it's still a good idea to fix it.
DRC [Fri, 15 Dec 2017 22:45:25 +0000 (16:45 -0600)]
BUILDING.md: Adjust Android recipes for NDK r16+
NDK r16b moved some things around, so modify the Android build recipes
to take that into account while preserving compatibility with previous
NDK releases.
NOTE: the GCC 4.9 NDK toolchain is deprecated, so we will need to
develop new Android build recipes for libjpeg-turbo 1.6 that use the
Clang toolchain.
DRC [Tue, 5 Dec 2017 21:27:34 +0000 (15:27 -0600)]
Fix lib state when skipping to end of 1-scan image
If jpeg_skip_scanlines() is used to skip to the end of a single-scan
image, then we need to change the library state such that subsequent
calls to jpeg_consume_input() will return JPEG_REACHED_EOI rather than
JPEG_SUSPENDED. (NOTE: not necessary for multi-scan images, since the
scans are processed prior to any call to jpeg_skip_scanlines().)
Unless I miss my guess, using jpeg_skip_scanlines() in this manner
will prevent any markers at the end of the JPEG image from being
read, but I don't think there is any way around that without actually
reading the data, which would defeat the purpose of
jpeg_skip_scanlines().
DRC [Wed, 29 Nov 2017 20:23:31 +0000 (14:23 -0600)]
Travis: Work around xcode7.3 image bug
Refer to travis-ci/travis-ci#8552. This was supposed to be fixed on
November 15, then on November 28. Travis blew through both deadlines,
so I have no confidence that the issue will be fixed as promised in a
timely manner. Adding 'brew update' to .travis.yml slows the OS X
build, but there is no choice at the moment.
DRC [Thu, 16 Nov 2017 01:39:45 +0000 (19:39 -0600)]
Uniquify tjbenchtest log file names based on args
+ clean up log files when 'make testclean' is invoked
+ fix 'tjbenchtest -yuv -alloc'
+ fix tjexampletest so that it creates images under /tmp
+ clean up tjexampletest
DRC [Tue, 14 Nov 2017 21:30:06 +0000 (15:30 -0600)]
tjbench.exe: Fix decompression access violation
The program crashed when a JPEG image was passed on the command line,
because we were mixing our metaphors vis-a-vis malloc()/free() and
tjAlloc()/tjFree() (malloc()/free() uses the tjbench.exe heap,
whereas tjAlloc()/tjFree() uses the turbojpeg.dll heap.)
DRC [Tue, 14 Nov 2017 03:01:53 +0000 (21:01 -0600)]
Further partial image decompression fixes
- Referring to 073b0e88a192adebbb479ee2456beb089d8b5de7 and #185, the
reason why BMP and RLE didn't (and won't) work with partial image
decompression is that the output engines for both formats maintain a
whole-image buffer, which is used to reverse the order of scanlines.
However, it was straightforward to add -crop support for GIF and
Targa, which is useful for testing partial image decompression along
with color quantization.
- Such testing reproduced a bug reported by Mozilla (refer to PR #182)
whereby jpeg_skip_scanlines() would segfault if color quantization was
enabled. To fix this issue, read_and_discard_scanlines() now sets up
a dummy quantize function in the same manner that it sets up a dummy
color conversion function.
DRC [Thu, 9 Nov 2017 03:01:57 +0000 (21:01 -0600)]
djpeg -crop: Exit gracefully with non-PPM formats
... and document that only PPM/PGM output images are supported with the
-crop option for the moment.
I investigated the possibility of supporting -crop with -bmp, but even
after resetting the buffer dimensions, I still kept getting virtual
array access errors. It seems that doing this the "right way" would
require creating a re-initialization function for each image format's
destination manager. I'm disinclined to do that right now, given that
this feature was Google's baby (developed as a prerequisite for
including libjpeg-turbo in Android), and the -crop option in djpeg is
intended only as an example of how to use the partial image
decompression API. Real-world applications would need to handle this
in their own destination managers.
It would probably be possible to make this work with Targa by employing
a similar hack to the one we used with PPM, but Targa isn't popular
enough to bother.
The version of RPM on RHEL 5 and older platforms defines _libdir
as %{_exec_prefix}/%{_lib}, so defining _lib in the spec file redefined
_libdir. However, newer versions of RPM (probably >= 4.6, since that
was the version that introduced the ISA macros) define _libdir as either
%{_prefix}/lib or %{_prefix}/lib64. Thus, we need to explicitly
override _libdir in our spec file.
It is necessary for the C code to be aware of the machine's endianness,
which is why the TurboJPEG Java wrapper sets a different pixel format
for integer BufferedImages depending on ByteOrder.nativeOrder().
However, it isn't necessary to handle endianness in pure Java code such
as TJUnitTest (d'oh!) This was a product of porting the C version of
TJUnitTest too literally, and of insufficient testing (historically,
the big endian systems I had available for testing didn't have Java.)
DRC [Mon, 14 Aug 2017 15:54:27 +0000 (10:54 -0500)]
Java: Fix NullPointerException in YUVImage
planes == null is a valid argument to setBuf() if alloc == true, so we
need to make sure that planes is non-null before validating its length.
We also need to allocate one dimension of the planes array if it's null.
DRC [Wed, 28 Jun 2017 19:47:45 +0000 (14:47 -0500)]
TurboJPEG: Document xform issue w/ big marker data
If the source image for a transform operation has a lot of EXIF or ICC
data embedded in it, then it may cause the output image size to exceed
the worst-case size returned by tjBufSize() (because tjTransform()
transfers all markers to the output image.) This is only a problem if
TJFLAG_NOREALLOC is passed to the function. Since the TurboJPEG C API
doesn't require the destination image size to be set in this case, it
makes the documented assumption that the calling program has allocated
the destination buffer to exactly the size returned by tjBufSize().
Changing this assumption would change the API behavior and necessitate
a new function name (tjTransform2().) At the moment, it's easier to
just document this as a known issue, since it's easy to work around in
the C API.
The Java API is unfortunately a different story, since it must always
use TJFLAG_NOREALLOC (because, when using the TurboJPEG Java API, all
buffers are allocated on the Java heap, and thus they can't be
reallocated by the C code.) There is no easy way to work around this
without changing the C API as discussed above, because if the source
image contains large amounts of marker data, it's virtually impossible
to determine how big the output image will be.
DRC [Tue, 27 Jun 2017 19:22:39 +0000 (14:22 -0500)]
TJBench: Improve consistency of results
Given that libjpeg-turbo can often process hundreds of megapixels/second
on modern hardware, the default of one warmup iteration was essentially
meaningless. Furthermore, the -warmup option was a bit clunky, since
it required some foreknowledge of how fast the benchmarks were going to
execute.
This commit introduces a 1-second warmup interval for each benchmark by
default, and the -warmup option has been retasked to control the length
of that interval.
DRC [Tue, 27 Jun 2017 01:48:02 +0000 (20:48 -0500)]
TJBench: Fix errors when decomp. files w/ ICC data
Embedded ICC profiles can cause the size of a JPEG file to exceed the
size returned by tjBufSize() (which is really meant to be used for
compression anyhow, not for decompression), and this was causing a
segfault (C) or an ArrayIndexOutOfBoundsException (Java) when
decompressing such files with TJBench. This commit modifies the
benchmark such that, when tiled decompression is disabled, it re-uses
the source buffer as the primary JPEG buffer.
DRC [Mon, 8 May 2017 13:15:11 +0000 (08:15 -0500)]
Travis: Fix OS X build
The Travis xcode7.3 image now apparently includes GnuPG 1.4.x by
default, so use it instead of installing GnuPG 2. Using GnuPG 2.1.x,
the default version in Homebrew as of this writing, is problematic for
this reason:
https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase
This code was submitted in the initial ARM NEON patches
(https://sourceforge.net/p/libjpeg-turbo/patches/7/) by Siarhei while he
was still a Nokia employee.
DRC [Sat, 18 Mar 2017 21:15:14 +0000 (16:15 -0500)]
Honor max_memory_to_use/JPEGMEM/-maxmemory
This re-introduces a feature of the obsolete system-specific libjpeg
memory managers-- namely the ability to limit the amount of main memory
used by the library during decompression or multi-pass compression.
This is mainly beneficial for two reasons:
- Works around a 2 GB limit in libFuzzer
- Allows security-sensitive applications to set a memory limit for the
JPEG decoder so as to work around the progressive JPEG exploit
(LJT-01-004) described here:
http://www.libjpeg-turbo.org/pmwiki/uploads/About/TwoIssueswiththeJPEGStandard.pdf
This commit also removes obsolete documentation regarding the MS-DOS
memory manager (which itself was removed long ago) and changes the
documentation of the -maxmemory switch and JPEGMEM environment variable
to reflect the fact that backing stores are never used in libjpeg-turbo.
DRC [Sat, 18 Mar 2017 18:24:50 +0000 (13:24 -0500)]
AppVeyor: Fix CI build
Something changed in the CI build environment, and our previous trick of
setting the Git URL to file://c:/projects/libjpeg-turbo no longer works.
Using cygpath to translate the Windows path to a MinGW-friendly format
is a better solution anyhow.
DRC [Sat, 18 Mar 2017 17:56:36 +0000 (12:56 -0500)]
TurboJPEG: Fix potential memory leaks
Referring to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=746,
it seems that the values of local buffer pointers in TurboJPEG API
functions aren't always preserved if longjmp() returns control to a
point prior to the allocation of the local buffers. This is known to
be an issue with GCC 4.x and clang with -O1 and higher optimization
levels but not with GCC 5.x and later. It is unknown why GCC 5.x and
6.x do not suffer from the issue, but possibly the local buffer pointers
are not allocated on the stack when using those more recent compilers.
In any case, this commit modifies the TurboJPEG API library code such
that the jump buffer is always updated after any local buffer pointers
are allocated but before any subsequent libjpeg API functions are
called.
DRC [Fri, 20 Jan 2017 00:51:41 +0000 (18:51 -0600)]
Always tweak EXIF w/h tags w/ lossless transforms
... even if using libjpeg v6b emulation. Previously
adjust_exif_parameters() was only called with libjpeg v7/v8 emulation,
but due to a bug (which this commit also fixes), it only worked properly
with libjpeg v8 emulation.
DRC [Thu, 19 Jan 2017 23:50:59 +0000 (17:50 -0600)]
Fix error w/ lossless crop & libjpeg v7 emulation
The JPEG_LIB_VERSION #ifdef in jtransform_adjust_parameters() was
incorrect, which caused a "Bogus virtual array access" error when
attempting to use the lossless crop feature.
projects / libjpeg-turbo / log