DRC [Fri, 18 Oct 2019 00:59:01 +0000 (19:59 -0500)]
x86 SIMD: Consistify capitalization of NASM types
byte, word, dword, qword, oword, and yword are all assembler keywords,
so it makes sense to use lowercase for these so as not to mistake them
for macros or constants.
DRC [Thu, 15 Aug 2019 18:24:25 +0000 (13:24 -0500)]
tjDecodeYUV*: Fix err if TJ inst used for prog dec
If the TurboJPEG instance passed to tjDecodeYUV[Planes]() was previously
used to decompress a progressive JPEG image, then we need to disable the
progressive decompression parameters in the underlying libjpeg instance
before calling jinit_master_decompress().
This commit also modifies the build system so that the "tjtest" target
will test for this issue, and it corrects a previous oversight in the
build system whereby tjbenchtest did not test progressive
compression/decompression unless WITH_JAVA was true.
DRC [Tue, 13 Aug 2019 21:03:48 +0000 (16:03 -0500)]
Travis: Enable additional sanitizer tests
- Re-purpose the non-SIMD test to test with MSan as well.
- Re-purpose the ASan test to test with UBSan as well.
- Use the default Travis build environment rather than specifying Ubuntu
14.04. I think I added 'dist: trusty' back when 14.04 was newer than
the default, but now it's older than the default.
- Enable verbose output for any unit tests that fail (so we can see the
sanitizer output.)
Updating Homebrew wasn't necessary when the CI build first started using
the xcode8.3 image (see c8e52741fdb623698a27a4c55cf63db93c95aa14), but
apparently it now is.
Prevent several integer overflow issues and subsequent segfaults that
occurred when attempting to compress or decompress gigapixel images with
the TurboJPEG API:
- Modify tjBufSize(), tjBufSizeYUV2(), and tjPlaneSizeYUV() to avoid
integer overflow when computing the return values and to return an
error if such an overflow is unavoidable.
- Modify tjunittest to validate the above.
- Modify tjCompress2(), tjEncodeYUVPlanes(), tjDecompress2(), and
tjDecodeYUVPlanes() to avoid integer overflow when computing the row
pointers in the 64-bit TurboJPEG C API.
- Modify TJBench (both C and Java versions) to avoid overflowing the
size argument to malloc()/new and to fail gracefully if such an
overflow is unavoidable.
In general, this allows gigapixel images to be accommodated by the
64-bit TurboJPEG C API when using automatic JPEG buffer (re)allocation.
Such images cannot currently be accommodated without automatic JPEG
buffer (re)allocation, due to the fact that tjAlloc() accepts a 32-bit
integer argument (oops.) Such images cannot be accommodated in the
TurboJPEG Java API due to the fact that Java always uses a signed 32-bit
integer as an array index.
Jonathan Wright [Thu, 9 May 2019 12:46:53 +0000 (13:46 +0100)]
Use bias pattern for 4:4:0 (h1v2) fancy upsampling
This commit modifies h1v2_fancy_upsample() so that it uses an ordered
dither pattern, similar to that of h2v1_fancy_upsample(), rounding up or
down the result for alternate pixels rather than always rounding down.
This ensures that the decompression error pattern for a 4:4:0 JPEG image
will be similar to the rotated decompression error pattern for a 4:2:2
JPEG image. Thus, the final result will be similar regardless of
whether a 4:2:2 JPEG image is rotated or transposed before or after
decompression.
DRC [Thu, 9 May 2019 14:11:09 +0000 (09:11 -0500)]
Travis: Use xcode8.3 image
xcode7.3 is based on El Capitan, which is EOL, and Homebrew no longer
provides El Cap bottles (pre-compiled binaries.) Thus, Homebrew was
trying to build GCC 5, YASM, and the other packages we need from source,
which caused the Mac CI builds to time out. I tried goading Homebrew
into installing GCC 5.5.0_2 and YASM 1.3.0_1, which still have El Cap
bottles available, by using the URLs of those specific versions of the
formulae (from the Homebrew GitHub repository) as package names. This
failed, however, because 'brew bundle' converted the URLs to all
lowercase. I then tried explicitly installing the old formulae by using
'brew install' with the aforementioned URLs (bypassing the Travis
Homebrew addon), but Homebrew still tried to build all of the
dependencies from source.
Upgrading to Xcode 8.3.x necessitated regression testing the performance
on iOS, but that proved less of a pain than figuring out how to install
all of the old Homebrew bottles we needed. Also, this future-proofs the
CI builds against the inevitable discontinuation of the xcode7.3 image.
Note that Xcode 8.3.x improves iOS 64-bit decompression performance
significantly relative to Xcode 7.2.x or 7.3.x. iOS 32-bit performance
unfortunately regresses by as much as 5%, but it can't be helped (32-bit
iOS apps are no longer supported on iOS 11+ anyhow, and the next major
release of libjpeg-turbo will remove support for them as well.)
Chris Blume [Tue, 16 Apr 2019 00:55:47 +0000 (17:55 -0700)]
x86 SIMD: Check for CPUID leaf 07H before using
According to Intel's manual [1], "If a value entered for CPUID.EAX is
higher than the maximum input value for basic or extended function for
that processor then the data for the highest basic information leaf is
returned."
Right now, libjpeg-turbo doesn't first check that leaf 07H is supported
before attempting to use it, so the ostensible AVX2 bit (Bit 05) of the
CPUID result might actually be Bit 05 from a lower leaf. That bit might
be set, even if the CPU doesn't support AVX2.
This commit modifies the x86 and x86-64 SIMD feature detection code so
that it first checks whether CPUID leaf 07H is supported before
attempting to use it to check for AVX2 instruction support.
DRC:
This commit should fix
https://bugzilla.mozilla.org/show_bug.cgi?id=1520760
However, I have not personally been able to reproduce that issue,
despite using a Nehalem (pre-AVX2) CPU on which the maximum CPUID leaf
has been limited via a BIOS setting.
jdhuff.c: Silence UBSan signed int overflow err #2
Same as d3a3a73f64041c6a6905faf6f9f9832e735fd880 but in the fast decode
path. It was necessary to use a different-sized test image in order to
trigger the error in this location.
Android: Fix "using JNI after critical get" errors
5ea77d8b771bf7b0be7acc7fb70c255f8c9c8f77 was insufficient to fix all of
these. In particular, we need to always release the primitive arrays
before throwing an exception, because throwing an exception qualifies as
"using JNI."
Some pathological test images have been created that can cause s to
overflow or underflow the signed int data type during decompression.
This is technically undefined behavior according to the C spec, although
every modern implementation I'm aware of will treat the signed int as a
2's complement unsigned int, thus causing the value to wrap around to
INT_MIN if it exceeds INT_MAX. This commit simply makes that behavior
explicit in order to shut up UBSan. At least when building for x86-64
or i386 using Clang or GCC, this commit does not change the
compiler-generated assembly code at all.
The code that triggered this error has existed in the libjpeg code base
for at least 20 years (and probably much longer), so the fact that it
hasn't produced a user-visible problem in all of that time strongly
suggests that UBSan is being overly pedantic here. But if someone can
cough up a platform that doesn't wrap around to INT_MIN when 1 is added
to INT_MAX, then I'll happily change my opinion.
DRC [Thu, 14 Feb 2019 16:01:16 +0000 (10:01 -0600)]
appveyor.yml: Cache NASM binary package
... since www.nasm.us seems to be down frequently. This doesn't help us
at the moment, but hopefully once the site is back up this will prevent
future build failures.
When simd/arm/jsimd.c is compiled with __ARM_NEON__ defined (which will
be the case if -mfpu=neon is passed to the compiler), the
parse_proc_cpuinfo() and check_feature() functions and the bufsize
variable are unused and thus need to be #ifdef'ed out in order to avoid
compiler warnings. Note that the bufsize variable was already #ifdef'ed
out on Linux but not on Android due to lack of parentheses (&& takes
precedence over ||.)
%{_libdir}/pkgconfig is a directory and should thus be prefixed by
%{dir} (oops.) This issue caused the debuginfo build under RHEL 8
(which is apparently now enabled by default-- regardless of whether the
RPM actually contains debug info, but that's another matter) to fail
with:
DRC [Wed, 13 Feb 2019 21:20:34 +0000 (15:20 -0600)]
rpm.spec.in: Fix doc packaging issues w/ RHEL 7+
On RHEL 7 and later (not sure exactly whether this is a product of the
newer RPM release or something distro-specific), macros are lazily
expanded, so we need to set _docdir using %global (which expands at
definition time) and prior to _prefix and _datarootdir (which affect
_defaultdocdir.) Otherwise, _docdir is set to a subdirectory of
/opt/libjpeg-turbo/share/doc or /opt/libjpeg-turbo/doc. The former
(which happens on RHEL 7) leads to incorrect documentation packaging
(the docs should be packaged under /usr/share/doc per Red Hat
standards), and the latter (which happens on RHEL 8) leads to an RPM
build error.
DRC [Tue, 12 Feb 2019 21:37:21 +0000 (15:37 -0600)]
rpm.spec.in: Update deprecated [Build]Prereq tags
AFAICT, Requires and BuildRequires subsumed the functionality of Prereq
and BuildPrereq in RPM 4.0, and none of the platforms we support with
libjpeg-turbo 2.0.x has RPM < 4.4.
DRC [Fri, 25 Jan 2019 22:46:02 +0000 (16:46 -0600)]
Build: Fix install of static build w/ VStudio IDE
Unfortunately, this hack is necessary because:
- install(TARGETS, ...) doesn't support the RENAME option.
- We can't modify OUTPUT_NAME for the "-static" targets without breaking
the regression tests.
- ${CMAKE_CFG_INTDIR} doesn't seem to work properly in an install()
command.
Including the license templates was confusing to some, since it made
it appear as if the copyright year and author were unspecified for the
libjpeg-turbo source. Thus, rather than include the zlib License
template, link to that template on opensource.org. For the Modified BSD
License, include a roll-up of copyright years and authors, since the
terms of that license require the text of it to be included in product
documentation for binary distributions without accompanying source code.
DRC [Tue, 22 Jan 2019 18:11:03 +0000 (12:11 -0600)]
BUILDING.md: Update/simplify Android build instr.
Use the android.toolchain.cmake toolchain file in the NDK (v13b or
later), since this toolchain file generally takes care of setting the
approprate compiler flags and dealing with the differences between
GCC and Clang. Our custom Android build procedure did not work with
Clang-based NDK toolchains, which meant that it could not be made to
work with NDK v18b or later.
DRC [Mon, 21 Jan 2019 19:57:55 +0000 (13:57 -0600)]
TurboJPEG: Decompress 4:4:4 JPEGs with unusual SFs
Normally, 4:4:4 JPEGs have horizontal x vertical luminance & chrominance
sampling factors of 1x1. However, it is technically legal to create
4:4:4 JPEGs with sampling factors of 2x1, 1x2, 3x1, or 1x3, since the
sums of the products of those sampling factors are still <= 10. The
libjpeg API correctly decodes such images, so the TurboJPEG API should
as well.
DRC [Wed, 2 Jan 2019 02:32:40 +0000 (20:32 -0600)]
wrbmp.c: Don't allow quantization w/ non-RGB CS
If cinfo->quantize_colors == 1, then jpeg_calc_output_dimensions() will
set cinfo->output_components to 1, and if cinfo->out_color_space is not
RGB (or extended RGB), hilarity will ensue.
DRC [Tue, 1 Jan 2019 20:26:48 +0000 (14:26 -0600)]
Build: Fix regr. that nuked RPATH in Mac/iOS build
Caused by 950580eb0c020598a4c6c8aa46c86e31062e1ddc. Since the code that
sets CMAKE_INSTALL_RPATH now depends on ENABLE_SHARED, that code needed
to be moved to after the point at which ENABLE_SHARED is defined.
DRC [Wed, 21 Nov 2018 04:52:36 +0000 (22:52 -0600)]
Travis: Fetch GPG key from libjpeg-turbo.org
I give up on the public keyserver. It inexplicably just fails
sometimes. I was trying to use it out of an abundance of caution
(<cough> paranoia <cough>), but it seems like most open source projects
just serve up their public keys from their project web sites. The
private and public pre-release keys are still stored on separate sites,
the private key is still strongly encrypted by Travis, and we use a
separate key for pre-releases anyhow, so even if it's compromised, we
can quickly and easily deploy a new one.
DRC [Fri, 2 Nov 2018 16:55:29 +0000 (11:55 -0500)]
Travis: Don't check key server SSL certificate
... when downloading the RPM signing key. Apparently the key server
URL sometimes redirects to an https URL, which may explain why fetching
the RPM signing keys failed frequently when we used to run wget inside
of the CentOS 5 Docker container.
I have a hunch that this is related to the CentOS 5 Docker container, so
this commit causes Travis to download the RPM signing key outside of
the container and share it with the container.
DRC [Fri, 26 Oct 2018 13:55:22 +0000 (08:55 -0500)]
Android: Fix "using JNI after critical get" error
We shouldn't be making JNI calls between GetPrimitiveArrayCritical() and
ReleasePrimitiveArrayCritical(). Apparently Android is stricter about
this than desktop Java.
Apparently Windows 7 without SP1 has O/S support for XSAVE but not for
YMM registers, and this exposed a bug in our usage of xgetbv. The test
instruction will set ZF only if none of the bits match between the two
operarands, so in effect, we were enabling AVX2 instructions if the O/S
supported XSAVE and the CPU supported AVX2 but the O/S only supported
XMM registers. This bug was not exposed on, for instance, Windows XP or
RHEL 5 because those O/S's do not support XSAVE.
- CMake 3.10.x or later must be used with JDK 11, or an error
("regex not supported") will occur when CMake tries to parse the Java
version number.
- The JDK is no longer available at java.com.
Enable DSPr2 SIMD extensions if CPU type is mipsel
The DSPr2 extensions have been verified to work with little endian MIPS.
Whether or not CMAKE_SYSTEM_PROCESSOR is set to "mips" or "mipsel" in a
little endian MIPS environment seems to be inconsistent, but our build
system needs to handle both cases.
(for instance, when passing -msoft-float to the compiler)
The instructions used by jsimd_quantize_float_dspr2() and
jsimd_convsamp_float_dspr2() don't work with the soft float ABI, so
disable those functions when soft float is enabled.
@rpath is only supported with 10.5 and later deployment targets.
libjpeg-turbo hasn't supported 10.4 "Tiger" since prior to 1.4, but I
still sometimes use the 10.4 SDK to test PowerPC code in a Snow Leopard
VM.
Fix JPEG spec references per ISO/ITU-T suggestions
- When referring to specific clauses, annexes, tables, and figures, a
"timed reference" (a reference that includes the year) must be used in
order to avoid confusion.
- "CCITT" = "ITU-T"
- Replace ambiguous "JPEG spec" with the specific document number.
... in which one or more of the color indices is out of range for the
number of palette entries.
Fix partly borrowed from jpeg-9c. This commit also adopts Guido's
JERR_PPM_OUTOFRANGE enum value in lieu of our project-specific
JERR_PPM_TOOLARGE enum value.
Normally the value of CMAKE_EXECUTABLE_SUFFIX is clobbered by project().
This allows for specifying an executable suffix of .html with Emscripten
builds, which causes Emscripten to build standalone HTML versions of the
libjpeg-turbo test programs.
The sentence:
"Indeed, one of the original reasons for developing this free software
was to help force convergence on common, interoperable format standards
for JPEG files."
might be seen to imply that JPEG 2000 and JPEG XR are not interoperable
with themselves, although it is certainly the case that those formats
are not interoperable with each other, nor with
ITU T.81 | ISO/IEC 10918. They are also certainly not as common as
ITU T.81 | ISO/IEC 10918, and (as an example) popular web browsers will
not display JPEG 2000 files.
The sentence in question was originally referring to proprietary,
non-standard formats and was meant to provide historical context.
libjpeg was originally released prior to the adoption of JFIF as an
official standard, so it encouraged adoption of JFIF as a de facto
standard by providing, under a business-friendly free software license,
a library for reading and writing images in that format.
DRC [Fri, 29 Jun 2018 17:45:57 +0000 (12:45 -0500)]
Build: Detect whether compiler supports DSPr2
This is basically the same test that was performed in acinclude.m4 in
the old autotools-based build system. It was not ported to the
CMake-based build system because I previously had no way of testing
a non-DSPr2 build environment.
DRC [Sun, 24 Jun 2018 22:31:15 +0000 (17:31 -0500)]
Fix infinite loop in partial image decompression
... caused by using certain specific combinations of
jpeg_skip_scanlines() and jpeg_read_scanlines() calls with progressive,
vertically-subsampled JPEG images.
DRC [Wed, 13 Jun 2018 01:27:00 +0000 (20:27 -0500)]
tjLoadImage(): Fix FPE triggered by malformed BMP
In rdbmp.c, it is necessary to guard against 32-bit overflow/wraparound
when allocating the row buffer, because since BMP files have 32-bit
width and height fields, the value of biWidth can be up to 4294967295.
Specifically, if biWidth is 1073741824 and cinfo->input_components = 4,
then the samplesperrow argument in alloc_sarray() would wrap around to
0, and a division by zero error would occur at line 458 in jmemmgr.c.
If biWidth is set to a higher value, then samplesperrow would wrap
around to a small number, which would likely cause a buffer overflow
(this has not been tested or verified.)
DRC [Tue, 12 Jun 2018 23:49:37 +0000 (18:49 -0500)]
TurboJPEG: Handle JERR_BMP*,JERR_PPM* error codes
... in tjLoadImage()/tjSaveImage(). These error codes require an add-on
message table, and if it isn't initialized, then format_message()
produces "Bogus message code XXXX" instead.
projects / libjpeg-turbo / log