mayeut [Sat, 24 Feb 2018 13:50:56 +0000 (14:50 +0100)]
Fix Win64 ABI conformance when using xmm8-xmm11
Referring to https://docs.microsoft.com/en-US/cpp/build/stack-usage:
"All memory beyond the current address of RSP is considered volatile:
The OS, or a debugger, may overwrite this memory during a user debug
session, or an interrupt handler. Thus, RSP must always be set before
attempting to read or write values to a stack frame."
Basically, if-- under extremely rare circumstances-- a context swap were
to occur between saving the values of xmm8-xmm11 and setting the new
value of rsp, the O/S might not preserve that area of the stack. In
general, libjpeg-turbo should not be using xmm8-xmm11 before or after
the call to jsimd_huff_encode_one_block_sse2(), so this is probably a
non-issue, but it's still a good idea to fix it.
DRC [Fri, 15 Dec 2017 22:45:25 +0000 (16:45 -0600)]
BUILDING.md: Adjust Android recipes for NDK r16+
NDK r16b moved some things around, so modify the Android build recipes
to take that into account while preserving compatibility with previous
NDK releases.
NOTE: the GCC 4.9 NDK toolchain is deprecated, so we will need to
develop new Android build recipes for libjpeg-turbo 1.6 that use the
Clang toolchain.
DRC [Tue, 5 Dec 2017 21:27:34 +0000 (15:27 -0600)]
Fix lib state when skipping to end of 1-scan image
If jpeg_skip_scanlines() is used to skip to the end of a single-scan
image, then we need to change the library state such that subsequent
calls to jpeg_consume_input() will return JPEG_REACHED_EOI rather than
JPEG_SUSPENDED. (NOTE: not necessary for multi-scan images, since the
scans are processed prior to any call to jpeg_skip_scanlines().)
Unless I miss my guess, using jpeg_skip_scanlines() in this manner
will prevent any markers at the end of the JPEG image from being
read, but I don't think there is any way around that without actually
reading the data, which would defeat the purpose of
jpeg_skip_scanlines().
DRC [Wed, 29 Nov 2017 20:23:31 +0000 (14:23 -0600)]
Travis: Work around xcode7.3 image bug
Refer to travis-ci/travis-ci#8552. This was supposed to be fixed on
November 15, then on November 28. Travis blew through both deadlines,
so I have no confidence that the issue will be fixed as promised in a
timely manner. Adding 'brew update' to .travis.yml slows the OS X
build, but there is no choice at the moment.
DRC [Sat, 18 Nov 2017 17:33:05 +0000 (11:33 -0600)]
tjLoadImage(): Don't convert RGB to grayscale
Loading RGB image files into a grayscale buffer isn't a particularly
useful feature, given that libjpeg-turbo can perform this conversion
much more optimally (with SIMD acceleration on some platforms) during
the compression process. Also, the RGB2GRAY() macro was not producing
deterministic cross-platform results because of variations in the
round-off behavior of various floating point implementations, so
`tjunittest -bmp` was failing in i386 builds.
DRC [Fri, 17 Nov 2017 02:43:12 +0000 (20:43 -0600)]
TurboJPEG: Add alpha offset array/method
Also, set the red/green/blue offsets for TJPF_GRAY to -1 rather than 0.
It was undefined behavior for an application to use those arrays/methods
with TJPF_GRAY anyhow, and this makes it easier for applications to
programmatically detect whether a given pixel format has red, green, and
blue components.
DRC [Fri, 17 Nov 2017 00:09:07 +0000 (18:09 -0600)]
TurboJPEG C API: Add BMP/PPM load/save functions
The main justification for this is to provide new libjpeg-turbo users
with a quick & easy way of developing a complete JPEG
compression/decompression program without requiring them to build
libjpeg-turbo from source (which was necessary in order to use the
project-private bmp API) or to use external libraries. These new
functions build upon significant enhancements to rdbmp.c, wrbmp.c,
rdppm.c, and wrppm.c which allow those engines to convert directly
between the native pixel format of the file and a pixel format
("colorspace" in libjpeg parlance) specified by the calling program.
rdbmp.c and wrbmp.c have also been modified such that the calling
program can choose to read or write image rows in the native (bottom-up)
order of the file format, thus eliminating the need to use an inversion
array. tjLoadImage() and tjSaveImage() leverage these new underlying
features in order to significantly improve upon the performance of the
old bmp API.
Because these new functions cannot work without the libjpeg-turbo
colorspace extensions, the libjpeg-compatible code in turbojpeg.c has
been removed. That code was only there to serve as an example of how
to use the TurboJPEG API on top of libjpeg, but more specific, buildable
examples now exist in the https://github.com/libjpeg-turbo/ijg
repository.
DRC [Wed, 15 Nov 2017 15:19:27 +0000 (09:19 -0600)]
Build: Fix 'tjtest' target on Windows
tjbenchtest and its Java derivatives are useful for rooting out hidden
problems with the more esoteric TJBench and TurboJPEG features. For
instance, on Windows, running tjbenchtest uncovered 5fce2e942136cb70e5a30ff15a2d58b07947aa84.
This commit also causes tjbenchtest and tjbenchtest.java to append -yuv
and -alloc to their log file names, depending on the arguments passed,
and it causes the build system to clean up those log files when the
'testclean' target is built.
DRC [Thu, 16 Nov 2017 01:39:45 +0000 (19:39 -0600)]
Uniquify tjbenchtest log file names based on args
+ clean up log files when 'make testclean' is invoked
+ fix 'tjbenchtest -yuv -alloc'
+ fix tjexampletest so that it creates images under /tmp
+ clean up tjexampletest
DRC [Tue, 14 Nov 2017 21:30:06 +0000 (15:30 -0600)]
tjbench.exe: Fix decompression access violation
The program crashed when a JPEG image was passed on the command line,
because we were mixing our metaphors vis-a-vis malloc()/free() and
tjAlloc()/tjFree() (malloc()/free() uses the tjbench.exe heap,
whereas tjAlloc()/tjFree() uses the turbojpeg.dll heap.)
DRC [Tue, 14 Nov 2017 03:01:53 +0000 (21:01 -0600)]
Further partial image decompression fixes
- Referring to 073b0e88a192adebbb479ee2456beb089d8b5de7 and #185, the
reason why BMP and RLE didn't (and won't) work with partial image
decompression is that the output engines for both formats maintain a
whole-image buffer, which is used to reverse the order of scanlines.
However, it was straightforward to add -crop support for GIF and
Targa, which is useful for testing partial image decompression along
with color quantization.
- Such testing reproduced a bug reported by Mozilla (refer to PR #182)
whereby jpeg_skip_scanlines() would segfault if color quantization was
enabled. To fix this issue, read_and_discard_scanlines() now sets up
a dummy quantize function in the same manner that it sets up a dummy
color conversion function.
DRC [Thu, 9 Nov 2017 03:01:57 +0000 (21:01 -0600)]
djpeg -crop: Exit gracefully with non-PPM formats
... and document that only PPM/PGM output images are supported with the
-crop option for the moment.
I investigated the possibility of supporting -crop with -bmp, but even
after resetting the buffer dimensions, I still kept getting virtual
array access errors. It seems that doing this the "right way" would
require creating a re-initialization function for each image format's
destination manager. I'm disinclined to do that right now, given that
this feature was Google's baby (developed as a prerequisite for
including libjpeg-turbo in Android), and the -crop option in djpeg is
intended only as an example of how to use the partial image
decompression API. Real-world applications would need to handle this
in their own destination managers.
It would probably be possible to make this work with Targa by employing
a similar hack to the one we used with PPM, but Targa isn't popular
enough to bother.
Setting _libdir to CMAKE_INSTALL_FULL_LIBDIR only works when doing an
in-tree RPM build. SRPMs are architecture-agnostic, so the spec needs
to compute_libdir at the time the SRPM is rebuilt, not at the time it
is generated.
This is a regression introduced when implementing the new CMake-based
cross-platform build system.
The version of RPM on RHEL 5 and older platforms defines _libdir
as %{_exec_prefix}/%{_lib}, so defining _lib in the spec file redefined
_libdir. However, newer versions of RPM (probably >= 4.6, since that
was the version that introduced the ISA macros) define _libdir as either
%{_prefix}/lib or %{_prefix}/lib64. Thus, we need to explicitly
override _libdir in our spec file.
It is necessary for the C code to be aware of the machine's endianness,
which is why the TurboJPEG Java wrapper sets a different pixel format
for integer BufferedImages depending on ByteOrder.nativeOrder().
However, it isn't necessary to handle endianness in pure Java code such
as TJUnitTest (d'oh!) This was a product of porting the C version of
TJUnitTest too literally, and of insufficient testing (historically,
the big endian systems I had available for testing didn't have Java.)
DRC [Mon, 14 Aug 2017 15:54:27 +0000 (10:54 -0500)]
Java: Fix NullPointerException in YUVImage
planes == null is a valid argument to setBuf() if alloc == true, so we
need to make sure that planes is non-null before validating its length.
We also need to allocate one dimension of the planes array if it's null.
DRC [Thu, 29 Jun 2017 21:49:09 +0000 (16:49 -0500)]
TJBench: Recover from non-fatal errors if possible
Previously, -stoponwarning only had an effect on the underlying
TurboJPEG C functions, but TJBench still aborted if a non-fatal error
occurred. This commit modifies the C version of TJBench such that it
always recovers from a non-fatal error unless -stoponwarning is
specified. Furthermore, the benchmark stores the details of the last
non-fatal error and does not print any subsequent non-fatal error
messages unless they differ from the last one.
Due to limitations in the Java API (specifically, the fact that it
cannot communicate errors, fatal or otherwise, to the calling program
without throwing a TJException), it was only possible to make
decompression operations fully recoverable within TJBench. With other
operations, -stoponwarning still has an effect on the underlying C
library but has no effect at the Java level.
The Java API documentation has been amended to reflect that only certain
methods are truly recoverable, regardless of the state of
TJ.FLAG_STOPONWARNING.
Allow progressive entropy coding to be enabled on a
transform-by-transform basis, and implement a new transform option for
disabling the copying of markers.
DRC [Wed, 28 Jun 2017 19:47:45 +0000 (14:47 -0500)]
TurboJPEG: Document xform issue w/ big marker data
If the source image for a transform operation has a lot of EXIF or ICC
data embedded in it, then it may cause the output image size to exceed
the worst-case size returned by tjBufSize() (because tjTransform()
transfers all markers to the output image.) This is only a problem if
TJFLAG_NOREALLOC is passed to the function. Since the TurboJPEG C API
doesn't require the destination image size to be set in this case, it
makes the documented assumption that the calling program has allocated
the destination buffer to exactly the size returned by tjBufSize().
Changing this assumption would change the API behavior and necessitate
a new function name (tjTransform2().) At the moment, it's easier to
just document this as a known issue, since it's easy to work around in
the C API.
The Java API is unfortunately a different story, since it must always
use TJFLAG_NOREALLOC (because, when using the TurboJPEG Java API, all
buffers are allocated on the Java heap, and thus they can't be
reallocated by the C code.) There is no easy way to work around this
without changing the C API as discussed above, because if the source
image contains large amounts of marker data, it's virtually impossible
to determine how big the output image will be.
DRC [Tue, 27 Jun 2017 19:22:39 +0000 (14:22 -0500)]
TJBench: Improve consistency of results
Given that libjpeg-turbo can often process hundreds of megapixels/second
on modern hardware, the default of one warmup iteration was essentially
meaningless. Furthermore, the -warmup option was a bit clunky, since
it required some foreknowledge of how fast the benchmarks were going to
execute.
This commit introduces a 1-second warmup interval for each benchmark by
default, and the -warmup option has been retasked to control the length
of that interval.
DRC [Tue, 27 Jun 2017 15:54:21 +0000 (10:54 -0500)]
TurboJPEG: Improve error handling
- Provide a new C API function and TJException method that allows
calling programs to query the severity of a compression/decompression/
transform error.
- Provide a new flag that instructs the library to immediately stop
compressing/decompressing/transforming if a warning is encountered.
DRC [Fri, 12 May 2017 02:02:29 +0000 (21:02 -0500)]
TurboJPEG: Thread-safe error message retrieval
Introduce a new C API function (tjGetErrorStr2()) that can be used to
retrieve compression/decompression/transform error messages in a
thread-safe (i.e. instance-specific) manner. Retrieving error messages
from global functions is still thread-unsafe.
DRC [Tue, 27 Jun 2017 01:48:02 +0000 (20:48 -0500)]
TJBench: Fix errors when decomp. files w/ ICC data
Embedded ICC profiles can cause the size of a JPEG file to exceed the
size returned by tjBufSize() (which is really meant to be used for
compression anyhow, not for decompression), and this was causing a
segfault (C) or an ArrayIndexOutOfBoundsException (Java) when
decompressing such files with TJBench. This commit modifies the
benchmark such that, when tiled decompression is disabled, it re-uses
the source buffer as the primary JPEG buffer.
DRC [Mon, 8 May 2017 13:15:11 +0000 (08:15 -0500)]
Travis: Fix OS X build
The Travis xcode7.3 image now apparently includes GnuPG 1.4.x by
default, so use it instead of installing GnuPG 2. Using GnuPG 2.1.x,
the default version in Homebrew as of this writing, is problematic for
this reason:
https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase
This code was submitted in the initial ARM NEON patches
(https://sourceforge.net/p/libjpeg-turbo/patches/7/) by Siarhei while he
was still a Nokia employee.
DRC [Sat, 18 Mar 2017 21:15:14 +0000 (16:15 -0500)]
Honor max_memory_to_use/JPEGMEM/-maxmemory
This re-introduces a feature of the obsolete system-specific libjpeg
memory managers-- namely the ability to limit the amount of main memory
used by the library during decompression or multi-pass compression.
This is mainly beneficial for two reasons:
- Works around a 2 GB limit in libFuzzer
- Allows security-sensitive applications to set a memory limit for the
JPEG decoder so as to work around the progressive JPEG exploit
(LJT-01-004) described here:
http://www.libjpeg-turbo.org/pmwiki/uploads/About/TwoIssueswiththeJPEGStandard.pdf
This commit also removes obsolete documentation regarding the MS-DOS
memory manager (which itself was removed long ago) and changes the
documentation of the -maxmemory switch and JPEGMEM environment variable
to reflect the fact that backing stores are never used in libjpeg-turbo.
projects / libjpeg-turbo / log