DRC [Fri, 15 Dec 2017 22:45:25 +0000 (16:45 -0600)]
BUILDING.md: Adjust Android recipes for NDK r16+
NDK r16b moved some things around, so modify the Android build recipes
to take that into account while preserving compatibility with previous
NDK releases.
NOTE: the GCC 4.9 NDK toolchain is deprecated, so we will need to
develop new Android build recipes for libjpeg-turbo 1.6 that use the
Clang toolchain.
DRC [Tue, 5 Dec 2017 21:27:34 +0000 (15:27 -0600)]
Fix lib state when skipping to end of 1-scan image
If jpeg_skip_scanlines() is used to skip to the end of a single-scan
image, then we need to change the library state such that subsequent
calls to jpeg_consume_input() will return JPEG_REACHED_EOI rather than
JPEG_SUSPENDED. (NOTE: not necessary for multi-scan images, since the
scans are processed prior to any call to jpeg_skip_scanlines().)
Unless I miss my guess, using jpeg_skip_scanlines() in this manner
will prevent any markers at the end of the JPEG image from being
read, but I don't think there is any way around that without actually
reading the data, which would defeat the purpose of
jpeg_skip_scanlines().
DRC [Wed, 29 Nov 2017 20:23:31 +0000 (14:23 -0600)]
Travis: Work around xcode7.3 image bug
Refer to travis-ci/travis-ci#8552. This was supposed to be fixed on
November 15, then on November 28. Travis blew through both deadlines,
so I have no confidence that the issue will be fixed as promised in a
timely manner. Adding 'brew update' to .travis.yml slows the OS X
build, but there is no choice at the moment.
DRC [Sat, 18 Nov 2017 17:33:05 +0000 (11:33 -0600)]
tjLoadImage(): Don't convert RGB to grayscale
Loading RGB image files into a grayscale buffer isn't a particularly
useful feature, given that libjpeg-turbo can perform this conversion
much more optimally (with SIMD acceleration on some platforms) during
the compression process. Also, the RGB2GRAY() macro was not producing
deterministic cross-platform results because of variations in the
round-off behavior of various floating point implementations, so
`tjunittest -bmp` was failing in i386 builds.
DRC [Fri, 17 Nov 2017 02:43:12 +0000 (20:43 -0600)]
TurboJPEG: Add alpha offset array/method
Also, set the red/green/blue offsets for TJPF_GRAY to -1 rather than 0.
It was undefined behavior for an application to use those arrays/methods
with TJPF_GRAY anyhow, and this makes it easier for applications to
programmatically detect whether a given pixel format has red, green, and
blue components.
DRC [Fri, 17 Nov 2017 00:09:07 +0000 (18:09 -0600)]
TurboJPEG C API: Add BMP/PPM load/save functions
The main justification for this is to provide new libjpeg-turbo users
with a quick & easy way of developing a complete JPEG
compression/decompression program without requiring them to build
libjpeg-turbo from source (which was necessary in order to use the
project-private bmp API) or to use external libraries. These new
functions build upon significant enhancements to rdbmp.c, wrbmp.c,
rdppm.c, and wrppm.c which allow those engines to convert directly
between the native pixel format of the file and a pixel format
("colorspace" in libjpeg parlance) specified by the calling program.
rdbmp.c and wrbmp.c have also been modified such that the calling
program can choose to read or write image rows in the native (bottom-up)
order of the file format, thus eliminating the need to use an inversion
array. tjLoadImage() and tjSaveImage() leverage these new underlying
features in order to significantly improve upon the performance of the
old bmp API.
Because these new functions cannot work without the libjpeg-turbo
colorspace extensions, the libjpeg-compatible code in turbojpeg.c has
been removed. That code was only there to serve as an example of how
to use the TurboJPEG API on top of libjpeg, but more specific, buildable
examples now exist in the https://github.com/libjpeg-turbo/ijg
repository.
DRC [Wed, 15 Nov 2017 15:19:27 +0000 (09:19 -0600)]
Build: Fix 'tjtest' target on Windows
tjbenchtest and its Java derivatives are useful for rooting out hidden
problems with the more esoteric TJBench and TurboJPEG features. For
instance, on Windows, running tjbenchtest uncovered 5fce2e942136cb70e5a30ff15a2d58b07947aa84.
This commit also causes tjbenchtest and tjbenchtest.java to append -yuv
and -alloc to their log file names, depending on the arguments passed,
and it causes the build system to clean up those log files when the
'testclean' target is built.
DRC [Thu, 16 Nov 2017 01:39:45 +0000 (19:39 -0600)]
Uniquify tjbenchtest log file names based on args
+ clean up log files when 'make testclean' is invoked
+ fix 'tjbenchtest -yuv -alloc'
+ fix tjexampletest so that it creates images under /tmp
+ clean up tjexampletest
DRC [Tue, 14 Nov 2017 21:30:06 +0000 (15:30 -0600)]
tjbench.exe: Fix decompression access violation
The program crashed when a JPEG image was passed on the command line,
because we were mixing our metaphors vis-a-vis malloc()/free() and
tjAlloc()/tjFree() (malloc()/free() uses the tjbench.exe heap,
whereas tjAlloc()/tjFree() uses the turbojpeg.dll heap.)
DRC [Tue, 14 Nov 2017 03:01:53 +0000 (21:01 -0600)]
Further partial image decompression fixes
- Referring to 073b0e88a192adebbb479ee2456beb089d8b5de7 and #185, the
reason why BMP and RLE didn't (and won't) work with partial image
decompression is that the output engines for both formats maintain a
whole-image buffer, which is used to reverse the order of scanlines.
However, it was straightforward to add -crop support for GIF and
Targa, which is useful for testing partial image decompression along
with color quantization.
- Such testing reproduced a bug reported by Mozilla (refer to PR #182)
whereby jpeg_skip_scanlines() would segfault if color quantization was
enabled. To fix this issue, read_and_discard_scanlines() now sets up
a dummy quantize function in the same manner that it sets up a dummy
color conversion function.
DRC [Thu, 9 Nov 2017 03:01:57 +0000 (21:01 -0600)]
djpeg -crop: Exit gracefully with non-PPM formats
... and document that only PPM/PGM output images are supported with the
-crop option for the moment.
I investigated the possibility of supporting -crop with -bmp, but even
after resetting the buffer dimensions, I still kept getting virtual
array access errors. It seems that doing this the "right way" would
require creating a re-initialization function for each image format's
destination manager. I'm disinclined to do that right now, given that
this feature was Google's baby (developed as a prerequisite for
including libjpeg-turbo in Android), and the -crop option in djpeg is
intended only as an example of how to use the partial image
decompression API. Real-world applications would need to handle this
in their own destination managers.
It would probably be possible to make this work with Targa by employing
a similar hack to the one we used with PPM, but Targa isn't popular
enough to bother.
Setting _libdir to CMAKE_INSTALL_FULL_LIBDIR only works when doing an
in-tree RPM build. SRPMs are architecture-agnostic, so the spec needs
to compute_libdir at the time the SRPM is rebuilt, not at the time it
is generated.
This is a regression introduced when implementing the new CMake-based
cross-platform build system.
The version of RPM on RHEL 5 and older platforms defines _libdir
as %{_exec_prefix}/%{_lib}, so defining _lib in the spec file redefined
_libdir. However, newer versions of RPM (probably >= 4.6, since that
was the version that introduced the ISA macros) define _libdir as either
%{_prefix}/lib or %{_prefix}/lib64. Thus, we need to explicitly
override _libdir in our spec file.
It is necessary for the C code to be aware of the machine's endianness,
which is why the TurboJPEG Java wrapper sets a different pixel format
for integer BufferedImages depending on ByteOrder.nativeOrder().
However, it isn't necessary to handle endianness in pure Java code such
as TJUnitTest (d'oh!) This was a product of porting the C version of
TJUnitTest too literally, and of insufficient testing (historically,
the big endian systems I had available for testing didn't have Java.)
DRC [Mon, 14 Aug 2017 15:54:27 +0000 (10:54 -0500)]
Java: Fix NullPointerException in YUVImage
planes == null is a valid argument to setBuf() if alloc == true, so we
need to make sure that planes is non-null before validating its length.
We also need to allocate one dimension of the planes array if it's null.
DRC [Thu, 29 Jun 2017 21:49:09 +0000 (16:49 -0500)]
TJBench: Recover from non-fatal errors if possible
Previously, -stoponwarning only had an effect on the underlying
TurboJPEG C functions, but TJBench still aborted if a non-fatal error
occurred. This commit modifies the C version of TJBench such that it
always recovers from a non-fatal error unless -stoponwarning is
specified. Furthermore, the benchmark stores the details of the last
non-fatal error and does not print any subsequent non-fatal error
messages unless they differ from the last one.
Due to limitations in the Java API (specifically, the fact that it
cannot communicate errors, fatal or otherwise, to the calling program
without throwing a TJException), it was only possible to make
decompression operations fully recoverable within TJBench. With other
operations, -stoponwarning still has an effect on the underlying C
library but has no effect at the Java level.
The Java API documentation has been amended to reflect that only certain
methods are truly recoverable, regardless of the state of
TJ.FLAG_STOPONWARNING.
Allow progressive entropy coding to be enabled on a
transform-by-transform basis, and implement a new transform option for
disabling the copying of markers.
DRC [Wed, 28 Jun 2017 19:47:45 +0000 (14:47 -0500)]
TurboJPEG: Document xform issue w/ big marker data
If the source image for a transform operation has a lot of EXIF or ICC
data embedded in it, then it may cause the output image size to exceed
the worst-case size returned by tjBufSize() (because tjTransform()
transfers all markers to the output image.) This is only a problem if
TJFLAG_NOREALLOC is passed to the function. Since the TurboJPEG C API
doesn't require the destination image size to be set in this case, it
makes the documented assumption that the calling program has allocated
the destination buffer to exactly the size returned by tjBufSize().
Changing this assumption would change the API behavior and necessitate
a new function name (tjTransform2().) At the moment, it's easier to
just document this as a known issue, since it's easy to work around in
the C API.
The Java API is unfortunately a different story, since it must always
use TJFLAG_NOREALLOC (because, when using the TurboJPEG Java API, all
buffers are allocated on the Java heap, and thus they can't be
reallocated by the C code.) There is no easy way to work around this
without changing the C API as discussed above, because if the source
image contains large amounts of marker data, it's virtually impossible
to determine how big the output image will be.
DRC [Tue, 27 Jun 2017 19:22:39 +0000 (14:22 -0500)]
TJBench: Improve consistency of results
Given that libjpeg-turbo can often process hundreds of megapixels/second
on modern hardware, the default of one warmup iteration was essentially
meaningless. Furthermore, the -warmup option was a bit clunky, since
it required some foreknowledge of how fast the benchmarks were going to
execute.
This commit introduces a 1-second warmup interval for each benchmark by
default, and the -warmup option has been retasked to control the length
of that interval.
DRC [Tue, 27 Jun 2017 15:54:21 +0000 (10:54 -0500)]
TurboJPEG: Improve error handling
- Provide a new C API function and TJException method that allows
calling programs to query the severity of a compression/decompression/
transform error.
- Provide a new flag that instructs the library to immediately stop
compressing/decompressing/transforming if a warning is encountered.
DRC [Fri, 12 May 2017 02:02:29 +0000 (21:02 -0500)]
TurboJPEG: Thread-safe error message retrieval
Introduce a new C API function (tjGetErrorStr2()) that can be used to
retrieve compression/decompression/transform error messages in a
thread-safe (i.e. instance-specific) manner. Retrieving error messages
from global functions is still thread-unsafe.
DRC [Tue, 27 Jun 2017 01:48:02 +0000 (20:48 -0500)]
TJBench: Fix errors when decomp. files w/ ICC data
Embedded ICC profiles can cause the size of a JPEG file to exceed the
size returned by tjBufSize() (which is really meant to be used for
compression anyhow, not for decompression), and this was causing a
segfault (C) or an ArrayIndexOutOfBoundsException (Java) when
decompressing such files with TJBench. This commit modifies the
benchmark such that, when tiled decompression is disabled, it re-uses
the source buffer as the primary JPEG buffer.
DRC [Mon, 8 May 2017 13:15:11 +0000 (08:15 -0500)]
Travis: Fix OS X build
The Travis xcode7.3 image now apparently includes GnuPG 1.4.x by
default, so use it instead of installing GnuPG 2. Using GnuPG 2.1.x,
the default version in Homebrew as of this writing, is problematic for
this reason:
https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase
This code was submitted in the initial ARM NEON patches
(https://sourceforge.net/p/libjpeg-turbo/patches/7/) by Siarhei while he
was still a Nokia employee.
DRC [Sat, 18 Mar 2017 21:15:14 +0000 (16:15 -0500)]
Honor max_memory_to_use/JPEGMEM/-maxmemory
This re-introduces a feature of the obsolete system-specific libjpeg
memory managers-- namely the ability to limit the amount of main memory
used by the library during decompression or multi-pass compression.
This is mainly beneficial for two reasons:
- Works around a 2 GB limit in libFuzzer
- Allows security-sensitive applications to set a memory limit for the
JPEG decoder so as to work around the progressive JPEG exploit
(LJT-01-004) described here:
http://www.libjpeg-turbo.org/pmwiki/uploads/About/TwoIssueswiththeJPEGStandard.pdf
This commit also removes obsolete documentation regarding the MS-DOS
memory manager (which itself was removed long ago) and changes the
documentation of the -maxmemory switch and JPEGMEM environment variable
to reflect the fact that backing stores are never used in libjpeg-turbo.
DRC [Sat, 18 Mar 2017 18:24:50 +0000 (13:24 -0500)]
AppVeyor: Fix CI build
Something changed in the CI build environment, and our previous trick of
setting the Git URL to file://c:/projects/libjpeg-turbo no longer works.
Using cygpath to translate the Windows path to a MinGW-friendly format
is a better solution anyhow.
DRC [Sat, 18 Mar 2017 17:56:36 +0000 (12:56 -0500)]
TurboJPEG: Fix potential memory leaks
Referring to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=746,
it seems that the values of local buffer pointers in TurboJPEG API
functions aren't always preserved if longjmp() returns control to a
point prior to the allocation of the local buffers. This is known to
be an issue with GCC 4.x and clang with -O1 and higher optimization
levels but not with GCC 5.x and later. It is unknown why GCC 5.x and
6.x do not suffer from the issue, but possibly the local buffer pointers
are not allocated on the stack when using those more recent compilers.
In any case, this commit modifies the TurboJPEG API library code such
that the jump buffer is always updated after any local buffer pointers
are allocated but before any subsequent libjpeg API functions are
called.
DRC [Thu, 19 Jan 2017 21:18:57 +0000 (15:18 -0600)]
libjpeg API: Support reading/writing ICC profiles
This commit does the following:
-- Merges the two glueware functions (read_icc_profile() and
write_icc_profile()) from iccjpeg.c, which is contained in downstream
projects such as LCMS, Ghostscript, Mozilla, etc. These functions were
originally intended for inclusion in libjpeg, but Tom Lane left the IJG
before that could be accomplished. Since then, programs and libraries
that needed to embed/extract ICC profiles in JPEG files had to include
their own local copy of iccjpeg.c, which is suboptimal.
-- The new functions were prefixed with jpeg_ and split into separate
files for the compressor and decompressor, per the existing libjpeg
coding standards.
-- jpeg_write_icc_profile() was made slightly more fault-tolerant.
It will now trigger a libjpeg error if it is called before
jpeg_start_compress() or if it is passed NULL arguments.
-- jpeg_read_icc_profile() was made slightly more fault-tolerant.
It will now trigger a libjpeg error if it is called before
jpeg_read_header() or if it is passed NULL arguments. It will also
now trigger libjpeg warnings if the ICC profile data is corrupt.
-- The code comments have been wordsmithed.
-- Note that the one-line setup_read_icc_profile() function was not
included. Instead, libjpeg.txt now documents the need to call
jpeg_save_markers(cinfo, JPEG_APP0 + 2, 0xFFFF) prior to calling
jpeg_read_header(), if jpeg_read_icc_profile() is to be used.
-- Adds documentation for the new functions to libjpeg.txt.
-- Adds an -icc switch to cjpeg and jpegtran that allows those programs
to embed an ICC profile in the JPEG files they generate.
-- Adds an -icc switch to djpeg that allows that program to extract an
ICC profile from a JPEG file while decompressing.
-- Adds appropriate unit tests for all of the above.
-- Bumps the SO_AGE of the libjpeg API library to indicate the presence
of new API functions.
Note that the licensing information was obtained from:
https://github.com/mm2/Little-CMS/issues/37#issuecomment-66450180
DRC [Fri, 20 Jan 2017 00:51:41 +0000 (18:51 -0600)]
Always tweak EXIF w/h tags w/ lossless transforms
... even if using libjpeg v6b emulation. Previously
adjust_exif_parameters() was only called with libjpeg v7/v8 emulation,
but due to a bug (which this commit also fixes), it only worked properly
with libjpeg v8 emulation.
DRC [Thu, 19 Jan 2017 23:50:59 +0000 (17:50 -0600)]
Fix error w/ lossless crop & libjpeg v7 emulation
The JPEG_LIB_VERSION #ifdef in jtransform_adjust_parameters() was
incorrect, which caused a "Bogus virtual array access" error when
attempting to use the lossless crop feature.
DRC [Sat, 10 Dec 2016 15:32:23 +0000 (09:32 -0600)]
LICENSE.md: Include text of BSD/zlib licenses
LICENSE.md is included in the binary distributions as well, so it
doesn't make much sense to refer to license headers in source files that
aren't necessarily going to be there.
DRC [Sat, 10 Dec 2016 15:00:39 +0000 (09:00 -0600)]
Build: Don't require sudo for `make tarball`
The whole point of `make tarball` is to make it easy for users to create
a binary distribution of libjpeg-turbo on platforms that aren't
supported by our official build system, so requiring root permissions
somewhat defeated that purpose. Intead, the script now attempts to
detect whether the system has GNU tar or a recent version of BSD tar
that supports setting the ownership of the files in the tarball.
DRC [Fri, 9 Dec 2016 23:12:12 +0000 (17:12 -0600)]
Mac pkg: Use PKGNAME for documentation directory
Although there is little chance that we will ever have a package
conflict on OS X, the convention from our Linux packages is to use the
package name, not the project name, for the name of the documentation
directory.
DRC [Fri, 9 Dec 2016 16:21:29 +0000 (10:21 -0600)]
Build: More GNUInstallDirs improvements
These improvements enable build systems to use GNUInstallDirs to define
custom directory variables.
- The set_dir() macro was renamed to GNUInstallDirs_set_install_dir(),
in keeping with the module's established macro naming convention.
- Rather than detecting whether the prefix has changed, the new
GNUInstallDirs_set_install_dir() macro instead examines whether the
default for the variable in question has changed. This allows for
more flexibility, since build systems may decide to change the
defaults based on factors other than the prefix. It also enables the
macro to work properly outside of the module.
- The module now performs directory variable substitution within the
body of GNUInstallDirs_get_absolute_install_dir().
- The JAVADIR variable is no longer included in GNUInstallDirs. That
directory is not part of the GNU spec, and it turns out that various
operating systems use different conventions for the location of Java
classes. Instead, the variable is now implemented in our build
system as a demonstration of the aforementioned GNUInstallDirs
enhancements.
DRC [Thu, 8 Dec 2016 20:43:59 +0000 (14:43 -0600)]
Build: Various improvements to install/pkg system
- GNUInstallDirs: any directory variable can now reference any other
directory variable by including its name in angle brackets (<>).
- Changed the documentation of the directory variables in BUILDING.md
accordingly. This commit also includes some formatting tweaks to
that section (using boldface for directory names, as is our
convention.)
- Changed the package scripts such that they use
CMAKE_INSTALL_DATAROOTDIR rather than CMAKE_INSTALL_DATADIR.
- We no longer override the install dir. defaults on Windows unless
performing an official build. It may be useful, for instance, to
use the GNU defaults when installing into an MSYS environment.
DRC [Thu, 8 Dec 2016 01:14:20 +0000 (19:14 -0600)]
Build: Minor tweaks to GNUInstallDirs defaults
It isn't actually necessary to specify `CMAKE_INSTALL_DEFAULT_MANDIR`
for our official build. Because `CMAKE_INSTALL_DEFAULT_DATAROOTDIR` is
blank for the official build, the default of "<DATAROOTDIR>/man" will
resolve to "man".
For the same reason, this commit changes the specification of
`CMAKE_INSTALL_DEFAULT_DOCDIR` and `CMAKE_INSTALL_DEFAULT_JAVADIR` in
the official build to be dependent on the data root directory (mainly to
make it obvious what we're doing.)
This commit also tweaks the example CMake command line in the directory
variable documentation so that it shows the correct location of the
CMake argument.
DRC [Thu, 8 Dec 2016 00:11:38 +0000 (18:11 -0600)]
Build: Fix Debug/RelWithDebInfo build with YASM
YASM requires a debug format to be specified with -g. Currently the
only combination that I can make work at all is DWARF-2/ELF (YASM
doesn't support Mach-O debugging at all, and its support for CV8/MSVC
and MinGW/DWARF-2 appears to be broken), so debugging is only enabled
automatically for ELF at the moment. For other formats, we don't
specify -g at all, which is how the old build system behaved.
projects / libjpeg-turbo / log