Jan Tojnar [Mon, 13 Apr 2020 14:16:11 +0000 (16:16 +0200)]
build: Produce correct pc file with absolute libdir
The CMAKE_INSTALL_INCLUDEDIR and CMAKE_INSTALL_LIBDIR variables
can be absolute paths so simply appending them to \${prefix}
might not work.
Unfortunately, CMake does not have simple function for joining paths [1]
so I had to write my own suboptimal one.
It somewhat handles the following cases
- prefix absolute + libdir relative
- prefix absolute + libdir absolute but not under prefix
but for
- prefix absolute + libdir absolute & under prefix
we would want to strip the prefix and make the libdir relative
so we can make it relative to the pkg-config variables.
Basically, we want the equivalent of the following Python function:
import os.path
from pathlib import PurePath
if os.path.isabs(libdir) and os.path.commonpath([libdir, prefix]) == str(PurePath(prefix)):
# Convert path to be relative to prefix
skip = len(prefix) + 1
libdir = libdir[skip:]
return os.path.join('${prefix}', libdir)
This would be non-trivial to replicate in CMake scripts as far as I can tell.
Jan Tojnar [Mon, 13 Apr 2020 12:57:57 +0000 (14:57 +0200)]
build: Install man pages correctly
CMAKE_INSTALL_MANDIR can be absolute path so simply appending it to CMAKE_INSTALL_PREFIX will not work an may produce paths like /nix/store/5r86wqbn6ncdxrdwvmskc381krm32pzh-zziplib-0.13.70/nix/store/5r86wqbn6ncdxrdwvmskc381krm32pzh-zziplib-0.13.70/…
There's various warnings in the generated documentation either due to
missing links or to mistaken use of simple comments `/* */` instead of
docstyle comments `/** */`. Fix some of these by converting to docstyle
comments or fixing the referenced function names.
Commit 5b660a3 (Merge pull request #72 from jelly/python3, 2019-08-04)
introduced a conversion to use new-style Python exceptions.
The conversion introduced a new print statements, though, breaking
compatibility with Python 3 again.
Convert the statement to use the print function, instead.
With recent changes CMake is now favoring Python 3 instead of Python 2.
As Ubuntu has both versions of the interpreter installed, Azure has thus
started to use Python 3, but will fail due to a missing dependency on
xmlreporting when executing tests. This is caused by us installing
xmlreporting for Python 2, only.
As there is no python3-xmlreporting package available in Ubuntu, install
python3-pip instead and use it to install unittest-xml-reporting.
Now that all Python scripts are compatible with Python 2 and Python 3,
we can instruct CMake to not care for which Python interpreter is
available. Thus, convert to use the newer FindPython package without
specifying a version.
Instead of inheriting str, we can simply implement `__repr__` to return
a string representation of the pattern itself. This fixes compatibility
with Python 3, which throws when executing `str.__init__`.
In Python 3, the function string.replace has been removed in favor of
using replace on the string class directly. As Python 2.7 already
supports this variant, convert all instances of the old way to call
replace on the object.
The `has_key` function for dictionaries has been removed with Python 3
in favor of the new `in` keyword. Convert all uses of the old function
in favor of the new to improve compatibility with Python 3.
docs: use print function for compatibility with Python 3
Another quite visible change with Python 3 is that the old `print`
statement has been replaced with a function. To ease migration to the
newer way, Python 2.7 provides a compatibility module "print_function".
Convert all code in docs to use the new function syntax.
docs: exception syntax compatibility with Python 3
Since Python 2.7, there exists a new syntax to catch errors: the old way
of saying `except IOError, error` has been replaced with `except IOError
as e`. The old syntax has finally been removed in Python 3, but as we're
still using it in zziplib newer interpreters will raise a parsing error.
Convert to use the new syntax to improve compatibility with Python 3.
One of the most important and user-visible changes in Python 3 was that
when reading strings from some , one always needs to specify the
encoding lest the dreaded UnicodeException would occur if there were any
characters outside of the ASCII range. In the test suite, we're using a
subprocess taht communicates with the shell, but we do not specify the
encoding of both its stdout and stderr previous to reading them.
Explicitly decode both their output and error streams to UTF-8 to avoid
any exceptions raised by Python 3.
In Python 2, strings were represented by the class basestring, which has
been removed in Python 3 in favor of the str class. To keep
compatibility with both versions of Python, let's alias basestring to
str if basestring is not available.
test: use Python3-compatible way to specify octal numbers
Support for zero-prefixed octal numbers ("0755") has been removed in
Python 3 in favor of the zero-oh-prefix ("0o755"). As the latter has
been introduced in Python 2.6, we can safely switch all occurrences of
the former with the new way to improve compatibility with Python 3.
Back in Python 2 days, there were two functions `range` and `xrange`.
While `range` returned a list, `xrange` would return a generator and
thus be the more efficient variant if a big range needs to be handled.
With Python 3, the `xrange` function was removed in favor of always
returning a generator for `range`.
Convert all uses of `xrange` to `range`. While this may be less
efficient in some places for old Python versions, their support is going
to be dropped in 2020 anyway. Also, `xrange` is only ever used in the
test suite.
With Python 3, StringIO has been moved from the "cStringIO" module to
the "io" module. Use a try-catch block to intercept ImportErrors and
fall back to the new module to support both Python 2 and Python 3.
In Python 3, the functions `quote_plus` and `urlretrieve` have been
moved from urllib to urllib.parse and urllib.request, respectively. To
provide compatibility with the old and new module layouts, use a
try-catch block and fall back to the new layout if an ImportError was
raised.
Furthermore, do not hand-code `urlretrieve` functionality with
`urlopen` to avoid additional compatibility issues.
tests: loosen expectations for external info-zip tool
In our test suite, we're verifying some archives by utilizing the
unzip(1) program provided by info-zip.org. Due to changes to info-zip,
our test case 59770 may fail on some systems as we expect it to spit out
error messages in a specific format. Loosen those restrictions to make
the test pass on more systems.
test: fix error message tests on non-glibc based systems
Our tests makes strict assumptions on the error messages generated by
strerror(3P), but those aren't in any way required to have a special
text. This works just fine on glibc based systems, but causes us to
trigger asserts on other systems like e.g. musl based ones.
Fix this by introducing a new `assertErrorMessage` method. It'll take as
inputs the observed error message as well as the error code we expect
and then use `os.strerror` to compute the expected message. Like this,
we're not dependent anymore on how the system represents those error
messages but generate the expected ones on the fly.
zzip: memdisk: use correct type when when copying extra blocks
The data field of `struct zzip_disk` is of type `zzip_byte_t *`, which
is unsigned, but we store them in a `char` field and thus cause
signedness warnings on platforms where `char` is signed. Fix this by
using `zzip_byte_t`, instead.
In `zzip_mem_disk_fdopen`, there's two debug calls with format strings
that print out the file descriptor that we're about to open. While the
first format string correctly uses "%i" to print the integer, the second
one accidentally uses "%s". Fix this by using "%i" for both.
zzip: info: avoid warning due to comparison with different signedness
In `zzip_compr_str` we compare the integer value `compr` with
`LENGTH(comprlevel)`, which returns an unsigned value, causing the
compiler to emit a warning. As we're already verifying that `compr` is a
positive value, we can just cast it to an unsigned integer to avoid that
warning.
zzip: fseeko: fix comparison between incompatible pointer types
The C standard does not allow for comparing incompatible pointer types
with each other. This causes the C compiler to emit a warning when we
try to compare a pointer of type `struct zzip_disk64_trailer` (cast to
`void *`) with a pointer of type `unsigned char`.
Fix this by comparing `p + sizeof(trailer)` with `buffer + mapsize`
instead of using the cast struct.
zzip: fseeko: fix signedness mismatch with `file->avail`
While the `avail` field of `struct zzip_entry_file` is of unsigned type
`zzip_size_t`, we assign it to a local variable of `zzip_off_t` in
`zzip_entry_fopen`, which is of signed type. This causes the compiler to
emit a warning due to signedness when comparing it with
`sizeof(file->buffer)`.
Fix this by using the correct type for the local variable.
zzip: format: explicitly mark header length constants as unsigned
The various different `headerlength` macros in "format.h" are defined as
simple additions of all the field lengths of its struct representation.
While it is obvious that the resulting constants are always positive,
the C language specifies the result as a signed integer, causing
warnings at several places due to comparisons with different signedness.
Fix the issue by explicitly marking the constants as unsigned. A simpler
fix would just replace those constants with `sizeof`, which should be
fine due to the struct's being marked as packed. But for the sake of
simplicitly, let's just stay with the more obvious fix.
When subtracting two pointers, the result is signed as the subtrahend
may point at a position after the minuend, in which case it is bigger
and the result would be negative. This causes a warning in
`_zzip_strnlen`, which is supposed to return an unsigned `size_t`.
As the minuend is computed by using `memchr` on the subtrahend, we know
that it is always either `NULL` or bigger than the subtrahend. And due
to the subtraction only being performed when it is non-`NULL`, we may
simply cast the result to `size_t` to fix the issue, fixing the warning.
The `_zzip_strnlen` function is annotated with the `zzip_new__` macro,
which is defined to `__attribute__((malloc))` on systems which support
this attribute. The malloc attribute can be applied to functions that
return a pointer value, indicating that a returned non-NULL pointer
cannot alias and that its contents have undefined content. But as
`_zzip_strnlen` neither performs an allocation nor returns a pointer to
memory, we cannot use that attribute here.
While there is a "--bindir" option added to the option parser of
zziptests.py, we never update the global "bindir" variable to the new
value. Do so to fix it not getting honored.
docs: dbk2man: fix deprecated way of checking for `None`
In oldish versions of Python, it was possible to check for `None` by
simply saying `if not X`, while the newer way would explicitly check `if
X is None`. Use the latter way of asking for None, which has been
supported at least since Python 2.6.
When doing out-of-tree builds, a golden rule is to not touch any files
inside of the source directory. This rule is being violated by our tests
CMake instructions, which generate a "test0.zip" file and copy it over
the file "test/test.zip" in our source directory.
Stop doing this. We already copy the "test0.zip" to "test.zip" in the
binary directory, and all tests should be using that one.
projects / zziplib / log