Richard Levitte [Fri, 12 Feb 2016 17:16:23 +0000 (18:16 +0100)]
Rethink logging of test recipes
The logging that was performed in OpenSSL::Test was initially set up
as a means not to let messages that test programs write to STDERR get
displayed when a test isn't running in verbose mode. However, the way
it was implemented, it meant that those messages were never displayed,
and you had to look in a test log. This also meant that output to
STDERR and output to STDOUT got broken apart, which isn't optimal.
So, we remove the whole test log file implementation, and instead,
we're sending STDERR to the null device unless one of these conditions
apply:
- the test recipe already redirects stderr. Just let it.
- the environment variable HARNESS_ACTIVE is undefined, meaning the
recipe is run directly as a perl script instead of being harnessed
by Test::Harness
- the environment variable HARNESS_VERBOSE is set.
Getting a full log of the tests now becomes as simple as this:
HARNESS_VERBOSE=yes make test 2>&1 | tee tests.log
Richard Levitte [Fri, 12 Feb 2016 18:44:55 +0000 (19:44 +0100)]
Better workaround for VMS getnameinfo() bug
The actual bug with current getnameinfo() on VMS is not that it puts
gibberish in the service buffer, but that it doesn't touch it at all.
The gibberish we dealt with before was simply stuff that happened to
be on the stack.
It's better to initialise the service buffer properly (with the empty
string) and check if it's still an empty string after the
getnameinfo() call, and fill it with the direct numerical translation
of the raw port if that's the case.
Richard Levitte [Fri, 12 Feb 2016 03:23:15 +0000 (04:23 +0100)]
Adjust transfer::Text::Template.pm for alternate directory name
On VMS, periods in directory names weren't allowed. To counter that,
unpackers such as VMSTAR convert periods in directory names to
underscores. We need to count that in and add an alternative library
path for Text::Template.
Richard Levitte [Thu, 11 Feb 2016 23:34:40 +0000 (00:34 +0100)]
Generate progs.h from a bunch of files instead of internal knowledge
apps/progs.pl counted on the caller to provide the exact command
files. The unified build doesn't have that knowledge, and the easier
and more flexible thing to do is to feed it all the apps/*.c files and
let it figure out the command names by looking inside (looking for
/int ([a-z0-9][a-z0-9_]*)_main\(int argc,/).
Also, add it to the generate command, since it's a versioned file.
Richard Levitte [Thu, 11 Feb 2016 20:47:30 +0000 (21:47 +0100)]
Perl's chop / chomp considered bad, use a regexp instead
Once upon a time, there was chop, which somply chopped off the last
character of $_ or a given variable, and it was used to take off the
EOL character (\n) of strings.
... but then, you had to check for the presence of such character.
So came chomp, the better chop which checks for \n before chopping it
off. And this worked well, as long as Perl made internally sure that
all EOLs were converted to \n.
These days, though, there seems to be a mixture of perls, so lines
from files in the "wrong" environment might have \r\n as EOL, or just
\r (Mac OS, unless I'm misinformed).
So it's time we went for the more generic variant and use s|\R$||, the
better chomp which recognises all kinds of known EOLs and chops them
off.
A few chops were left alone, as they are use as surgical tools to
remove one last slash or one last comma.
NOTE: \R came with perl 5.10.0. It means that from now on, our
scripts will fail with any older version.
Andy Polyakov [Wed, 3 Feb 2016 17:21:00 +0000 (18:21 +0100)]
util/mk1mf.pl: use LINK_CMD instead of LINK variable.
Trouble is that LINK variable assignment in make-file interferes with
LINK environment variable, which can be used to modify Microsoft's
LINK.EXE behaviour.
RT#4289
Reviewed-by: Richard Levitte <levitte@openssl.org>
Viktor Dukhovni [Thu, 11 Feb 2016 18:44:53 +0000 (13:44 -0500)]
Fix MacOS/X build warnings
Commit 7823d792d0cad3b44ad5389a8d3381becefe7f44 added DEFINE_LHASH_OF
to a C source file. DEFINE_LHASH_OF() and DEFINE_STACK_OF() must
be used only in header files to avoid clang warnings for unused
static-inline functions.
Richard Levitte [Thu, 11 Feb 2016 12:10:11 +0000 (13:10 +0100)]
Make shared library targets more consistent
On Windows POSIX layers, two files are produced for a shared library,
there's {shlibname}.dll and there's the import library {libname}.dll.a
On some/most Unix platforms, a {shlibname}.{sover}.so and a symlink
{shlibname}.so are produced.
For each of them, unix-Makefile.tmpl was entirely consistent on which
to have as a target when building a shared library or which to use as
dependency.
This change clears this up and makes it consistent, we use the
simplest form possible, {lib}.dll.a on Windows POSIX layers and
{shlibname}.so on Unix platforms. No exception.
Richard Levitte [Thu, 11 Feb 2016 14:22:27 +0000 (15:22 +0100)]
Unified build: Keep track of generated header files
If someone runs a mixed unixmake / unified environment (the unified
build tree would obviously be out of the source tree), the unified
build will pick up on the unixmake crypto/buildinf.h because of
assumptions made around this sort of declaration (found in
crypto/build.info):
DEPENDS[cversion.o]=buildinf.h
The assumption was that if such a header could be found in the source
tree, that was the one to depend on, otherwise it would assume it
should be in the build tree.
This change makes sure that sort of mix-up won't happen again.
Richard Levitte [Wed, 10 Feb 2016 21:33:44 +0000 (22:33 +0100)]
Rework BIO_ADDRINFO_protocol() to return correct values
As noted already, some platforms don't fill in ai_protocol as
expected. To circumvent that, we have BIO_ADDRINFO_protocol() to
compute a sensible answer in that case.
Matt Caswell [Wed, 10 Feb 2016 19:41:27 +0000 (19:41 +0000)]
Fix errstr error code parsing
Error codes are printed in hex, and previous OpenSSL versions expected
the error codes to be provided to errstr in hex. In 1.1.0, for some
reason, it was expecting them to be decimal.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Richard Levitte [Wed, 10 Feb 2016 22:27:28 +0000 (23:27 +0100)]
After auto init, check that the deprecated functions exist before using
The functions that have been deprecated by the auto init changes are
now guarded with deprecation checks, so it's fairly easy to see if
they can be used.
In test/dtlsv1listentest, we simply remove all init and cleanup code,
as they are call automatically when needed.
Richard Levitte [Wed, 10 Feb 2016 19:23:40 +0000 (20:23 +0100)]
Don't assert protocol equality
It seems that some platforms' getaddrinfo don't fill in the
ai_protocol field properly. On those, the assertion
'protocol == BIO_ADDRINFO_protocol(res)' will fail. Best to remove
it.
Richard Levitte [Wed, 10 Feb 2016 18:09:05 +0000 (19:09 +0100)]
Quote the CFLAG in Unixly Makefiles, for buildinf.h
Because the command line definitions of OPENSSLDIR and ENGINESDIR
contain quotes, we need a variant of CFLAG where backslashes and
quotes are escaped when we produce buildinf.h
Matt Caswell [Wed, 10 Feb 2016 15:16:06 +0000 (15:16 +0000)]
Attempt to log an error if init failed
If init failed we'd like to set an error code to indicate that. But if
init failed then when the error system tries to load its strings its going
to fail again. We could get into an infinite loop. Therefore we just set
a single error the first time around. After that no error is set.
Matt Caswell [Wed, 10 Feb 2016 16:46:33 +0000 (16:46 +0000)]
Variable was declared static when it shouldn't be
The local variable tmp was declared static when it shouldn't be. This
is in the no-threads implementation, and it was immediately initialised
to something else on every invokation of the function so it doesn't break
anything...but still shouldn't be there.
Richard Levitte [Sat, 30 Jan 2016 06:15:30 +0000 (07:15 +0100)]
clean away old VMS cruft
The old building scripts get removed, they are hopelessly gone in bit
rot by now.
Also remove the old symbol hacks. They were needed needed to shorten
some names to 31 characters, and to resolve other symbol clashes.
Because we now compile with /NAMES=(AS_IS,SHORTENED), this is no
longer required.
Richard Levitte [Sat, 30 Jan 2016 06:14:58 +0000 (07:14 +0100)]
unified build scheme: add a "unified" template for VMS descrip.mms
As part of this, change util/mkdef.pl to stop adding libraries to
depend on in its output. mkdef.pl should ONLY output a symbol
vector.
Because symbol names can't be longer than 31 characters, we use the
compiler to shorten those that are longer down to 23 characters plus
an 8 character CRC. To make sure users of our header files will pick
up on that automatically, add the DEC C supported extra headers files
__decc_include_prologue.h and __decc_include_epilogue.h.
Furthermore, we add a config.com, so VMS people can configure just as
comfortably as any Unix folks, thusly:
Richard Levitte [Sat, 30 Jan 2016 04:45:29 +0000 (05:45 +0100)]
unified build scheme: add the tweaks to build on Cygwin & Mingw
Cygwin and Mingw name their libraries a bit differently from the rest
of the POSIXly universe, we need to adapt to that.
In Makefile.tmpl, it means that some hunks will only be output
conditionally.
This also means that shared_extension for the Cygwin and Mingw
configurations in Configurations/10-main.conf are changing from .dll.a
to .dll. Makefile.shared does a fine job without having them
specified, and it's much easier to work with tucking an extra .a at
the end of files in the installation recipes than any amount of name
rewrites, especially with the support of the SHARED_NAME in the top
build.info.
Richard Levitte [Sat, 30 Jan 2016 02:21:39 +0000 (03:21 +0100)]
unified build scheme: give util/dofile.pl the possibility to output selectively
Under certain conditions, one might not want to output certain
sections of a template file. This adds the functions output_off() and
output_on(), reachable inside the templates. And example usage in a
Makefile template could be this:
@ : {- output_off() if $config{no_shared}; "" -}
... lines dealing with shared libraries
@ : {- output_on() -}
Richard Levitte [Sat, 30 Jan 2016 01:57:19 +0000 (02:57 +0100)]
Configure et al: move the installation directory logic to Makefiles
The logic to figure out the combinations of --prefix and --openssldir
has stayed in Configure so far, with Unix paths as defaults.
However, since we're making Configure increasingly platform agnostic,
these defaults need to change and adapt to the platform, along with
the logic to combine them.
The easiest to provide for this is to move the logic and the defaults
away from Configure and into the build files.
This also means that the definition of the macros ENGINESDIR and
OPENSSLDIR move away from include/openssl/opensslconf.h and into the
build files.
Richard Levitte [Sat, 30 Jan 2016 01:17:05 +0000 (02:17 +0100)]
Configure et al: treat C defines separately
With some compilers, C macros are defined differently on the command
line than on Unix. It could be that the flad to define them isn't -D,
it could also be that they need to be grouped together and not be mixed
in with the other compiler flags (that's how it's done on VMS, for
example).
On Unix family platform configurations, we can continue to have macro
definitions mixed in with the rest of the flags, so the changes in
Configurations/*.conf are kept to an absolute minimum.
Matt Caswell [Wed, 10 Feb 2016 09:47:51 +0000 (09:47 +0000)]
If we've not been inited don't deinit
If you call an explicit deinit when we've not been inited then a seg
fault can occur. We should check that we've been inited before attempting
to deinit.
Richard Levitte [Wed, 10 Feb 2016 04:12:52 +0000 (05:12 +0100)]
VMS getnameinfo() seems to have a bug with returned service string
It seems like it gives back gibberish. If we asked for a numeric
service, it's easy to check for a digit in the first position, and
if there isn't any, rewrite it using older methods.
Richard Levitte [Wed, 10 Feb 2016 01:18:38 +0000 (02:18 +0100)]
Simplify the specification of include dirs in the build dir
In build.info files, make the include directory in the build directory
absolute, or Configure will think it should be added to the source
directory top. Configure will turn it into a relative path if
possible.
Richard Levitte [Wed, 10 Feb 2016 01:00:37 +0000 (02:00 +0100)]
Make the processing of build.info files more aware of absolute dirs
There were cases where some input was absolute, and concatenating it
to the diretory to the source or build top could fail spectacularly.
Let's check the input first to see if it's absolute.
And while we're on the subject of checking if a file or dir spec is
absolute using file_name_is_absolute() has its own quirks on VMS,
where a logical name is considered absolute under most circumstances.
This is perfectly correct from a VMS point of view, but when parsing
the build.info files, we want single word file or directory names to
only be checked syntactically. A function isabsolute() that does the
right thing is the solution.
Matt Caswell [Tue, 9 Feb 2016 23:09:44 +0000 (23:09 +0000)]
NULL the thread_local_inits_st pointer after use
After the final use of the thread_local_inits_st we should ensure it is
set to NULL, just in case OPENSSL_INIT_thread_stop gets called again and
it tries to use garbage.
Matt Caswell [Tue, 9 Feb 2016 22:09:56 +0000 (22:09 +0000)]
Stop library before checking for mem leaks
With the new init framework resources aren't released until the process
exits. This means checking for mem leaks before that point finds a lot of
things! We should explicitly close down the library if we're checking for
mem leaks.