Corrected 'cms' exit status when key or certificate cannot be opened

A backport of #4997.
Fixes #4996.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5021)

Stop using unimplemented cipher classes.
Add comments to no longer usable ciphers.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5023)

(cherry picked from commit 643d91fea409b0f010ce990f8f0fac234ae058bc)

Add x509(1) reference

Reviewed-by: Rich Salz <rsalz@openssl.org>

Remove old config that used non-exist util script

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5016)

(cherry picked from commit b44a65512a4a0a299f8f817b63df472e74a0007a)

ec/ecp_nistp*.c: sanitize for undefined/implmentation-specific behaviour.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4974)

(cherry picked from commit 8af7e94d7b720224547efa7e3364857f7f666dd4)

Add comments to NULL func ptrs in bio_method_st

This commit adds comments to bio_method_st definitions where the
function pointers are defined as NULL. Most of the structs have comments
but some where missing and not all consitent.

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4881)

(cherry picked from commit b4ff66223b53ad9d2f5b4efd75bf8d52cc59c8dd)

Fix invalid function type casts.
Rename bio_info_cb to BIO_info_cb.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4943)

Fix a gcc fallthru warning

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4941)

Remove test-runs dir

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4942)

Document the X509_V_FLAG_PARTIAL_CHAIN flag

Also improved documentation of TRUSTED_FIRST

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix more OCSP_resp_get0_signer() nits

Fix a typo for "retrieve" and some indentation.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(cherry picked from commit eb48052ec96a7551391b5955f03f5ef70b3528f6)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4723)

Wrap more of ocspapitest.c in OPENSSL_NO_OCSP

make_dummy_resp() uses OCSP types, and get_cert_and_key() is unused
once make_dummy_resp() is compiled out, so neither can be included
in the build when OCSP is disabled and strict warnings are active.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit cb091295a9ff16f4de1a8b00be444d40ac068d04)

(Merged from https://github.com/openssl/openssl/pull/4723)

Fix coverity-reported errors in ocspapitest

Avoid memory leaks in error paths, and correctly apply
parentheses to function calls in a long if-chain.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit b6306d8049b04dca7fa738a86c892c43ba6a5fc4)

(Merged from https://github.com/openssl/openssl/pull/4723)

Fix typo that cause find-doc-nits failure

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(cherry picked from commit b741fcd2ddc4e94faee75a47c241fa136854c81f)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4723)

make update

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit f1d3de718bc96e70ce76bf2b4ccee128ecbcd100)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4723)

Add OCSP API test executable

Some of the OCSP APIs (such as the recently added OCSP_resp_get0_signer)
do not really merit inclusion in the ocsp(1) utility, but we should still
have unit tests for them.

For now, only test OCSP_resp_get0_signer(), but it should be easy to
add more tests in the future.

Provide an X509 cert and private key in the test's data directory
to use for signing responses, since constructing those on the fly
is more effort than is needed.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 27da13430bfb3c178716cec10e8d5d6134e54f90)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4723)

Add an API to get the signer of an OCSP response

Add a new function OCSP_resp_get0_signer() that looks in the
certs bundled with the response as well as in additional certificates
provided as a function argument, returning the certificate that signed
the given response (if present).

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit ce5886dda80b6f60fb30762381506d5c6f2d995c)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4723)

crypto/bio/bss_dgram.c: annotate fallthrough (-Wimplicit-fallthrough)

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4916)

(cherry picked from commit 147167548bb692dc1b772cf4ee1d0917e45b1aff)

Fix leak in ERR_get_state() when OPENSSL_init_crypto() isn't called yet

If OPENSSL_init_crypto() hasn't been called yet when ERR_get_state()
is called, it need to be called early, so the base initialization is
done.  On some platforms (those who support DSO functionality and
don't define OPENSSL_USE_NODELETE), that includes a call of
ERR_set_mark(), which calls this function again.
Furthermore, we know that ossl_init_thread_start(), which is called
later in ERR_get_state(), calls OPENSSL_init_crypto(0, NULL), except
that's too late.
Here's what happens without an early call of OPENSSL_init_crypto():

    => ERR_get_state():
         => CRYPTO_THREAD_get_local():
         <= NULL;
         # no state is found, so it gets allocated.
         => ossl_init_thread_start():
              => OPENSSL_init_crypto():
                   # Here, base_inited is set to 1
                   # before ERR_set_mark() call
                   => ERR_set_mark():
                        => ERR_get_state():
                             => CRYPTO_THREAD_get_local():
                             <= NULL;
                             # no state is found, so it gets allocated!!!!!
                             => ossl_init_thread_start():
                                  => OPENSSL_init_crypto():
                                       # base_inited is 1,
                                       # so no more init to be done
                                  <= 1
                             <=
                             => CRYPTO_thread_set_local():
                             <=
                        <=
                   <=
              <= 1
         <=
         => CRYPTO_thread_set_local()      # previous value removed!
    <=

Result: double allocation, and we have a leak.

By calling the base OPENSSL_init_crypto() early, we get this instead:

    => ERR_get_state():
         => OPENSSL_init_crypto():
              # Here, base_inited is set to 1
              # before ERR_set_mark() call
              => ERR_set_mark():
                   => ERR_get_state():
                        => OPENSSL_init_crypto():
                             # base_inited is 1,
                             # so no more init to be done
                        <= 1
                        => CRYPTO_THREAD_get_local():
                        <= NULL;
                        # no state is found, so it gets allocated
                        # let's assume we got 0xDEADBEEF
                        => ossl_init_thread_start():
                             => OPENSSL_init_crypto():
                                  # base_inited is 1,
                                  # so no more init to be done
                             <= 1
                        <= 1
                        => CRYPTO_thread_set_local():
                        <=
                   <=
              <=
         <= 1
         => CRYPTO_THREAD_get_local():
         <= 0xDEADBEEF
    <= 0xDEADBEEF

Result: no leak.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4913)

(cherry picked from commit aef84bb4efbddfd95d042f3f5f1d362ed7d4faeb)

Fix typo in comment

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4903)

Fix a switch statement fallthrough

SSL_trace() has a case which was inadvertently falling through.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4888)

Useless conf != NULL test

check is already made 10 line above.
clean commented code

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4885)

(cherry picked from commit df364297495e1d7409f1fcb0f4432c8671019cd8)

Make BIO_METHOD struct definitions consistent

I noticed that some of the BIO_METHOD structs are placing the name on
the same line as the type and some don't. This commit places the name
on a separate line for consistency (which looks like what the majority
do)

CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4878)

(cherry picked from commit 27ab91951c96364351f1ea0652dbf14622440345)

Standardize syntax of sizeof(foo)

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4876)

Correct minor typo in ssl_locl.h comment

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4880)

(cherry picked from commit 6aff543b9bf14bdac7f5385a9e56b8785b30f6d8)

Fix an incoherent test.

Pointer 'o' is set inside a local buffer, so it can't be NULL.
Also fix coding style and add comments

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4754)
(cherry picked from commit cef115ff0ca4255d3decc1dda87c5418a961fd2c)

In apps_startup(), call OPENSSL_init_ssl() rather than OPENSSL_init_crypto()

Otherwise, any command that relies on ssl modules may fail, because
SSL_add_ssl_module() will be called after the config file has already
been loaded.

Fixes #4788

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4792)

(cherry picked from commit 0488c0bbbe87eee3a800797b91350c653e9f1711)

In OPENSSL_init_ssl(), run the base ssl init before OPENSSL_init_crypto()

IF OPENSSL_init_ssl() is called with the option flag
OPENSSL_INIT_LOAD_CONFIG, any SSL config will be handled wrongly
(i.e. there will be an attempt to load libssl_conf.so or whatever
corresponds to that on non-Unix platforms).  Therefore, at least
SSL_add_ssl_module() MUST be called before OPENSSL_init_crypto() is
called.  The base ssl init does that, plus adds all kinds of ciphers
and digests, which is harmless.

Fixes #4788

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4792)

(cherry picked from commit 0a90a6831e02e00d9043ada635421cfd3da5ffe2)

Remove unicode characters from source

Some compilers react badly to non-ASCII characters

Fixes #4877

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4879)

(cherry picked from commit d68a0eaf45f12392065f3cf716a1a2682d55d3ce)

Fix the buffer sizing in the fatalerrtest

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4868)

Update CHANGES and NEWS for the new release

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add a test for CVE-2017-3737

Test reading/writing to an SSL object after a fatal error has been
detected. This CVE only affected 1.0.2, but we should add it to other
branches for completeness.

Reviewed-by: Rich Salz <rsalz@openssl.org>

bn/asm/rsaz-avx2.pl: fix digit correction bug in rsaz_1024_mul_avx2.

Credit to OSS-Fuzz for finding this.

CVE-2017-3738

Reviewed-by: Rich Salz <rsalz@openssl.org>

Update eng_fat.c

This line will allow use private keys, which created by Crypto Pro, to
sign with OpenSSL.

CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4836)

(cherry picked from commit b35bb37a3d6ecf11b43ef8717600ab61718c3cc2)

Adjusted Argument Indices
CLA: trivial

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4835)

(cherry picked from commit 1e2804f25c80136c33f3508adb54b24106b6b6f6)

Make possible variant SONAMEs and symbol versions

This small change in the Unix template and shared library build
scripts enables building "variant" shared libraries.  A "variant"
shared library has a non-default SONAME, and non default symbol
versions.  This makes it possible to build (say) an OpenSSL 1.1.0
library that can coexist without conflict in the same process address
space as the system's default OpenSSL library which may be OpenSSL
1.0.2.

Such "variant" shared libraries make it possible to link applications
against a custom OpenSSL library installed in /opt/openssl/1.1 or
similar location, and not risk conflict with an indirectly loaded
OpenSSL runtime that is required by some other dependency.

Variant shared libraries have been fully tested under Linux, and
build successfully on MacOS/X producing variant DYLD names.  MacOS/X
Darwin has no symbol versioning, but has a non-flat library namespace.
Variant libraries may therefore support multiple OpenSSL libraries
in the same address space also with MacOS/X, despite lack of symbol
versions, but this has not been verified.

Variant shared libraries are optional and off by default.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Fix docs for EVP_EncryptUpdate and EVP_DecryptUpdate

Fixes #4775

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4815)

Check for malloc failure

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4805)

(cherry picked from commit 378db52bb0177ae03cac3c3ba194bb6dec34a2d7)

Pretty-print large INTEGERs and ENUMERATEDs in hex.

This avoids taking quadratic time to pretty-print certificates with
excessively large integer fields. Very large integers aren't any more
readable in decimal than hexadecimal anyway, and the i2s_* functions
will parse either form.

Found by libFuzzer.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4790)

(cherry picked from commit 10a3195fcf7d04ba519651cf12e945a8fe470a3c)

Fix EVP_MD_meth_new.pod

A name too many in the NAME section, and a copyright year update

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4789)

(cherry picked from commit 92793648945affdfe529fa711666d19528815789)

Correct EVP_CIPHER_meth_new.pod and EVP_MD_meth_new.pod

One had some lines copied from the other, and both were missing a
proper RETURN VALUES section.

Fixes #4781

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4787)

(cherry picked from commit 51e47d5f6a7944c3e3ddc5f6d376fc1320639277)

Avoid unnecessary MSYS2 conversion of some arguments

Fixes #4740

The MSYS2 run-time convert arguments that look like paths when
executing a program unless that application is linked with the MSYS
run-time. The exact conversion rules are listed here:

    http://www.mingw.org/wiki/Posix_path_conversion

With the built-in configurations (all having names starting with
"mingw"), the openssl application is not linked with the MSYS2
run-time, and therefore, it will receive possibly converted arguments
from the process that executes it. This conversion is fine for normal
path arguments, but it happens that some arguments to the openssl
application get converted when they shouldn't. In one case, it's
arguments like '-passin file:something', and in another, it's a file:
URI (what typically happens is that URIs without an authority
component get converted, 'cause the conversion mechanism doesn't
recognise them as URIs).

To avoid conversion where we don't want it, we simply assign
MSYS2_ARG_CONV_EXCL a pattern to avoid specific conversions. As a
precaution, we only do this where we obviously need it.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4766)

bn/bn_add.c: address performance regression.

Performance regression was reported for EC key generation between
1.0.2 and 1.1.x [in GH#2891]. It naturally depends on platform,
values between 6 and 9% were observed.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4743)

(cherry picked from commit a78324d95bd4568ce2c3b34bfa1d6f14cddf92ef)

asn1/a_strex.c: fix flags truncation in do_esc_char.

|flags| argument to do_esc_char  was apparently truncated by implicit
cast. [Caught by VC warning subsytem.]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4721)

(cherry picked from commit 372463103917fcc2b68bd2ba3db55b29ce325705)

lhash.c: Replace Unicode EN DASH with the ASCII char '-'.

 * addressing", Proc. 6th Conference on Very Large Databases: 212–223
                                                                 ^
The EN DASH ('–') in this line is one UTF-8 character (hex: e2 80 93).
Under some code page setting (e.g. 936), Visual Studio may report C4819
warning: The file contains a character that cannot be represented in the
current code page.

Replace this character with the ASCII char '-' (Hex Code: 2D).

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4691)

(cherry picked from commit b4d0fa49d9d1a43792e58b0c8066bb23b9e53ef4)

Fix possible leaks on sk_X509_EXTENSION_push() failure ...

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4677)

(cherry picked from commit 1687aa760cdd164b12c5b70e65cadcbce1e7ccfa)

util/copy.pl: work around glob quirk in some of earlier 5.1x Perl versions.

In earlier 5.1x Perl versions quoting globs works only if there is
white space. If there is none, it's looking for names starting with ".

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4695)

(cherry picked from commit 1097d2a39e3f85d4dac2c4d1c238792d6e1d959f)

Configurations/unix-Makefile.tmpl: fix HP-UX build.

HP-UX make doesn't recognize $< in explict target rules, only in
inference ones such as .c.o.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4697)

(cherry picked from commit b6705d4893d1566c3a5427e387ce99344497758d)

rc4/build.info: fix HP-UX rc4-ia64 rule.

HP-UX make doesn't recognize $< in explict target rules, only in
inference ones such as .c.o.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4694)

Configure: cleanup @disable_cascade

'rsa', 'sha' and 'tlsext' can't be disabled, not even as a consequence
of other conditions, so having cascading disables that depend on them
is futile.  Clean up!

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4693)

(cherry picked from commit 89635075d84353fc0c3d44a82fd0903ccd4ab24a)

Mark a zero length record as read

If SSL_read() is called with a zero length buffer, and we read a zero length
record then we should mark that record as read.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4686)

Fix race condition in TLSProxy

Normally TLSProxy waits for the s_server process to finish before
continuing. However in cases where serverconnects > 1 we need to keep the
s_server process around for a later test so we continue immediately. This
means that TAP test output can end up being printed to stdout at the same
time as s_server is printing stuff. This confuses the test runner and can
cause spurious test failures. This commit introduces a small delay in cases
where serverconnects > 1 in order to give s_server enough time to finish
what it was doing before we continue to the next test.

Fixes #4129

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4661)

Remove 4 broken macros from ocsp.h

There were 4 macros in ocsp.h that have not worked since 1.1.0 because
they attempt to access the internals of an opaque structure.

For OCSP_REQUEST_sign() applications should use OCSP_request_sign() instead.
For OCSP_BASICRESP_sign() applications should use OCSP_basic_sign() instead.
For OCSP_REQUEST_verify() applications should use OCSP_request_verify()
instead.
For OCSP_BASICRESP_verify() applications should use OCSP_basic_verify()
instead.

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4635)

(cherry picked from commit 9f5671c7e9f30dfa53b1a2b553f234c2761ceb66)

Consolidate the locations where we have our internal perl modules

Instead of having perl modules under test/testlib and util,
consolidate them all to be inside util/perl.

(this is an adaptation of the part of #4069 that wasn't included in #4666)

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4667)

Perl: Use our own globbing wrapper rather than File::Glob::glob

File::Glob::glob is deprecated, it's use generates this kind of
message:

    File::Glob::glob() will disappear in perl 5.30. Use File::Glob::bsd_glob() instead. at ../master/Configure line 277.

The first idea was to use a construction that makes the caller glob()
use File::Glob::bsd_glob().  That turned out not to work well
everywhere, so instead, we make our own wrapper, OpenSSL::Glob and use
that.

Fixes #4636

(this is an adaptation of #4040 and part of #4069, for 1.1.0)

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4666)

aes/asm/{aes-armv4|bsaes-armv7}.pl: make it work with binutils-2.29.

It's not clear if it's a feature or bug, but binutils-2.29[.1]
interprets 'adr' instruction with Thumb2 code reference differently,
in a way that affects calculation of addresses of constants' tables.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4669)

(cherry picked from commit b82acc3c1a7f304c9df31841753a0fa76b5b3cda)

Spelling doc #3580
Duplicated tests descriptions

Backport of #3580 to 1.1.0
plus a few other typo fixes found at fligth.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4645)

Add error handling in dsa_main and ASN1_i2d_bio.

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4600)

(cherry picked from commit a6f622bc99ffdc7b34199babb9d200b24a7a6431)

Check return value of OBJ_nid2obj in dsa_pub_encode.

CLA: trivial

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4600)

(cherry picked from commit 7760384b403a61824c43cc767a11cd22abfa9e49)

Travis: if "make update" created a diff, please show it

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4652)

(cherry picked from commit d7948767556e68378b75196841b3d32dd70d169a)

Prepare for 1.1.0h-dev

Reviewed-by: Andy Polyakov <appro@openssl.org>

Prepare for 1.1.0g release

Reviewed-by: Andy Polyakov <appro@openssl.org>

Update CHANGES and NEWS for new release

Reviewed-by: Andy Polyakov <appro@openssl.org>

bn/asm/x86_64-mont5.pl: fix carry bug in bn_sqrx8x_internal.

Credit to OSS-Fuzz for finding this.

CVE-2017-3736

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix small but important regression

In OpenSSL pre 1.1.0, 'openssl x509 -CAkeyformat engine' was possible
and supported.  In 1.1.0, a small typo ('F' instead of 'f') removed
that possibility.  This restores the pre 1.1.0 behavior.

Fixes #4366

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4643)

(cherry picked from commit bd6eba79d70677f891f1bb55b6f5bc5602c47cbc)

Address a timing side channel whereby it is possible to determine some

information about the length of the scalar used in ECDSA operations
from a large number (2^32) of signatures.

This doesn't rate as a CVE because:

* For the non-constant time code, there are easier ways to extract
  more information.

* For the constant time code, it requires a significant number of signatures
  to leak a small amount of information.

Thanks to Neals Fournaise, Eliane Jaulmes and Jean-Rene Reinhard for
reporting this issue.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4576)

(cherry picked from commit 4a089bbdf11f9e231cc68f42bba934c954d81a49)

 Address a timing side channel whereby it is possible to determine some

information about the length of a value used in DSA operations from
a large number of signatures.

This doesn't rate as a CVE because:

* For the non-constant time code, there are easier ways to extract
  more information.

* For the constant time code, it requires a significant number of signatures
  to leak a small amount of information.

Thanks to Neals Fournaise, Eliane Jaulmes and Jean-Rene Reinhard for
reporting this issue.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4576)

(cherry picked from commit c0caa945f6ef30363e0d01d75155f20248403df4)

Travis: Add a docs checking job

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4631)

docs: assign section 7 where appropriate

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

doc/crypto/OPENSSL_secure_malloc: add missing names

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

docs: fixup OpenSSL version style

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

Adapt util/find-doc-nits back to 1.1.0

This version was a direct port from 1.1.1-dev, which has a different
source structure for the docs.  Adjustment done.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4633)

Fix EVP_PKEY_ASN1_METHOD manual

Missing names slipped through

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4630)

afalg: Fix kernel version check

The check should reject kernel versions < 4.1.0, not <= 4.1.0.

The issue was spotted on OpenSUSE 42.1 Leap, since its linux/version.h
header advertises 4.1.0.

CLA: trivial
Fixes: 7f458a48 ("ALG: Add AFALG engine")
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4618)

afalg: Use eventfd2 syscall instead of eventfd

The eventfd syscall is deprecated and is not available on aarch64, causing
build to fail:

    engines/e_afalg.c: In function 'eventfd':
    engines/e_afalg.c:108:20: error: '__NR_eventfd' undeclared (first use in this function)
         return syscall(__NR_eventfd, n);
                        ^

Instead, switch to the newer eventfd2 syscall, which is supposed to be
supported by all architectures.

This kind of issues would be avoided by simply using the eventfd(2)
wrapper from the libc, but there must be subtle reasons not to...

Tested on a aarch64 system running OpenSUSE Leap 42.1 (gcc118 from
https://cfarm.tetaneutral.net/machines/list/ ) and also cross-compiling
for aarch64 with LEDE (kernel 4.9).

This properly fixes #1685.

CLA: trivial
Fixes: 7f458a48 ("ALG: Add AFALG engine")
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4618)

EVP_PKEY_asn1_add0(): Check that this method isn't already registered

No two public key ASN.1 methods with the same pkey_id can be
registered at the same time.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4620)

Document EVP_PKEY_ASN1_METHOD and associated functions

[skip ci]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4620)

Only reset the ctx when a cipher is given

This restores the 1.0.2 behaviour

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Benjamin Kaduk <bkaduk@akamai.com>
GH: #4613
(cherry picked from commit ffd23209933ea0ad5543f15ca6303d63d8dac826)

Add missing paren.

Thanks to Remi Gacogne for pointing this out.
Also indented the two macro bodies

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4608)

x509v3/v3_utl.c: avoid double-free.

Thanks to David Benjamin for spotting this.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4532)

(cherry picked from commit 432f8688bb72e21939845ac7a69359ca718c6676)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4514)

crypto/x509v3/v3_utl.c: fix Coverity problems.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4492)

(cherry picked from commit 32f3b98d1302d4c0950dc1bf94b50269b6edbd95)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4514)

Don't use strcasecmp and strncasecmp for IA5 strings

The functions strcasecmp() and strncasecmp() will use locale specific rules
when performing comparison. This could cause some problems in certain
locales. For example in the Turkish locale an 'I' character is not the
uppercase version of 'i'. However IA5 strings should not use locale specific
rules, i.e. for an IA5 string 'I' is uppercase 'i' even if using the
Turkish locale.

This fixes a bug in name constraints checking reported by Thomas Pornin
(NCCGroup).

This is not considered a security issue because it would require both a
Turkish locale (or other locale with similar issues) and malfeasance by
a trusted name-constrained CA for a certificate to pass name constraints
in error. The constraints also have to be for excluded sub-trees which are
extremely rare. Failure to match permitted subtrees is a bug, not a
vulnerability.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4569)

(cherry picked from commit 9cde5f81222fd491d6d56eb8f37ab9c40a26f745)

Fix doc-nits in doc/man3/DEFINE_STACK_OF.pod

<compar> to <compare> to match the var name in function prototype

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4559)

(cherry picked from commit d9c989fe3f137580ee627c91e01245e78b0b41ff)

doc/man3/d2i_X509.pod: add {d2i,i2d}_DSA_PUBKEY in NAME section

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4584)

(cherry picked from commit 82d89ef72515ad3d78c0160641faf30b8b024dda)

asn1_item_embed_new(): if locking failed, don't call asn1_item_embed_free()

asn1_item_embed_free() will try unlocking and fail in this case, and
since the new item was just allocated on the heap, free it directly
with OPENSSL_free() instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4579)

(cherry picked from commit fe6fcd31546db1ab019e55edd15c953c5b358559)

asn1_item_embed_new(): don't free an embedded item

The previous change with this intention didn't quite do it.  An
embedded item must not be freed itself, but might potentially contain
non-embedded elements, which must be freed.

So instead of calling ASN1_item_ex_free(), where we can't pass the
embed flag, we call asn1_item_embed_free() directly.

This changes asn1_item_embed_free() from being a static function to
being a private non-static function.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4579)

(cherry picked from commit 03996c19c30575c48b254f10625d24f86058605b)

Don't make any changes to the lhash structure if we are going to fail

The lhash expand() function can fail if realloc fails. The previous
implementation made changes to the structure and then attempted to do a
realloc. If the realloc failed then it attempted to undo the changes it
had just made. Unfortunately changes to lh->p were not undone correctly,
ultimately causing subsequent expand() calls to increment num_nodes to a
value higher than num_alloc_nodes, which can cause out-of-bounds reads/
writes. This is not considered a security issue because an attacker cannot
cause realloc to fail.

This commit moves the realloc call to near the beginning of the function
before any other changes are made to the lhash structure. That way if a
failure occurs we can immediately fail without having to undo anything.

Thanks to Pavel Kopyl (Samsung) for reporting this issue.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4550)

(cherry picked from commit 4ce8bebcca90a1f8a3347be29df7a501043d4464)

Fix memory leak in GENERAL_NAME_set0_othername.

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4544)

(cherry picked from commit 04761b557a53f026630dd5916b2b8522d94579db)

asn1_item_embed_new(): don't free an embedded item

An embedded item wasn't allocated separately on the heap, so don't
free it as if it was.

Issue discovered by Pavel Kopyl

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4572)

(cherry picked from commit 590bbdfdf43b97abf8817f506f8ab46687d1eadd)

Correct value for BN_security_bits()

The function BN_security_bits() uses the values from SP800-57 to assign
security bit values for different FF key sizes. However the value for 192
security bits is wrong. SP800-57 has it as 7680 but the code had it as
7690.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4546)

(cherry picked from commit c9fe362303fc54ff19bde7511475f28663f7d554)

s390x assembly pack: define OPENSSL_s390xcap_P in s390xcap.c

Remove all .comm definitions from the asm modules.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4563)

ECDSA_* is deprecated. EC_KEY_* is used instead

CLA: trivial

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Ben Laurie <ben@links.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4561)
(cherry picked from commit 9b02dc97e4963969da69675a871dbe80e6d31cda)

Additional name for all commands

Add openssl-foo as a name for the openssl "foo" command.
Addresses an issue found by a usability study to be published.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4553)

(cherry picked from commit 3f2181e6fadea9e7ad8810b3f170fd0b2154e8b8)

s390x assembly pack: remove capability double-checking.

An instruction's QUERY function is executed at initialization, iff the required
MSA level is installed. Therefore, it is sufficient to check the bits returned
by the QUERY functions. The MSA level does not have to be checked at every
function call.
crypto/aes/asm/aes-s390x.pl: The AES key schedule must be computed if the
required KM or KMC function codes are not available. Formally, the availability
of a KMC function code does not imply the availability of the corresponding KM
function code.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4501)

(cherry picked from commit af1d638730bdfad85a7fa8c3f157b2828eda7c1d)

crypto/aes/asm/aes-s390x.pl: fix $softonly=1 code path.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4501)

(cherry picked from commit 4c5100ce7d66ccff48d6435c1761b5e3281de61f)

Update RAND_load_file return value.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4537)

(cherry picked from commit fe7a4d7c4c8148f732bc47ef7585f4aa41b7391a)

Backport key redirection test from master branch

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4520)

Skip ssl-tests/19-mac-then-encrypt.conf for no-tls1_2

The second set of tests in that configuration uses the AES-SHA256
ciphers, which are only available for TLS 1.2.  Thus, when TLS 1.2
is disabled, there are no ciphers available and the handshake fails
with an internal error.  Apply the same treatment as for
13-fragmentation.conf, which uses the same ciphers.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4526)

Document EVP_PKEY_set1_engine()

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

(cherry picked from commit 8e826a339f8cda20a4311fa88a1de782972cf40d)

make update

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)