apps: when the 'compat' nameopt has been set, leave it be

XN_FLAG_COMPAT has a unique property, its zero for value.  This means
it needs special treatment; if it has been set (which can only be
determined indirectly) and set alone (*), no other flags should be
set.

(*) if any other nameopt flag has been set by the user, compatibility
mode is blown away.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6382)

(cherry picked from commit 3190d1dca43ecfd748c06aa06752de06af3768b9)

Add APIs for custom X509_LOOKUP_METHOD creation

OpenSSL 1.1.0 made the X509_LOOKUP_METHOD structure opaque, so
applications that were previously able to define a custom lookup method
are not able to be ported.

This commit adds getters and setters for each of the current fields of
X509_LOOKUP_METHOD, along with getters and setters on several associated
opaque types (such as X509_LOOKUP and X509_OBJECT).

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6152)

(cherry picked from commit 0124f32a01b2b4f4f7146f226b6a9dfe227c4008)

The result of a ^ 0 mod -1 is 0 not 1

Thanks to Guido Vranken and OSSFuzz for finding this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6355)

(cherry picked from commit 4aa5b725d549b3ebc3a4f2f1c44e44a11f68752b)

Try to work around ubuntu gcc-5 ubsan build failure

[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6362)

Save and restore the Windows error around TlsGetValue.

TlsGetValue clears the last error even on success, so that callers may
distinguish it successfully returning NULL or failing. This error-mangling
behavior interferes with the caller's use of GetLastError. In particular
SSL_get_error queries the error queue to determine whether the caller should
look at the OS's errors. To avoid destroying state, save and restore the
Windows error.

Fixes #6299.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 2de108dfa343c3e06eb98beb122cd06306bb12fd)

(Merged from https://github.com/openssl/openssl/pull/6349)

Improve compatibility of point and curve checks

We check that the curve name associated with the point is the same as that
for the curve.

Fixes #6302

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6323)

(cherry picked from commit b14e60155009f4f1d168e220fa01cd2b75557b72)

Skip CN DNS name constraint checks when not needed

Only check the CN against DNS name contraints if the
`X509_CHECK_FLAG_NEVER_CHECK_SUBJECT` flag is not set, and either the
certificate has no DNS subject alternative names or the
`X509_CHECK_FLAG_ALWAYS_CHECK_SUBJECT` flag is set.

Add pertinent documentation, and touch up some stale text about
name checks and DANE.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>

Limit scope of CN name constraints

Don't apply DNS name constraints to the subject CN when there's a
least one DNS-ID subjectAlternativeName.

Don't apply DNS name constraints to subject CN's that are sufficiently
unlike DNS names.  Checked name must have at least two labels, with
all labels non-empty, no trailing '.' and all hyphens must be
internal in each label.  In addition to the usual LDH characters,
we also allow "_", since some sites use these for hostnames despite
all the standards.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>

ssl/ssl_txt: fix NULL-check

NULL-check for cipher is redundant, instead check if cipher->name is NULL

While here fix formatting of BIO_printf calls as suggested by Andy Polyakov.

CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6282)

(cherry picked from commit d61e6040a02464f1dd41942ee1e17ef59822102d)

Fix undefined behaviour in X509_NAME_cmp()

If the lengths of both names is 0 then don't attempt to do a memcmp.

Issue reported by Simon Friedberger, Robert Merget and Juraj Somorovsky.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6291)

(cherry picked from commit 511190b691183a1fb160e7e05e2974dc73cab0c6)

Windows: don't install __DECC_*.H

This adds the possibility to exclude files by regexp in util/copy.pl

Partial fix for #3254

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6303)

(cherry picked from commit 246bd8fd0507f4555432c148eed5a9322c113bf5)

Quiet pod2html warnings

--quiet stops warnings of this sort:

    Cannot find "BIO_read_ex" in podpath: cannot find suitable replacement path, cannot resolve link

We know what causes these warnings, it's perfectly innocuous, and we
don't want to hear it any more.

Partial fix for #3254

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6304)

(cherry picked from commit 6439e343fa64f06941197d085acd11bd13856596)

Restore check of |*xn| against |name| in X509_NAME_set

A previous change of this function introduced a fragility when the
destination happens to be the same as the source.  Such alias isn't
recommended, but could still happen, for example in this kind of code:

    X509_NAME *subject = X509_get_issuer_name(x);

    /* ... some code passes ... */

    X509_set_issuer_name(x, subject);

Fixes #4710

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6280)

(cherry picked from commit c1c1783d45a5e91951e6328a820939d0256c841c)

Fix a possible crash in BN_from_montgomery_word

Thanks to Darovskikh Andrei for for reporting this issue.

Fixes: #5785
Fixes: #6302
Cherry-picked from f91e026e3832 (without test/bntest.c)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6310)

apps/speed: fix possible OOB access in some EC arrays

because there are actually 17 curves defined, but only 16 are plugged for
ecdsa test.
Deduce array size using OSSL_NELEM and so remove various magic numbers,
which required some declarations moving.
Implement OPT_PAIR list search without a null-ending element.
Fix some comparison between signed and unsigned integer expressions.

cherry-picking from commit 5c6a69f539.

Partial Back-port of #6133 to 1.1.0

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6245)

Make BN_GF2m_mod_arr more constant time

Experiments have shown that the lookup table used by BN_GF2m_mod_arr
introduces sufficient timing signal to recover the private key for an
attacker with access to cache timing information on the victim's host.
This only affects binary curves (which are less frequently used).

No CVE is considered necessary for this issue.

The fix is to replace the lookup table with an on-the-fly calculation of
the value from the table instead, which can be performed in constant time.

Thanks to Youngjoo Shin for reporting this issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6270)

(cherry picked from commit b336ce57f2d5cca803a920d2a9e622b588cead3c)

Add a note on CHANGES and NEWS in CONTRIBUTING

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6249)

(cherry picked from commit 029c11c21fdd018ec51badaafd34118223055274)

When producing man-pages, ensure NAME section is one line only

There are *roff parsers that are strict about the NAME section being
one line only.  The man(7) on Debian GNU/Linux suggests that this is
appropriate, so we compensate our multi-line NAME sections by fixing
the *roff output.

Noted by Eric S. Raymond

Related to #6264

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6268)

(cherry picked from commit 8d483b2de78619e8592f2558301f3295daf59690)

Mark DTLS records as read when we have finished with them

The TLS code marks records as read when its finished using a record. The DTLS code did
not do that. However SSL_has_pending() relies on it. So we should make DTLS consistent.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6160)

CI config: no need to make both install and install_docs

'install' depends on 'install_docs', so making the latter explicit is
a waste.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6250)

(cherry picked from commit 986caf9e34fd60366a5b3711bb12e239e43a2431)

UI console: Restore tty settings, do not force ECHO after prompt

The Console UI method always set echo on after prompting without
echo.  However, echo might not have been on originally, so just
restore the original TTY settings.

Fixes #2373

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6157)

Fix memory leaks in CA related functions.

(cherry picked from commit aebd0e5ca12d1ba0b229a4121a54afa5ea2d8aa1)

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6237)

Don't memcpy the contents of an empty fragment

In DTLS if we have buffered a fragment for a zero length message (e.g.
ServerHelloDone) then, when we unbuffered the fragment, we were attempting
to memcpy the contents of the fragment which is zero length and a NULL
pointer. This is undefined behaviour. We should check first whether we
have a zero length fragment.

Fixes a travis issue.

[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6224)

In cases where we ask PEM_def_callback for minimum 0 length, accept 0 length

Fixes #4716

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6173)

(cherry picked from commit c82c3462267afdbbaa53e11da0508ce4e03c02b3)

Fix typo 'is an error occurred' in documentation

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6230)

Keep the DTLS timer running after the end of the handshake if appropriate

During a full handshake the server is the last one to "speak". The timer
should continue to run until we know that the client has received our last
flight (e.g. because we receive some application data).

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6196)

Only auto-retry for DTLS if configured to do so

Otherwise we may end up in a hang when using blocking sockets

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6196)

Fix s_client and s_server so that they correctly handle the DTLS timer

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6196)

Don't fail on an out-of-order CCS in DTLS

Fixes #4929

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6196)

Fix a mem leak in CMS

The function CMS_RecipientInfo_set0_pkey() is a "set0" and therefore
memory management passes to OpenSSL. If the same function is called again
then we should ensure that any previous value that was set is freed first
before we set it again.

Fixes #5052

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6142)

(cherry picked from commit 3d551b20df1acd01f80d3ae00d37177e0fdf344a)

X509 time: tighten validation per RFC 5280

- Reject fractional seconds
- Reject offsets
- Check that the date/time digits are in valid range.
- Add documentation for X509_cmp_time

GH issue 2620

Backported from 80770da39e

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6181)

Configure: fix Mac OS X builds that still require makedepend

Earlier Apple Xcode compilers, e.g. one targeting Mac OS X 10.7, don't
support dependency generation and one still has to use makedepend.  It's
unclear when it was fixed, but all clang-based Apple compilers seem to
support -M options.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6167)

BIO_s_mem() write: Skip early when input length is zero

When the input length is zero, just return zero early.  Otherwise,
there's a small chance that memory allocation is engaged, fails and
returns -1, which is a bit confusing when nothing should be written.

Fixes #4782 #4827

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/6175)

(cherry picked from commit 0d94212a046e87fafea6e83e8ea2b2a58db49979)

docs: Fix typo EVP_PKEY_new_id -> EVP_PKEY_CTX_new_id

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6178)

bn/asm/*-mont.pl: harmonize with BN_from_montgomery_word.

Montgomery multiplication post-conditions in some of code paths were
formally non-constant time. Cache access pattern was result-neutral,
but a little bit asymmetric, which might have produced a signal [if
processor reordered load and stores at run-time].

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6163)

(cherry picked from commit 774ff8fed67e19d4f5f0df2f59050f2737abab2a)

Resolved conflicts in ppc-mont.pl and x86_64-mont.pl.

v3_purp.c: add locking to x509v3_cache_extensions()

Fixes #6121

Thanks to Mingtao Yang for reporting this bug.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6162)

Return an error from BN_mod_inverse if n is 1 (or -1)

Calculating BN_mod_inverse where n is 1 (or -1) doesn't make sense. We
should return an error in that case. Instead we were returning a valid
result with value 0.

Fixes #6004

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6119)

(cherry picked from commit b1860d6c71733314417d053a72af66ae72e8268e)

Make X509_VERIFY_PARAM_get_hostflags() take a const arg

Commit 14e55560 added this function which should have taken a const
argument.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6155)

Add some documentation for SSL_get_shared_ciphers()

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6114)

Fix comment in ssl_locl.h

The ciphers field in a session contains the stack of ciphers offered by
the client.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6114)

Fix SSL_get_shared_ciphers()

The function SSL_get_shared_ciphers() is supposed to return ciphers shared
by the client and the server. However it only ever returned the client
ciphers.

Fixes #5317

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6114)

a_strex.c: prevent out of bound read in do_buf()

which is used for ASN1_STRING_print_ex*() and X509_NAME_print_ex*().

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6117)

Fix regression with session cache use by clients

Commit d316cdcf6d8d6934663278145fe0a8191e14a8c5 introduced some extra
checks into the session-cache update procedure, intended to prevent
the caching of sessions whose resumption would lead to a handshake
failure, since if the server is authenticating the client, there needs to
be an application-set "session id context" to match up to the authentication
context.  While that change is effective for its stated purpose, there
was also some collatoral damage introduced along with the fix -- clients
that set SSL_VERIFY_PEER are not expected to set an sid_ctx, and so
their usage of session caching was erroneously denied.

Fix the scope of the original commit by limiting it to only acting
when the SSL is a server SSL.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5967)

(cherry picked from commit c4fa1f7fc016919a5b3d4ea2aa66c77e0cc40c9d)

Add getter for X509_VERIFY_PARAM_get_hostflags

Fixes #5061

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6140)

Clarify BN_mod_exp docs

Specifically this is not supported with an even modulus and
BN_FLG_CONSTTIME.

Fixes #5082

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6138)

Fix some errors and missing info in the CMS docs

Fixes #5063

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6135)

Update version docs

Make it clear that you should not attempt to get the version before the
first handshake is complete.

Fixes #2893

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6014)

apps/s_server.c: Avoid unused variable due to 'no-dtls'

Fixes #6098

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6104)

(cherry picked from commit ff25dd1a47525c88500dd61cdeb27faaea6ee599)

fix: BN_swap mishandles flags (1.1.0)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6101)

Fix memleaks in async api

Fixes: #5950
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6038)

(cherry picked from commit 74a8acbdfb2c7f398d1ae2fe914cd32b437f6df4)

Update the *use_certificate* docs

Note that calling the *use_certificate* functions will replace any existing
certificate of the same type. The same thing applies for private keys.

Fixes #2147

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6093)

Fix no-ui

In a recent commit, a OPENSSL_NO_UI guard was dropped by mistake in
PEM_def_callback(), so this puts it back.  At the same time, that
previous commit got rid of the necessity to have a OPENSSL_NO_STDIO
guard, so we drop that instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6096)

Use  get_last_sys_error() instead of get_last_rtl_error()

get_last_sys_error() already exists, so there's no need for yet
another macro that fulfills the same purpose.

Fixes #4120

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6088)

(cherry picked from commit b4c35656441929ce6f6be112648d637cba51dc91)

PEM_def_callback(): use same parameter names as for pem_password_cb

Add a bit more commentary to explain what's going on.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6080)

(cherry picked from commit d6d94d339756332bbabe2a1032ac511ae31b3fdc)

PEM_def_callback(): don't loop because of too short password given

That error is already caught by EVP_read_pw_string_min, and causes
this function to return -1, so the code detecting too short passwords
in this function is practically dead.

Fixes #5465

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6080)

(cherry picked from commit 4977b4e9281c981efcf6a8b31378b8bbd6a8a96f)

openssl rehash: exit 0 on warnings, same as c_rehash

Fixes #6083

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6084)

(cherry picked from commit e6a833cb97ed762408b57ea3efa83bd10c1d2a78)

ms/uplink-x86.pl: close the file handle that was opened

Fixes #5656

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6079)

(cherry picked from commit 3625f6c977beccd7bb0d80a1a670c19f4338a27a)

Fix documentation for the -showcerts s_client option

This option shows the certificates as sent by the server. It is not the
full verified chain.

Fixes #4933

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6068)

Fix the MAX_CURVELIST definition

The MAX_CURVELIST macro defines the total number of in-built SSL/TLS curves
that we support. However it has not been updated as new curves are added.

Fixes #5232

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6065)

(cherry picked from commit ca50cd911ca3c9dc9ec8dd956f8eb45557585a98)

In a reneg use the same client_version we used last time

In 1.0.2 and below we always send the same client_version in a reneg
ClientHello that we sent the first time around, regardless of what
version eventually gets negotiated. According to a comment in
statem_clnt.c this is a workaround for some buggy servers that choked if
we changed the version used in the RSA encrypted premaster secret.

In 1.1.0+ this behaviour no longer occurs. This restores the original
behaviour.

Fixes #1651

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6060)

Document when a session gets removed from cache

Document the fact that if a session is not closed down cleanly then the
session gets removed from the cache and marked as non-resumable.

Fixes #4720

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6054)

Allow intermediate CAs to use RSA PSS in 1.1.0

In 1.1.0 and above we check the digest algorithm used to create signatures
in intermediate CA certs. If it is not sufficiently strong then we reject
the cert. To work out what digest was used we look at the OID for the
signature. This works for most signatures, but not for RSA PSS where the
digest is stored as parameter of the SignatureAlgorithmIdentifier. This
results in the digest look up routines failing and the cert being rejected.

PR #3301 added support for doing this properly in master. So in that
branch this all works as expected. It also works properly in 1.0.2 where we
don't have the digest checks at all. So the only branch where this fails is
1.1.0.

PR #3301 seems too significant to backport to 1.1.0. Instead we simply skip
the signature digest algorithm strength checks if we detect RSA PSS.

Fixes #3558.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6052)

Improve backwards compat with 1.0.2 for ECDHParameters

In 1.0.2 you could configure automatic ecdh params by using the
ECDHParameters config directive and setting it to the value
"+Automatic" or just "Automatic". This is no longer required in 1.1.0+
but we still recognise the "+Automatic" keyword for backwards compatibility.
However we did not recognise just "Automatic" without the leading "+" which
is equally valid. This commit fixes that omission.

Fixes #4113

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6036)

Remove superfluous NULL checks. Add Andy's BN_FLG comment.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 39df51522ba2e3773ae2f1d4df5a6031ef41c1ba)

Move up check for EC_R_INCOMPATIBLE_OBJECTS and for the point at infinity case

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 736b31e5ea33166d89d5cff5774697d0c15d96bd)

Pass through

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit f4675379275c304dbfa593cc573b4e4c4eb54bd4)

ladder description: why it works

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit a067a8705a654c85d43b942e0d1616e282667969)

Address code style comments

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 36bed230b580f92d2e10d13e4ba472236e622562)

Elliptic curve scalar multiplication with timing attack defenses

Co-authored-by: Nicola Tuveri <nic.tuv@gmail.com>
Co-authored-by: Cesar Pereida Garcia <cesar.pereidagarcia@tut.fi>
Co-authored-by: Sohaib ul Hassan <soh.19.hassan@gmail.com>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6009)

(cherry picked from commit 40e48e54582e46c1a01e184ecf5bd31f4f7f8294)

ARM assembly pack: make it work with older assembler.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6043)

(cherry picked from commit 198a2ed791e8f4f00d0b92272987f564ca1d9783)

Fix openssl ca, to correctly make output file binary when using -spkac

On Unix, this doesn't matter, but on other platforms, it may.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6050)

(cherry picked from commit 25642ad29e6a2c15c10ceb5e4f029638f73a879e)

Fix late opening of output file

For 'openssl dhparams', the output file was opened after calculations
were made, which is a waste of cycles and time if the output file
turns out not to be writable.

Fixes #3404

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6051)

(cherry picked from commit 10b37541dc7f27b0ca74102970691ce4636cc0bd)

Ensure the thread keys are always allocated in the same order

Back-port of #5911
Fixes: #5899
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6037)

Fix a warning about missing prototype on arm

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6028)

Fix a gcc-8 warning -Wcast-function-type

Casting to the generic function type "void (*)(void)"
prevents the warning.

Backport of #5816

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6027)

Fix building linux-armv4 with --strict-warnings

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6026)

(cherry picked from commit 0e0f8116e247912f5c48f8b3786e543f37fc1f87)

Add a test for SSL_pending()

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6021)

Fix SSL_pending() for DTLS

DTLS was not correctly returning the number of pending bytes left in
a call to SSL_pending(). This makes the detection of truncated packets
almost impossible.

Fixes #5478

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6021)

Fix the alert sent if no shared sig algs

We were sending illegal parameter. This isn't correct. The parameters are
legal, we just don't have an overlap. A more appropriate alert is
handshake failure.

Fixes #2919

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6010)

Don't distribute team internal config targets

Configurations/90-team.conf isn't for public consumption, so we rename
it to 90-team.norelease.conf and make sure 'make dist' and 'make tar'
don't include it in the tarball.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6016)

Add a test for a NULL X509_STORE in X509_STORE_CTX_init

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6002)

Don't crash if there are no trusted certs

The X509_STORE_CTX_init() docs explicitly allow a NULL parameter for the
X509_STORE. Therefore we shouldn't crash if we subsequently call
X509_verify_cert() and no X509_STORE has been set.

Fixes #2462

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6002)

Fix ocsp app exit code

If we run the ocsp command line app and the responder returns a
non-successful status code then the app should exit with a failure code.

Based on an original patch by Tatsuhiro Tsujikawa.

Fixes #2387

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5999)

X509_cmp_time: only return 1, 0, -1.

The behaviour of X509_cmp_time used to be undocumented.

The new behaviour, documented in master, is to return only 0, 1, or -1.
Make the code in the other branches to adhere to this behaviour too,
to reduce confusion. There is nothing to be gained from returning
other values.

Fixes GH#4954

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4955)

Update EVP_DigestSignInit() docs

Explicitly state which digests can be used with which algorithms.

Fixes #5854

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5995)

poly1305/asm/poly1305-armv4.pl: remove unintentional relocation.

Branch to global symbol results in reference to PLT, and when compiling
for THUMB-2 - in a R_ARM_THM_JUMP19 relocation. Some linkers don't
support this relocation (ld.gold), while others can end up truncating
the relocation to fit (ld.bfd).

Convert this branch through PLT into a direct branch that the assembler
can resolve locally.

See https://github.com/android-ndk/ndk/issues/337 for background.

The current workaround is to disable poly1305 optimization assembly,
which is not optimal and can be reverted after this patch:
https://github.com/freedesktop/gstreamer-cerbero/commit/beab607d2b1ff23c41b7e01aa9c64be5e247d1e6

CLA: trivial

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5949)

(cherry picked from commit 5bb1cd2292b388263a0cc05392bb99141212aa53)

Check the return from EVP_PKEY_get0_DH()

Fixes #5934

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5993)

Fix assertion failure in SSL_set_bio()

If SSL_set_bio() is called with a NULL wbio after a failed connection then
this can trigger an assertion failure. This should be valid behaviour and
the assertion is in fact invalid and can simply be removed.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5966)

(cherry picked from commit bd7775e14a19c326d3720f2345c2ae324409e979)

Update fingerprints.txt

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5987)

(cherry picked from commit 2bdeffefddd8e8a65a51a7b020f8d51a4a3b1602)

Ignore the status_request extension in a resumption handshake

We cannot provide a certificate status on a resumption so we should
ignore this extension in that case.

Fixes #1662

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5897)

p5_scrypt.c: fix error check of RAND_bytes() call

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/5977)

Revert "Add OPENSSL_VERSION_AT_LEAST"

Fixes #5961

This reverts commit d8adfdcd2e5de23f3e1d1a1d10c2fda6f4a65c4a.

The macros OPENSSL_MAKE_VERSION() and OPENSSL_VERSION_AT_LEAST() contain
errors and don't work as designed. Apart from that, their introduction
should be held back until a decision has been mad about the future
versioning scheme.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5968)

RSA key generation: ensure BN_mod_inverse and BN_mod_exp_mont both get called with BN_FLG_CONSTTIME flag set.

CVE-2018-0737

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Custome built dladdr() for AIX.

Implemented a stripped down dladdr()-implementation using AIX' own
loadquery()-function. Following the SGI example in the same code, the
DL_info only has the dli_fname member. As the scope of
dlfcn_pathbyaddr() is the filename, this implementation does not
consider archive members, which can be dlopen()ed in AIX.
Added DATA segment checking to catch ptrgl virtual addresses.

Added test case for DSO_dsobyaddr(), but only for DSO_DLFCN.
Added PIC-flag to aix*-cc build targets.

Signed-off-by: Matthias Kraft <makr@gmx.eu>
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5626)

test/recipes/test_genrsa.t : don't fail because of size limit changes

There is a test to check that 'genrsa' doesn't accept absurdly low
number of bits.  Apart from that, this test is designed to check the
working functionality of 'openssl genrsa', so instead of having a hard
coded lower limit on the size key, let's figure out what it is.

Partially fixes #5751

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5754)

Docs for OpenSSL_init_crypto: there is no way to specify another file

The documentation erroneously stated that one can change the default
configuration file name.

Fixes #5939

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5941)

(cherry picked from commit 0320e8e2869fb6cde4579375e65f6d576bbec95e)

openssl rehash: use libcrypto variables for default dir

X509_get_default_cert_dir_env() returns the default environment
variable to check for certificate directories.
X509_get_default_cert_dir() returns the default configured certificate
directory.

Use these instead of hard coding our own values, and thereby be more
integrated with the rest of OpenSSL.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5937)

(cherry picked from commit 362ff3c39703633ac53ba9130c7acc02177bedc5)

openssl rehash: document -compat

Fixes #5902

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5937)

(cherry picked from commit adaf3cfad2319fe5c0d6fe179c726cc85eebf2d3)

Clarify default section in config.pod

This is a minor update which hopefully makes these particular lines
read a little easier.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5938)

(cherry picked from commit 0652e8a7fe6dd8cbdf4516b221642b10bbfc68fe)

Fix bugs in X509_NAME_ENTRY_set

The wrong "set" field was incremented in the wrong place and would
create a new RDN, not a multi-valued RDN.
RDN inserts would happen after not before.
Prepending an entry to an RDN incorrectly created a new RDN

Anything which built up an X509_NAME could get a messed-up structure,
which would then be "wrong" for anyone using that name.

Thanks to Ingo Schwarze for extensive debugging and the initial
fix (documented in GitHub issue #5870).

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5882)

(cherry picked from commit bbf27cd58337116c57a1c942153330ff83d5540a)