Enable find-doc-nits to find undocumented symbols since 1.1.1

A previous commit added the ability to find newly undocumented symbols.
We extend this capability to check anything that was newly added since
1.1.1 which is undocumented. A new option -o is added to find-doc-nits
to amend the behaviour of -v or -e to check symbols that were newly
added since the release of 1.1.1.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9094)

Fix an incorrect macro

A macro was missing a space which was confusing find-doc-nits

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9094)

i2d_PublicKey was listed in 2 different man pages

find-doc-nits complains if a symbol is documented in more than one
location.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9094)

Make find-doc-nits check for newly added undocumented symbols

We create lists of undocumented functions and macros as they are now so
that find-doc-nits can check for newly introduced functions/macros that
are undocumented.

This works in a similar way to the -u and -d options to find-doc-nits.
These count undocumented symbols and print a detailed list of undocumented
symbols repsectively. This commit adds the -v and -e options to restrict
the count/detailed list to newly added undocumented symbols only.

There is also a new -s option that does the same as -e except that it
produces no output if there are no newly undocumented symbols.

We also amend "make doc-nits" to add the -s option which should cause
travis to fail if a PR adds undocumented symbols.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9094)

Configure: count basenames for all library sources

Make sure that each basename only appears once.  This is due to the
static library archiver on Unix, that indexes archived object files by
base name only, thereby making base name clashes...  interesting.

This is a safety net for OpenSSL developer!

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9133)

Ensure code is compiled with correct BIGNUM assembler defines

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9130)

Add some dummy BIGNUM calls from inside the FIPS provider

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9130)

Make BIGNUM code available from within the FIPS module

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9130)

Temporarily disable RAND call in FIPS_MODE until RAND is available

Other commits will enable the RAND code in FIPS_MODE. Until those commits
are in place we temporarily disable making RAND calls while in FIPS_MODE.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9130)

Convert bn_rand.c to use EVP

Replace the low level SHA512_* function calls with EVP calls.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9130)

Create BN_CTX_new_ex() and BN_CTX_secure_new_ex()

These variants of BN_CTX_new() and BN_CTX_secure_new() enable passing
an OPENSSL_CTX so that we can access this where needed throughout the
BIGNUM sub library.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9130)

Remove redundant include

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9137)

OSSL_PARAM example code bug fix.

Technically not a bug since the code worked but the array index shouldn't have
been constant after searching for the field.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9136)

Move a fall through comment

When compiling with --strict-warnings using gcc 7.4.0 the compiler
complains that a case falls through, even though there is an explicit
comment stating this. Moving the comment outside of the conditional
compilation section resolves this.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9131)

Fix a URL to the NMBRTHRY list archive

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9123)

Fix app opt compile failure due to missing <inttypes.h>

opt.c uses functions that are only available if inttypes.h exists.
It now checks a define which is unavailable if
inttypes.h is included. The include is done automagically inside e_os2.h.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8986)

Fix Windows Compile failure due to missing <inttypes.h>

Including <inttypes.h> caused a windows build failure.
The test is now skipped if strtoimax & strtoumax are not supported.
It does this by checking for a define which is only available if
inttypes.h is not included. The include is done automagically inside e_os2.h.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8986)

Add defines to indicate if intypes.h and stdint.h are unavailable

Use the defines OPENSSL_NO_INTTYPES_H & OPENSSL_NO_STDINT_H to determine
if the headers are unavailable for a platform.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8986)

PBKDF2 updates to conform to SP800-132

The existing code used PKCS5 specifications.
SP800-132 adds the following additional constraints for:
  - the range of the key length.
  - the minimum iteration count (1000 recommended).
  - salt length (at least 128 bits).
These additional constraints may cause errors (in scrypt, and
some PKCS5 related test vectors). To disable the new
constraints use the new ctrl string "pkcs5".
For backwards compatability, the checks are only enabled by
default for fips mode.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8868)

Make EVP_MD_CTX_ctrl() work for legacy use cases (ssl3).

This is still required currently by engines and digestsign/digestverify.
This PR contains merged in code from Richard Levitte's PR #9126.

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9103)

Fix Keccak structure name reference in S390 legacy code

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9120)

Fix doc to remove const arg in GEN_SESSION_CB

ae3947de095 changed the callback arg not to have a const parameter.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9104)

Change cipher default strings to a function

Making the default cipher strings a function gives the library more
control over the defaults. Potentially allowing a change in the
future as ciphers become deprecated or dangerous.
Also allows third party distributors to change the defaults for their
installations.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8686)

BIO_lookup_ex: Retry with AI_ADDRCONFIG cleared if getaddrinfo fails

The lookup for ::1 with getaddrinfo() might return error even if
the ::1 would work if AI_ADDRCONFIG flag is used.

Fixes: #9053
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9108)

Improve the Windows OneCore target support. (Add targets for building libraries for Windows Store apps.)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8917)

doc/internal/man3/ossl_method_construct.pod: follow common conventions

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9109)

EVP fetching: make operation_id part of the method identity

Because the operation identity wasn't integrated with the created
methods, the following code would give unexpected results:

    EVP_MD *md = EVP_MD_fetch(NULL, "MD5", NULL);
    EVP_CIPHER *cipher = EVP_CIPHER_fetch(NULL, "MD5", NULL);

    if (md != NULL)
        printf("MD5 is a digest\n");
    if (cipher != NULL)
        printf("MD5 is a cipher\n");

The message is that MD5 is both a digest and a cipher.

Partially fixes #9106

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9109)

Revert the DEVRANDOM_WAIT feature

The DEVRANDOM_WAIT feature added a select() call to wait for the
`/dev/random` device to become readable before reading from the
`/dev/urandom` device. It was introduced in commit 38023b87f037
in order to mitigate the fact that the `/dev/urandom` device
does not block until the initial seeding of the kernel CSPRNG
has completed, contrary to the behaviour of the `getrandom()`
system call.

It turned out that this change had negative side effects on
performance which were not acceptable. After some discussion it
was decided to revert this feature and leave it up to the OS
resp. the platform maintainer to ensure a proper initialization
during early boot time.

Fixes #9078

This partially reverts commit 38023b87f037.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9084)

Add documentation for new functions taking an OPENSSL_CTX parameter

Various functions have been added that take an OPENSSL_CTX parameter as
a result of moving the RAND code into the FIPS module. We document all of
those functions.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9039)

Move the rand_nonce_lock code into drbg_lib.c

It was previously rand_lib but it makes more sense in drbg_lib.c since
all the functions that use this lock are only ever called from drbg_lib.c

We add some FIPS_MODE defines in preparation for later moving this code
into the FIPS module.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9039)

Make the rand_crng code OPENSSL_CTX aware

This is in preparation for moving this code inside the FIPS module.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9039)

Convert drbg_lib to use OPENSSL_CTX for its global data

In preparation for moving the RAND code into the FIPS module we make
drbg_lib.c OPENSSL_CTX aware.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9039)

Replace BIO_printf with ASN1_STRING_print in GENERAL_NAME_print

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9101)

Fixed typo in code comment.
[skip ci]

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9102)

Change EVP_MAC method from copy to dup

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
GH: #7651

Replace EVP_MAC_CTX_copy() by EVP_MAC_CTX_dup()

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
GH: #7651

Fix typo in macro argument of SSL_set1_client_sigalgs_list()

Fixes #9092

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9093)

Address property documentation concerns.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9090)

rename the digest provider files to avoid any name clashes with other folders

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9083)

Coverity fixes

covID 1445689 Resource leak (in error path)
covID 1445318 Resource leak (in test - minor)
covID 1443705 Unchecked return value (Needed if CRYPTO_atomic_add() was used)
covID 1443691 Resource leak (in app - minor)

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9071)

man: fix doc-nit in property.pod

This causes travis build failures on master

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9087)

Document property hierarchy.

Add documentation to indicate the interaction between global (context level)
property queries and local (passed to fetch) ones.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9042)

EVP_DigestInit_ex(): drop previous context engine earlier

If a EVP_MD_CTX holds a reference to a previously given engine, and
the type of its digest isn't the same as the one given in the new
call, drop that engine reference, allowing providers or other engines
to provide the new algorithm on an equal basis.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9077)

Document deprecation of version-specific SSL/TLS methods

In commit 2b8fa1d56cd3 the version-specific SSL/TLS methods were
deprecated. This patch improves the documentation of that change
by stating the deprecation more prominently in the manual page
and explaining the reason for the deprecation.

Fixes #8989

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9001)

build_SYS_str_reasons: Fix a crash caused by overlong locales

The 4 kB SPACE_SYS_STR_REASONS in crypto/err/err.c isn't enough for some locales.
The Russian locales consume 6856 bytes, Ukrainian even 7000.

build_SYS_str_reasons() contains an overflow check:

if (cnt > sizeof(strerror_pool))
    cnt = sizeof(strerror_pool);

But since commit 9f15e5b911ba6053e09578f190354568e01c07d7 it no longer
works as cnt is incremented once more after the condition.

cnt greater than sizeof(strerror_pool) results in an unbounded
OPENSSL_strlcpy() in openssl_strerror_r(), eventually causing a crash.

When the first received error string was empty or contained only
spaces, cur would move in front of the start of the strerror_pool.

Also don't call openssl_strerror_r when the pool is full.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/8966)

Remove last references to DEBUG_SAFESTACK

The DEBUG_SAFESTACK preprocessor define is obsolete since 2008
when the non-safestack code was removed by commit 985de8634000.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9070)

Move digests to providers

Move digest code into the relevant providers (fips, default, legacy).
The headers are temporarily moved to be internal, and will be moved
into providers after all external references are resolved. The deprecated
digest code can not be removed until EVP_PKEY (signing) is supported by
providers. EVP_MD data can also not yet be cleaned up for the same reasons.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8763)

Add missing parentheses in macro

Add missing parentheses in macro

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9044)

Really fail if we have a test failure

Commit c5f7a99645a broke the test framework such that some tests might
fail, but the test framework still gives a PASS result overall.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9056)

Ensure we get all the right defines for AES assembler in FIPS module

There are various C macro definitions that are passed via the compiler
to enable AES assembler optimisation. We need to make sure that these
defines are also passed during compilation of the FIPS module.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9038)

Make basic AES ciphers available from within the FIPS providers

These ciphers were already provider aware, and were available from the
default provider. We move them into the FIPS provider too.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9038)

Write a test for receiving a KeyUpdate (update requested) while writing

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8773)

Defer sending a KeyUpdate until after pending writes are complete

If we receive a KeyUpdate message (update requested) from the peer while
we are in the middle of a write, we should defer sending the responding
KeyUpdate message until after the current write is complete. We do this
by waiting to send the KeyUpdate until the next time we write and there is
no pending write data.

This does imply a subtle change in behaviour. Firstly the responding
KeyUpdate message won't be sent straight away as it is now. Secondly if
the peer sends multiple KeyUpdates without us doing any writing then we
will only send one response, as opposed to previously where we sent a
response for each KeyUpdate received.

Fixes #8677

Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/8773)

Remove file reference from property documentation.

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9066)

Add the content type attribute to additional CMS signerinfo.

Fixes #8923

Found using the openssl cms -resign option.
This uses an alternate path to do the signing which was not adding the required signed attribute
content type. The content type attribute should always exist since it is required is there are
any signed attributes.
As the signing time attribute is always added in code, the content type attribute is also required.
The CMS_si_check_attributes() method adds validity checks for signed and unsigned attributes
e.g. The message digest attribute is a signed attribute that must exist if any signed attributes
exist, it cannot be an unsigned attribute and there must only be one instance containing a single
value.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8944)

Fix a typo in the property grammar that creates an ambiguous parse.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9065)

Windows: Add missing quotes in build file

All invokations of $(PERL) need to be quoted, in case it contains
spaces.  That was forgotten in one spot.

Fixes #9060

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9062)

Add sparse array of const pointer type

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/8840)

Fix various mistakes in ec_GFp_nistp_recode_scalar_bits comment.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9050)

Fix CHECK_BSD_STYLE_MACROS for OpenBSD and cryptodev-linux

CLA: trivial

Fixes #8911

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/8866)

Update X509_STORE_new.pod

Minor typo.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/8866)

Constify OSSL_PROVIDER getter input parameters

Some OSSL_PROVIDER getters took a non-const OSSL_PROVIDER parameter.
There's no reason to do so.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9054)

Update EVP_VerifyInit.pod

Method name correction.

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9052)

issue-8998: Ensure that the alert is generated and reaches the remote

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9041)

Certificate Management Protocol (CMP, RFC 4210) extension to OpenSSL
Also includes CRMF (RFC 4211) and HTTP transfer (RFC 6712)

CMP and CRMF API is added to libcrypto, and the "cmp" app to the openssl CLI.
    Adds extensive man pages and tests.  Integration into build scripts.

Incremental pull request based on OpenSSL commit 1362190b1b78 of 2018-09-26

3rd chunk: CMP ASN.1 structures (in crypto/cmp/cmp_asn.c) and related files

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8669)

Add man 7 page about properties.

Add a page about properties in the man7 section of the public documentation.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9011)

Join the x509 and x509v3 directories

This has been long overdue.

Note that this does not join the X509 and X509V3 error modules, that
will be too many macro changes at this stage.

Fixes #8919

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8925)

Fuzz: add a few more types into Fuzzing for ESS

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8117)

CAdES: Fix SignerInfo attribute construction order.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8117)

crypto/conf: openssl_config_int() returns unitialized value

openssl_config_int() returns the uninitialized variable `ret`
when compiled with OPENSSL_SYS_UEFI.

Fixes #9026

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9029)

Typo BIO_SOCK_REUSADDR => BIO_SOCK_REUSEADDR

CLA: trivial

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9021)

Test of uniformity of BN_rand_range output.

Rework the test so that it fails far less often.

A number of independent tests are executed and 5% are expected to fail.
The number of such failures follows a binomial distribution which permits
a statistical test a 0.01% expected failure rate.

There is a command line option to enable the stochastic range checking.
It is off by default.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8830)

fix timeout in 30-test_evp_pkey_dparam

enabling the 'enable-crypto-mdebug' option and running parameter generation
causes timeouts.
Loading pregenerated params is more suited for these tests.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9022)

Configurations/unix-Makefile.tmpl: not -v for rm

Not all Unixen know the -v option

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9012)

Configure: link AIX modules correctly

Use -bnoentry, not -bexpall

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9012)

The SHA256 is not a mandatory digest for DSA.

The #7408 implemented mandatory digest checking in TLS.
However this broke compatibility of DSS support with GnuTLS
which supports only SHA1 with DSS.

There is no reason why SHA256 would be a mandatory digest
for DSA as other digests in SHA family can be used as well.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9015)

Configure: Remove extra warning and sanitizer options from CXXFLAGS

We add the extra warning and sanitizer options to check our code,
which is entirely in C.  We support C++ compilers uniquely for the
sake of certain external test suites, and those projects can probably
sanitize their own code themselves.

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9013)

Add d2i_KeyParams/i2d_KeyParams API's.

Convert EVP_PKEY Parameters to/from binary.
This wraps the low level i2d/d2i calls for DH,DSA and EC key parameters
in a similar way to Public and Private Keys.
The API's can be used by applications (including openssl apps) that only
want to use EVP_PKEY without needing to access low level key API's.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8903)

Changed ssl layer to use EVP_KDF API for TLS1_PRF and HKDF.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9010)

doc/man3/X509_LOOKUP_meth_new.pod: clarify the requirements

The documentation of what a X509_LOOKUP implementation must do was
unclear and confusing.  Most of all, clarification was needed that it
must store away the found objects in the X509_STORE.

Fixes #8707

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8755)

Added Conforming To section to EVP_KDF_TLS1_PRF documentation.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9008)

OCSP: fix memory leak in OCSP_url_svcloc_new method.
Add a few coverage test case.

Fixes #8949

[extended tests]

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8959)

Fix input checks wrt legacy code

In all legacy code ctx->cipher is dereferenced without checks, so it
makes no sense to jump there is ctx->cipher is NULL as it will just lead
to a crash. Catch it separately and return an error.

This is simlar to the fix in d2c2e49eab69c7446c1c2c7227f63f8618ca99a5

Signed-off-by: Simo Sorce <simo@redhat.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9002)

crypto/evp/evp_key.c: #define BUFSIZ if <stdio.h> doesn't #define it

CLA: trivial

Fixes #8904

Commit 48feaceb53fa ("Remove the possibility to disable the UI module
entirely", 2017-07-03) made the BUFSIZ references in "evp_key.c"
unconditional, by deleting the preprocessing directive "#ifndef
OPENSSL_NO_UI". This breaks the build when compiling OpenSSL for edk2
(OPENSSL_SYS_UEFI), because edk2's <stdio.h> doesn't #define BUFSIZ.

Provide a fallback definition, like we do in "crypto/ui/ui_util.c" (from
commit 984d6c605216, "Fix no-stdio build", 2015-09-29).

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8922)

Use fixed length for formatting standard cipher names

Example with patch:
```
$ openssl ciphers -stdname 'TLS_AES_256_GCM_SHA384:ECDHE-ECDSA-CHACHA20-POLY1305'
TLS_AES_256_GCM_SHA384                        - TLS_AES_256_GCM_SHA384         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256                  - TLS_CHACHA20_POLY1305_SHA256   TLSv1.3 Kx=any      Au=any   Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256                        - TLS_AES_128_GCM_SHA256         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(128) Mac=AEAD
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 - ECDHE-ECDSA-CHACHA20-POLY1305  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
```

Example without patch:
```
$ openssl ciphers -stdname 'TLS_AES_256_GCM_SHA384:ECDHE-ECDSA-CHACHA20-POLY1305'
TLS_AES_256_GCM_SHA384 - TLS_AES_256_GCM_SHA384  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 - TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any      Au=any  Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256 - TLS_AES_128_GCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(128) Mac=AEAD
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 - ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
```

CLA: Trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8999)

Update format string for ciphers to account for newer ciphers

* Cipher name: from 23 to 30 (example: ECDHE-ECDSA-AES128-GCM-SHA256)
* Fixed length for TLS version (examples: TLSv1, TLSv1.3)
* Au length from 4 to 5 (example: ECDSA)

Example (without patch):
```
$ openssl ciphers -v 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA'
TLS_AES_256_GCM_SHA384  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any      Au=any  Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA1
```

Example (with patch):
```
$ openssl ciphers -v 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA'
TLS_AES_256_GCM_SHA384         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256   TLSv1.3 Kx=any      Au=any   Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-SHA         TLSv1   Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA1
```

CLA: trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8999)

issue-8973: Added const to parameters for values that were not altered

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8995)

Doc update: minor typo in CMS_verify.pod
CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8952)

Cleaned up tls1_prf and added comments.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8987)

Added algorithm description comments to HKDF.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8994)

Fix a crash in the speed command with wrap ciphers

e.g. openssl speed -evp id-aes256-wrap-pad
was crashing because the return code from EVP_CipherInit_ex
was ignored.
Not going to allow that cipher mode because wrap ciphers
produces more bytes output than the input length
and EVP_Update_loop is not really prepared for that.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8739)

Add more commentary about recursive Provider intialisation in the FIPS module

In addition this commit ensures that the "provctx" value is defaulted to the current
library context when we are recurively initialising the FIPS provider when already inside
the FIPS module.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8728)

Move where include path for providers/common/include gets specified

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8728)

Make some EVP code available from within the FIPS module

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8728)

Make core code available within the FIPS module

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8728)

Add the NIST CAVS test vectors for CCM

This imports all of the NIST CAVS test vectors for CCM (SP800-38C) and
coverts them for use within evp_test. This commit also adds a script to
convert the .rsp CAVS files into the evp_test format.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8981)

Avoid trailing space in "openssl version -o".

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8963)

Always show application and library versions in "openssl version".

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8963)

Fix GOST OID

CLA: trivial

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8979)

EVP_Digest*: enable SHA3 pre-hashing for DSA

Fixes #7323

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8970)