Fix bogus check for EVP_PKEY_supports_digest_nid() in check_cert_usable()

In commit 2d263a4a73 ("Honour mandatory digest on private key in
has_usable_cert()" I added two checks for the capabilities of the
EVP_PKEY being used. One of them was wrong, as it should only be
checking the signature of the X.509 cert (by its issuer) against the
sigalgs given in a TLS v1.3 signature_algorithms_cert extension.

Remove it and provide the code comments which, if they'd been present
in the first place, would hopefully have prevented the mistake.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9672)

testing: set OPENSSL_MODULES to the providers directory by default

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9618)

Fix SCA vulnerability when using PVK and MSBLOB key formats

This commit addresses a side-channel vulnerability present when
PVK and MSBLOB key formats are loaded into OpenSSL.
The public key was not computed using a constant-time exponentiation
function.

This issue was discovered and reported by the NISEC group at TAU Finland.

Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9587)

Do not have duplicate section heads

Change find-doc-nits to complain if a section header is repeated,
within a parent header (i.e., duplicate =head2 within a =head1).
In almost all cases, we just remove the duplicate header, as
it was a "continuation" of the =head1 that was already in affect.
In some cases, just remove "=head1 NOTES", possibly moving text
around, because the "NOTES" were really important parts of the
DESCRIPTION section.

No =headX sections should end with a period.
All =head1 labels should be in all uppercase.
No sub-head (=head2, etc) should be in all uppercase.
Update find-doc-nits to reject the above.

Fixup an internal POD link

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9631)

Include mac_meth and mac_lib in the FIPS provider

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9694)

Cleanup ciphers and Add 3des ciphers.

Moved the relevant ciphers into default and restructed headers to allow the move.
This removed most of the cases of #ifdef NO_XXX (which are now specified in build.info)

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9482)

Fix Issue OSS-Fuzz: Branch on uninitialized memory (in ccm code).

This would also happen for aes-ccm. There was one branch path where it just returned 1
without setting *padlen, It now branches so that the value is set to 0.

Fixes #9691

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9692)

Update EVP_MAC.pod

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9667)

Get rid of the diversity of names for MAC parameters

The EVP_PKEY MAC implementations had a diversity of controls that were
really the same thing.  We did reproduce that for the provider based
MACs, but are changing our minds on this.  Instead of that, we now use
one parameter name for passing the name of the underlying ciphers or
digests to a MAC implementation, "cipher" and "digest", and one
parameter name for passing the output size of the MAC, "size".

Then we leave it to the EVP_PKEY->EVP_MAC bridge to translate "md"
to "digest", and "digestsize" to "size".

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9667)

Remove MAC cruft

A few declarations and static functions / arrays that are no longer
used were left behind when MACs moved to be implemented by providers.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9667)

Add app for fips installation

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9634)

Avoid overflowing FDSET when using select(2).

There is a problem in the rand_unix.c code when the random seed fd is greater
than or equal to FD_SETSIZE and the FDSET overruns its limit and walks the
stack.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9686)

test/evp_test.c: distinguish parsing errors from processing errors

Parsing functions are at liberty to return:

1:  when parsing on processing of the parsed value succeeded
0:  when the parsed keyword is unknown
-1: when the parsed value processing failed

Some parsing functions didn't do this quite right, they returned 0
when they should have returned -1, causing a message like this:

    Line 123: unknown keyword PeerKey

When this message (which is displayed when the parsing function
returns -1) would have been more appropriate:

    Line 123: error processing keyword PeerKey = ffdhe2048-2-pub

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9682)

crypto/bn/build.info: define OPENSL_IA32_SSE2 globally when needed

This is done conditionally.

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9679)

Configure: Allow 'DEFINE[]=def'

DEFINE[] definitions end up pushed in @{$config{defines}} instead of
being added to the output file list of defines.  This allows for the
unusual case where we need something to be defined globally, so it
gets picked up by anything using $(CPPFLAGS).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9679)

Deprecate unprefixed manual entries for openssl commands

Initially, the manual page entry for the 'openssl cmd' command used
to be available at 'cmd(1)'. Later, the aliases 'openssl-cmd(1)' was
introduced, which made it easier to group the openssl commands using
the 'apropos(1)' command or the shell's tab completion.

In order to reduce cluttering of the global manual page namespace,
the manual page entries without the 'openssl-' prefix have been
deprecated in OpenSSL 3.0 and will be removed in OpenSSL 4.0.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9666)

openssl dgst, openssl enc: check for end of input

The input reading loop in 'openssl dgst' and 'openssl enc' doesn't
check for end of input, and because of the way BIO works, it thereby
won't detect that the end is reached before the read is an error.
With the FILE BIO, an error occurs when trying to read past EOF, which
is fairly much ok, except when the command is used interactively, at
least on Unix.  The result in that case is that the user has to press
Ctrl-D twice for the command to terminate.

The issue is further complicated because both these commands use
filter BIOs on top of the FILE BIO, so a naïve attempt to check
BIO_eof() doesn't quite solve it, since that only checks the state of
the source/sink BIO, and the filter BIO may have some buffered data
that still needs to be read.  Fortunately, there's BIO_pending() that
checks exactly that, if any filter BIO has pending data that needs to
be processed.

We end up having to check both BIO_pending() and BIO_eof().

Thanks to Zsigmond Lőrinczy for the initial effort and inspiration.

Fixes #9355

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9668)

Support parsing of SM2 ID in hexdecimal

The current EVP_PEKY_ctrl for SM2 has no capability of parsing an ID
input in hexdecimal.

The newly added ctrl string is called: sm2_hex_id

Test cases and documentation are updated.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9584)

Add basic aria and camellia ciphers modes to default provider

The aes code has been refactored into generic and algorithn specific parts,
so that most of the code can be shared.
The cipher related files have been broken up into smaller parts.
Add chunked variant of mode ciphers - aria uses this (many other ciphers will use this new code instead of the
generic code used by aes).

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9451)

Fix drbg_ossl_ctx_free() and drbg_nonce_ossl_ctx_free() to handle NULL

If these were passed NULL, the crashed with a SIGSEGV, when they
should do like all other freeing functions and become a no-op.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9650)

Modify ossl_method_store_add() to accept an OSSL_PROVIDER and check for it

If ossl_method_store_add() gets called with a method that already exists
(i.e. the store has one with matching provider, nid and properties), that
method should not be stored.  We do this check inside ossl_method_store_add()
because it has all the locking required to do so safely.

Fixes #9561

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9650)

Modify ossl_method_store_add() to handle reference counting

Because this function affects the reference count on failure (the call
to impl_free() does this), it may as well handle incrementing it as
well to indicate the extra reference in the method store.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9650)

Fix ossl_param_bld_push_{utf8,octet}_string() / param_bld_convert()

ossl_param_bld_push_{utf8,octet}_string() saved the constant string
pointer to a non-constant structure field, so we change that field to
a pointer to a constant.  We also modify param_bld_convert() to
pretend the resulting pointer for PTR types points to a constant as
well.

Completes #9649

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9651)

Constify param builder string functions.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9649)

Correct documented return value for BIO_get_mem_data()

CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9643)

Start up DEVRANDOM entropy improvement for older Linux devices.

Improve handling of low entropy at start up from /dev/urandom by waiting for
a read(2) call on /dev/random to succeed.  Once one such call has succeeded,
a shared memory segment is created and persisted as an indicator to other
processes that /dev/urandom is properly seeded.

This does not fully prevent against attacks weakening the entropy source.
An attacker who has control of the machine early in its boot sequence
could create the shared memory segment preventing detection of low entropy
conditions.  However, this is no worse than the current situation.

An attacker would also be capable of removing the shared memory segment
and causing seeding to reoccur resulting in a denial of service attack.
This is partially mitigated by keeping the shared memory alive for the
duration of the process's existence.  Thus, an attacker would not only need
to have called call shmctl(2) with the IPC_RMID command but the system
must subsequently enter a state where no instances of libcrypto exist in
any process.  Even one long running process will prevent this attack.

The System V shared memory calls used here go back at least as far as
Linux kernel 2.0.  Linux kernels 4.8 and later, don't have a reliable way
to detect that /dev/urandom has been properly seeded, so a failure is raised
for this case (i.e. the getentropy(2) call has already failed).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9595)

prevent endless recursion when trace API is used within OPENSSL_init_crypto()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
(Merged from https://github.com/openssl/openssl/pull/9567)

cleanup provider digests

Added some missing #ifdef NO_XXX around some of the digest functions.
Renamed core_mkdigest.h to digestcommon.h
Added ERR_raise() to set/get params for digest.
Moved common code for get_params/gettable_params into digest_common.c
Renamed #defines in digestcommon.
Removed null_prov.c (It should not be needed)

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9625)

Add aes_ccm to provider

Add Cleanups for gcm - based on the changes to ccm.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/9280)

Get rid of using deprecated function isascii

Avoid using functions deprecated in some libcs (e.g. uClibc-ng).
Fixes #9557

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9628)

Add a fallback definition for __NR_getrandom for x86 linux

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9639)

Use macros internally for algorithm names

The macros are defined in include/openssl/core_names.h and follow the
naming standard OSSL_{OPNAME}_NAME_{ALGONAME}, where {OPNAME} is the
name of the operation (such as MAC) and {ALGONAME} is the name of the
algorithm.  Example: OSSL_MAC_NAME_HMAC

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9635)

doc: fix link in BN_new.pod

Fixes #9622

CLA: trivial

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9627)

Untangle / retangle opensslv.h, openssslconf.h and macros.h

When openssl/macros.h is included without openssl/opensslv.h, it can't
define OPENSSL_API_4 properly (with sufficient warnings enabled, the
compiler will complain about OPENSSL_VERSION_MAJOR not being defined).
The quick fix could have been to include openssl/opensslv.h in
openssl/macros.h, but that would create a nasty include loop, since
openssl/opensslv.h includes openssl/opensslconf.h, which includes
openssl/macros.h, in an order that leads back to macro check errors.

The objective is to make these headers more independent:

- openssl/opensslconf.h should really be completely independent, as it
  only defines macros for configuration values.  However, it needs to
  include openssl/macros.h for backward compatibility reasons.  We do
  this at the very end, under inclusion guards.
- openssl/macros.h is changed to include openssl/opensslconf.h, so it
  gets necessary configuration values to build some macros.  This will
  not cause an endless inclusion loop, since opensslconf.h's inclusion
  of macros.h is under guard.
- openssl/opensslv.h is changed to include openssl/macros.h instead of
  openssl/opensslconf.h.

Only one last piece needs to be done to make openssl/macros.h
independent from openssl/opensslv.h.  We can realise that the
definition of OPENSSL_API_4 doesn't need to depend on the current
version number.  There's nothing in our configuration that  would have
OPENSSL_API_4 defined to 1, and if the user sets OPENSSL_API_COMPAT or
OPENSSL_API_LEVEL to a high enough value, we consider that a
deliberate and knowledgable action on their part.

Fixes #7874
Fixes #9601

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9626)

Add a fallback definition for __NR_getrandom for ARM linux

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9619)

cipher cleanups.

Add test to evp_test_extra for ciphers (that is similiar to the digest_fetch).
Move some of the aes and gcm methods that can be shared with other ciphers into ciphers_common.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9580)

Add fips provider code for handling self test data

More PR's related to self test will be derived from this PR.

Note: the code removed in core_get_params() was causing a freeze since the
fips module was being loaded from a config file, which then called core_get_params()
which then tried to init the config fle again...

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9596)

Use WARNINGS heading not WARNING

Also update find-doc-nits to reject "=head1 WARNING"

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9602)

Use EXAMPLES not EXAMPLE for section title

And update find-doc-nits to complain if "=head1 EXAMPLE" is found.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9602)

Fix some pod-page ordering nits

The find-doc-nits script only looked for EXAMPLES, not EXAMPLE.
Fix the pattern and then fix the errors that resulted.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9602)

Fix --strict-warnings build

Appease -Wstring-plus-int.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9608)

Test for out-of-bounds write when requesting zero bytes from shake

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9433)

Directly return from final sha3/keccak_final if no bytes are requested

Requesting zero bytes from shake previously led to out-of-bounds write
on some platforms.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9433)

.travis.yml: Use travis_terminate on failure

Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9620)

Fix error handling in X509_chain_up_ref

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9614)

Windows UWP builds: determine automatically if asm should be disabled

Earlier Windows SDK versions lack the necessary support for our ASM
builds, so we check for the SDK version that has the support.

Information on exactly what registry key to check was found here:

https://stackoverflow.com/questions/2665755/how-can-i-determine-the-version-of-the-windows-sdk-installed-on-my-computer

Ref: #9125

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9440)

Fix 9bf682f which broke nistp224_method

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9607)

Rename ctx_{get,set}_params to {get,set}_ctx_params

Recently, we added dispatched functions to get parameter descriptions,
and those for operation context parameters ended up being called
something_gettable_ctx_params and something_settable_ctx_params.

The corresponding dispatched functions to actually perform parameter
transfers were previously called something_ctx_get_params and
something_ctx_set_params, which doesn't quite match, so we rename them
to something_get_ctx_params and something_set_ctx_params.

An argument in favor of this name change is English, where you'd
rather say something like "set the context parameters".

This only change the libcrypto <-> provider interface.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9612)

Ignore function code collisions

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9600)

Re-implement 'openssl list -mac-algorithms'

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Implement EVP_MAC_do_all_ex()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Add EVP_MAC_provider()

For information processing.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Rename the hash implementations KMAC{128,256} to KECCAK_KMAC{128,256}

This avoids getting them confused with the MAC implementations.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

OSSL_PARAM_construct_from_text(): handle non-hex octet string input

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Adjust some provider reason codes

BLAKE2 MACs came with a set of new reason codes.  Those talking about
lengths are consistently called PROV_R_INVALID_FOO_LENGTH, for any
name FOO.  The cipher messages were briefer.  In the interest of
having more humanly readable messages, we adjust the reasons used by
the ciphers (that's just IV length and key length).

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Adapt the MAC tests, and tests for other things that use EVP_MAC

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Adapt diverse code to provider based MACs.

CRMF, SSKDF, TLS1_PRF and SIV are affected by this.

This also forces the need to check MAC names, which leads to storing
the names in the created methods, which affects all EVP APIs, not just
EVP_MAC.  We will want that kind of information anyway (for example
for 'openssl list')...  Consequently, EVP_MAC_name() is re-implemented.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Adapt apps/mac.c to use provider based MACs

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Remove init of MACs from EVP

Now that all our MACs have moved to the default provider, we let it
take over completely

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Move Poly1305 to providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Move SipHash to providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Move KMAC to providers

Instead of using evp_keccak_kmac128() and evp_keccak_kmac256(), we refer
to the hash implementation by name, and fetch it, which should get us the
implementation from providers/common/digests/sha3_prov.c.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Move HMAC to providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Move GMAC to providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Move CMAC to providers

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Move BLAKE2 MACs to the providers

This also moves the remaining parts of BLAKE2 digests to the default
provider, and removes the legacy EVP implementation.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

Prepare EVP_MAC infrastructure for moving all MACs to providers

Quite a few adaptations are needed, most prominently the added code
to allow provider based MACs.

As part of this, all the old information functions are gone, except
for EVP_MAC_name().  Some of them will reappear later, for example
EVP_MAC_do_all() in some form.

MACs by EVP_PKEY was particularly difficult to deal with, as they
need to allocate and deallocate EVP_MAC_CTXs "under the hood", and
thereby implicitly fetch the corresponding EVP_MAC.  This means that
EVP_MACs can't be constant in a EVP_MAC_CTX, as their reference count
may need to be incremented and decremented as part of the allocation
or deallocation of the EVP_MAC_CTX.  It may be that other provider
based EVP operation types may need to be handled in a similar manner.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)

OPENSSL_s390xcap.pod: list msa9 facility bit (155)

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9348)

s390x assembly pack: accelerate ECDSA

for NIST P-256, P-384 and P-521 using KDSA instruction.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9348)

Enable curve-spefific ECDSA implementations via EC_METHOD

which are already enabled for ECDH.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9348)

s390x assembly pack: accelerate scalar multiplication

for NIST P-256, P-384 and P-521 using PCC instruction.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9348)

s390xcpuid.pl: fix comment

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9348)

Test SSL_set_ciphersuites

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9079)

Fix SSL_set_ciphersuites to set even if no call to SSL_set_cipher_list

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9079)

INSTALL: clarify documentation of the --api=x.y.z deprecation option

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9599)

Fix syntax error for the armv4 assembler

The error was from the alignment syntax of the code.
More details:
https://stackoverflow.com/questions/57316823/arm-assembly-syntax-in-vst-vld-commands?noredirect=1#comment101133590_57316823

CLA: trivial

Fixes: #9518
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9518)

Rename provider and core get_param_types functions

It was argued that names like SOMETHING_set_param_types were confusing,
and a rename has been proposed to SOMETHING_settable_params, and by
consequence, SOMETHING_get_param_types is renamed
SOMETHING_gettable_params.

This changes implements this change for the dispatched provider and
core functions.

Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9591)

crypto/engine/eng_openssl.c: define TEST_ENG_OPENSSL_RC4_P_INIT conditionally

When OpenSSL is configured with 'no-stdio', TEST_ENG_OPENSSL_RC4_P_INIT
shouldn't be defined, as that test uses stdio.

Fixes #9597

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9598)

Add {get,set}table_params() functions for provider digests

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9576)

Modify 'openssl list' to give more verbose descriptions

With the diverse {get,set}table_params functions, it's possible to
give a more verbose description of the diverse algorithm
implementations.  Most notably, we add a description of the parameters
that each implementation is willing to share.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9576)

Add missing EVP param utility functions

These functions were missing for a completes API:

EVP_MD_get_params(), EVP_CIPHER_get_params(), EVP_CIPHER_CTX_set_params(),
and EVP_CIPHER_CTX_get_params

Additionally, we also add all the corresponding parameter descriptor
returning functions, along the correspoding provider dispatches:

EVP_MD_gettable_params(), EVP_MD_CTX_settable_params(),
EVP_MD_CTX_gettable_params(), EVP_CIPHER_gettable_params(),
EVP_CIPHER_CTX_settable_params(), and EVP_CIPHER_CTX_gettable_params()

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9576)

Access data after obtaining the lock not before.

It isn't completely clear that this constitutes a race condition, but it will
always be conservative to access the locked data after getting the lock.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9579)

If we are multiblock capable make sure we use it

Recent changes to the assembler defines meant that they weren't being
set for libssl code. This resulted in the multiblock code never being
used.

Fixes #9571

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9574)

Add missing EBCDIC strings

Fix a few places where calling ossl_isdigit does the wrong thing on
EBCDIC based systems.
Replaced with ascii_isdigit.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9556)

Add description in X509_STORE manipulation

Add memory management description in X509_STORE_add_cert, otherwise
users will not be aware that they are leaking memory...

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9484)

BIO_lookup_ex: Do not retry on EAI_MEMORY

We should not retry on EAI_MEMORY as that error is most probably
fatal and not depending on AI_ADDRCONFIG hint.

Also report the error from the first call if the second call fails
as that one would be most probably the more interesting one.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9535)

BIO_lookup_ex: Always retry the lookup on failure with AI_NUMERICHOST set

Do not try to discern the error return value on
getaddrinfo() failure but when retrying set the AI_NUMERICHOST
to avoid DNS lookups.

Fixes: #9053
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9535)

30-test_evp.t: make the tested configs depend on what's been built

If the FIPS module hasn't been built, don't use test/fips.cnf, for example.

Fixes #9568

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9570)

Fix doc example code to follow coding style

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9577)

HMAC: use EVP_MD_flags(), not EVP_MD_meth_get_flags()

The latter should only be ussed with legacy methods.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9391)

Adapt the provider digests for more use of OSSL_PARAM

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9391)

Rework the provider digest constructor to provide implementation get_params

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9391)

Adapt diverse EVP_MD functions to use get_params and set_params interfaces

Also clean up EVP_MD_CTX_ctrl(), which did use these interfaces, but
development since allows for more elegant code.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9391)

Make more use of OSSL_PARAM for digests

A lot of the different numbers associated with digests are really
algorithm parameters.  block size, digest length, that sort of
thing.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9391)

Add OSSL_PARAM_construct_from_text() and OSSL_PARAM_allocate_from_text()

These are utility functions that can be used to replace calls to
ctrl_str type functions with get_params / set_params types of calls.
They work by translating text values to something more suitable for
OSSL_PARAM, and by interpretting parameter keys in a compatible
fashion.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9303)

Add OPENSSL_hexstr2buf_ex() and OPENSSL_buf2hexstr_ex()

They do the same thing as OPENSSL_hexstr2buf() and OPENSSL_buf2hexstr(),
except they take a result buffer from the caller.

We take the opportunity to break out the documentation of the hex to /
from buffer conversion routines from the OPENSSL_malloc() file to its
own file.  These routines aren't memory allocation routines per se.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9303)

enable DECLARE_DEPRECATED macro for Oracle Developer Studio compiler

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9434)

Configurations/unit-Makefile.tmpl: Don't clean away dotted files

A local 'make clean' did some sweeping removals of files execpt for
the .git directory.  This is a little too sweeping, as other dotted
files might be cleaned away if they happen to match the pattern that's
searched for.

An example is a symlink .dir-locals.el that would keep disappearing if
you build in the source tree and do a make clean...

So we change this to leave all dotted files alone.  Our builds do not
produce such files anyway, so this is a harmless (or rather, less
harmful) change.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9573)

Remove some duplicate words from the documentation

Fixup INSTALL and a couple man pages to get rid of "the the" and "in the
in the".

CLA: trivial

Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9563)

Fix windows compile errors in params.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9569)