Prior to
72ceb6a we treated all failures from the call to
EVP_DigestVerifyFinal() as if it were a bad signature, and failures in
EVP_DigestUpdate() as an internal error. After that commit we replaced
this with the one-shot function EVP_DigestVerify() and treated a 0 return
as a bad signature and a negative return as an internal error. However,
some signature errors can be negative (e.g. according to the docs if the
form of the signature is wrong). Therefore we should treat all <=0
returns as a bad signature.
This fixes a boringssl test failure.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3756)
rv = EVP_DigestVerify(md_ctx, PACKET_data(&signature),
PACKET_remaining(&signature), tbs, tbslen);
OPENSSL_free(tbs);
- if (rv < 0) {
- al = SSL_AD_INTERNAL_ERROR;
- SSLerr(SSL_F_TLS_PROCESS_KEY_EXCHANGE, ERR_R_EVP_LIB);
- goto err;
- } else if (rv == 0) {
+ if (rv <= 0) {
al = SSL_AD_DECRYPT_ERROR;
SSLerr(SSL_F_TLS_PROCESS_KEY_EXCHANGE, SSL_R_BAD_SIGNATURE);
goto err;