Fix tls_cbc_digest_record is slow using SHA-384 and short messages

The formula used for this is now

kVarianceBlocks = ((255 + 1 + md_size + md_block_size - 1) / md_block_size) + 1

Notice that md_block_size=64 for SHA256, which results on the
magic constant kVarianceBlocks = 6.
However, md_block_size=128 for SHA384 leading to kVarianceBlocks = 4.

CLA:trivial

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7342)

diff --git a/ssl/s3_cbc.c b/ssl/s3_cbc.c
index 7d9c3776973decb1398ff9b4b971dc2230824950..8e11864b07f028776261e9c23879ee6b3a70b662 100644 (file)
--- a/ssl/s3_cbc.c
+++ b/ssl/s3_cbc.c
@@ -256,12 +256,13 @@ int ssl3_cbc_digest_record(const EVP_MD_CTX *ctx,
      * of hash termination (0x80 + 64-bit length) don't fit in the final
      * block, we say that the final two blocks can vary based on the padding.
      * TLSv1 has MACs up to 48 bytes long (SHA-384) and the padding is not
-     * required to be minimal. Therefore we say that the final six blocks can
+     * required to be minimal. Therefore we say that the final |variance_blocks|
+     * blocks can
      * vary based on the padding. Later in the function, if the message is
      * short and there obviously cannot be this many blocks then
      * variance_blocks can be reduced.
      */
-    variance_blocks = is_sslv3 ? 2 : 6;
+    variance_blocks = is_sslv3 ? 2 : ( ((255 + 1 + md_size + md_block_size - 1) / md_block_size) + 1);
     /*
      * From now on we're dealing with the MAC, which conceptually has 13
      * bytes of `header' before the start of the data (TLS) or 71/75 bytes