Remove instances in libssl of the constant 28 (for size of IPv4 header + UDP)
and instead use the value provided by the underlying BIO. Also provide some
new DTLS_CTRLs so that the library user can set the mtu without needing to
know this constant. These new DTLS_CTRLs provide the capability to set the
link level mtu to be used (i.e. including this IP/UDP overhead). The previous
DTLS_CTRLs required the library user to subtract this overhead first.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 59669b6abf620d1ed2ef4d1e2df25c998b89b64d)

Conflicts:
ssl/d1_both.c
ssl/ssl.h
ssl/ssl_lib.c

There are a number of instances throughout the code where the constant 28 is
used with no explanation. Some of this was introduced as part of RT#1929. The
value 28 is the length of the IP header (20 bytes) plus the UDP header (8
bytes). However use of this constant is incorrect because there may be
instances where a different value is needed, e.g. an IPv4 header is 20 bytes
but an IPv6 header is 40. Similarly you may not be using UDP (e.g. SCTP).
This commit introduces a new BIO_CTRL that provides the value to be used for
this mtu "overhead". It will be used by subsequent commits.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 0d3ae34df573f477b6b1aaf614d52dcdfcff5fce)

Conflicts:
crypto/bio/bio.h
crypto/bio/bss_dgram.c

The first call to query the mtu in dtls1_do_write correctly checks that the
mtu that we have received is not less than the minimum. If its less it uses the
minimum instead. The second call to query the mtu does not do that, but
instead uses whatever comes back. We have seen an instance in RT#3592 where we
have got an unreasonably small mtu come back. This commit makes both query
checks consistent.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 6abb0d1f8e702a0daa9c32b8021d01eda0483018)

The SSL_OP_NO_QUERY_MTU option is supposed to stop the mtu from being
automatically updated, and we should use the one provided instead.
Unfortunately there are a couple of locations where this is not respected.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 001235778a6e9c645dc0507cad6092d99c9af8f5)

Verify that we have a sensible message len and fail if not
RT#3592 provides an instance where the OPENSSL_assert that this commit
replaces can be hit. I was able to recreate this issue by forcing the
underlying BIO to misbehave and come back with very small mtu values. This
happens the second time around the while loop after we have detected that the
MTU has been exceeded following the call to dtls1_write_bytes.

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit cf75017bfd60333ff65edf9840001cd2c49870a3)

Check for FindNextFile when defining it rather than FindFirstFile

Reviewed-by: Matt Caswell <matt@openssl.org>

[PR3597] Advance to the next state variant when reusing messages.

Previously, state variant was not advanced, which resulted in state
being stuck in the st1 variant (usually "_A").

This broke certificate callback retry logic when accepting connections
that were using SSLv2 ClientHello (hence reusing the message), because
their state never advanced to SSL3_ST_SR_CLNT_HELLO_C variant required
for the retry code path.

Reported by Yichun Zhang (agentzh).

Signed-off-by: Piotr Sikora <piotr@cloudflare.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Correct some layout issues, convert all remaining tabs to appropriate amounts of spaces.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit 8123d158ab6f0a6a468748c133e33c2063ff36b5)

Improves the proxy certificates howto doc.

The current documentation contains a bunch of spelling and grammar mistakes. I also
found it hard to understand some paragraphs, so here is my attempt to improve its
readability.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit 03b637a730e4a298c360cc143de7564060c06324)

Fixed warning in ssl2_enc

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 2db95e094d23b41d2305c0a01db66694ea77f936)

Check EVP_Cipher return values for SSL2

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 5fc8bb6ab71b5f7ce2dd67a47494bb5b5c6dc374)

Add checks to the return value of EVP_Cipher to prevent silent encryption failure.

PR#1767

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit 244d0955adc027c0f41a3251e55d145bf940f9ce)

Remove redundant checks in ssl_cert_dup. This was causing spurious error messages when using GOST

PR#3613

Reviewed-by: Richard Levitte <levitte@openssl.org>
(cherry picked from commit fc3968a25ce0c16cab8730ec0d68a59856158029)

Remove duplicated code

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

Tidy up ocsp help output

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>
(cherry picked from commit 5e31a40f47c6bfd09c718d2af42ba8d8fe6bb932)

Conflicts:
apps/ocsp.c

(cherry picked from commit e16458269036f4334525009906d346f68a73b2a4)

Add documentation on -timeout option in the ocsp utility

PR#3612

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>
(cherry picked from commit de87dd46c1283f899a9ecf4ccc72db74f36afbf2)
(cherry picked from commit 4d3df37bc7fd33d0bec5da04d2572caa0cdbab75)

Fixed memory leak due to incorrect freeing of DTLS reassembly bit mask

PR#3608

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 8a35dbb6d89a16d792b79b157b3e89443639ec94)

Corrected comments in ssl.h about SSLv23_method and friends

PR#3574

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>
(cherry picked from commit 3a0765882c4b3b67960b7efb203570764dd4ed29)

Fix cross reference table generator.

If the hash or public key algorithm is "undef" the signature type
will receive special handling and shouldn't be included in the
cross reference table.
Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 55f7fb8848b6e4bec291724a479e1580d6f407d6)

Conflicts:
crypto/objects/obj_xref.h

Fixes a minor typo in the EVP docs.

Out is the buffer which needs to contain at least inl + cipher_block_size - 1 bytes. Outl
is just an int*.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(cherry picked from commit 5211e094dec9486a540ac480f345df1a8d2b2862)

Correct timestamp output when clock_precision_digits > 0

PR#3535

Reviewed-by: Stephen Henson <steve@openssl.org>

Fix free of garbage pointer. PR#3595

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(cherry picked from commit e04d426bf98ebb22abf0f15b6f09d333a6e8b2ad)

Fix warning about negative unsigned intergers

Reviewed-by: Richard Levitte <levitte@openssl.org>

Use only unsigned arithmetic in constant-time operations

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Emilia Käsper <emilia@openssl.org>

Fix and improve SSL_MODE_SEND_FALLBACK_SCSV documentation.

Reviewed-by: Rich Salz <rsalz@openssl.org>

When processing ClientHello.cipher_suites, don't ignore cipher suites
listed after TLS_FALLBACK_SCSV.

RT: 3575
Reviewed-by: Emilia Kasper <emilia@openssl.org>

Keep old method in case of an unsupported protocol

When we're configured with no-ssl3 and we receive an SSL v3 Client Hello, we set
the method to NULL.  We didn't used to do that, and it breaks things.  This is a
regression introduced in 62f45cc27d07187b59551e4fad3db4e52ea73f2c.  Keep the old
method since the code is not able to deal with a NULL method at this time.

CVE-2014-3569, PR#3571

Reviewed-by: Emilia Käsper <emilia@openssl.org>
(cherry picked from commit 392fa7a952e97d82eac6958c81ed1e256e6b8ca5)

no-ssl2 with no-ssl3 does not mean drop the ssl lib

Reviewed-by: Geoff Thorpe <geoff@openssl.org>
(cherry picked from commit c882abd52269a59ed8e0510e5febf667428ece85)

e_os.h: refine inline override logic (to address warnings in debug build).

Reviewed-by: Dr Stephen Henson <steve@openssl.org>
(cherry picked from commit 55c7a4cf112bf154ed405ee05a6b7924b6b1ba92)

e_os.h: allow inline functions to be compiled by legacy compilers.

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit 40155f408985aff2e9f1b61b7cb04a3e518633a1)

RT3547: Add missing static qualifier

Reviewed-by: Ben Laurie <ben@openssl.org>
(cherry picked from commit 87d388c955c14a7c1371f9c7555fb429a406a3d3)

Prepare for 1.0.0p-dev

Reviewed-by: Stephen Henson <steve@openssl.org>

Prepare for 1.0.0o release

Reviewed-by: Stephen Henson <steve@openssl.org>

Updates to NEWS

Reviewed-by: Dr Stephen Henson <steve@openssl.org>

Update to CHANGES file

Reviewed-by: Bodo Möller <bodo@openssl.org>

Fix no-ssl3 configuration option

CVE-2014-3568

Reviewed-by: Emilia Kasper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix for session tickets memory leak.

CVE-2014-3567

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Fix SSL_R naming inconsistency.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add constant_time_locl.h to HEADERS,
so the Win32 compile picks it up correctly.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add the constant time test to the VMS build and tests

Reviewed-by: Tim Hudson <tjh@openssl.org>
Conflicts:
test/maketests.com
test/tests.com

Include "constant_time_locl.h" rather than "../constant_time_locl.h".
The different -I compiler parameters will take care of the rest...

Reviewed-by: Tim Hudson <tjh@openssl.org>
Conflicts:
crypto/evp/evp_enc.c

Spaces were added in some strings for better readability. However, those spaces do not belong in file names, so when picking out the individual parts, remove the spaces

Reviewed-by: Tim Hudson <tjh@openssl.org>

Adjust VMS build to Unix build.  Most of all, make it so the disabled
algorithms MD2 and RC5 don't get built.
Also, disable building the test apps in crypto/des and crypto/pkcs7, as
they have no support at all.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Conflicts:
crypto/crypto-lib.com
makevms.com
ssl/ssl-lib.com

Make sure test/tests.com exit gracefully, even when openssl.exe wasn't properly built.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Make sure that disabling the MAYLOSEDATA3 warning is only done when the compiler supports it. Otherwise, there are warnings about it lacking everywhere, which is quite tedious to read through while trying to check for other warnings.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Conflicts:
ssl/ssl-lib.com

Add TLS_FALLBACK_SCSV documentation, and move s_client -fallback_scsv
handling out of #ifndef OPENSSL_NO_DTLS1 section.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Oops -- fix typo in coment added with TLS_FALLBACK_SCSV support.

Reviewed-by: Steve Henson <steve@openss.org>

Support TLS_FALLBACK_SCSV.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Removed duplicate definition of PKCS7_type_is_encrypted

Patch supplied by Matthieu Patou <mat@matws.net>, and modified to also
remove duplicate definition of PKCS7_type_is_digest.

PR#3551

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit e0fdea3e49e7454aa76bd5ecf3a3747641354c68)

Add additional DigestInfo checks.

Reencode DigestInto in DER and check against the original: this
will reject any improperly encoded DigestInfo structures.

Note: this is a precautionary measure, there is no known attack
which can exploit this.

Thanks to Brian Smith for reporting this issue.
Reviewed-by: Tim Hudson <tjh@openssl.org>

Add missing tests

Accidentally omitted from commit 455b65dfab0de51c9f67b3c909311770f2b3f801

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(cherry picked from commit fdc35a9d3e8cf4cfd9330d5df9883f42cf5648ad)

Use correct function name: CMS_add1_signer()

Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit 5886354dcca4f8445ed35b6995a035b75409590c)

crypto/bn/bn_nist.c: work around MSC ARM compiler bug.

RT: 3541
Reviewed-by: Emilia Kasper <emilia@openssl.org>
(cherry picked from commit 8b07c005fe006044d0e4a795421447deca3c9f2c)

RT3425: constant-time evp_enc

Do the final padding check in EVP_DecryptFinal_ex in constant time to
avoid a timing leak from padding failure.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e)

Conflicts:
crypto/evp/evp_enc.c

(cherry picked from commit 738911cde68b2b3706e502cf8daf5b14738f2f42)

Conflicts:
crypto/evp/evp_enc.c

RT3067: simplify patch

(Original commit adb46dbc6dd7347750df2468c93e8c34bcb93a4b)

Use the new constant-time methods consistently in s3_srvr.c

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
(cherry picked from commit 455b65dfab0de51c9f67b3c909311770f2b3f801)

Conflicts:
ssl/Makefile

This change alters the processing of invalid, RSA pre-master secrets so
that bad encryptions are treated like random session keys in constant
time.

(cherry picked from commit adb46dbc6dd7347750df2468c93e8c34bcb93a4b)

Reviewed-by: Rich Salz <rsalz@openssl.org>

RT3066: rewrite RSA padding checks to be slightly more constant time.

Also tweak s3_cbc.c to use new constant-time methods.
Also fix memory leaks from internal errors in RSA_padding_check_PKCS1_OAEP_mgf1

This patch is based on the original RT submission by Adam Langley <agl@chromium.org>,
as well as code from BoringSSL and OpenSSL.

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Conflicts:
crypto/rsa/rsa_oaep.c

Fixed error introduced in commit f2be92b94dad3c6cbdf79d99a324804094cf1617
that fixed PR#3450 where an existing cast masked an issue when i was changed
from int to long in that commit

Picked up on z/linux (s390) where sizeof(int)!=sizeof(long)

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(cherry picked from commit b5ff559ff90124c6fd53bbb49dae5edb4e821e0a)

psk_client_callback, 128-byte id bug.

Fix a bug in handling of 128 byte long PSK identity in
psk_client_callback.

OpenSSL supports PSK identities of up to (and including) 128 bytes in
length. PSK identity is obtained via the psk_client_callback,
implementors of which are expected to provide a NULL-terminated
identity. However, the callback is invoked with only 128 bytes of
storage thus making it impossible to return a 128 byte long identity and
the required additional NULL byte.

This CL fixes the issue by passing in a 129 byte long buffer into the
psk_client_callback. As a safety precaution, this CL also zeroes out the
buffer before passing it into the callback, uses strnlen for obtaining
the length of the identity returned by the callback, and aborts the
handshake if the identity (without the NULL terminator) is longer than
128 bytes.

(Original patch amended to achieve strnlen in a different way.)

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit be0d851732bad7370640702bc9c4a33189ede287)

Ensure that x**0 mod 1 = 0.

(cherry picked from commit 2b0180c37fa6ffc48ee40caa831ca398b828e680)

Reviewed-by: Ben Laurie <ben@openssl.org>

Followup on RT3334 fix: make sure that a directory that's the empty
string returns 0 with errno = ENOENT.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(cherry picked from commit 360928b7d0f16dde70e26841bbf9e1af727e8b8f)

RT3334: Fix crypto/LPdir_win.c

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(cherry picked from commit 6a14fe7576e7a14a46ba14df8be8fe478536b4fb)

Make the inline const-time functions static.

"inline" without static is not correct as the compiler may choose to ignore it
and will then either emit an external definition, or expect one.

Reviewed-by: Geoff Thorpe <geoff@openssl.org>
(cherry picked from commit 86f50b36e63275a916b147f9d8764e3c0c060fdb)

md5-x86_64.pl: work around warning.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit 4d86e8df6be69ed13abb73fd564f1f894eea0a98)

Add tags/TAGS

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 9d6253cfd3395dfe7147dae134579c6b16544c53)

RT2379: Bug in BIO_set_accept_port.pod

The doc says that port can be "*" to mean any port.
That's wrong.

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>
(cherry picked from commit 07e3b31fae98b985d3d2aad7066144b11833f688)

Fixed double inclusion of string.h

PR2693

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 5d33b70ef5a4768fdfb77a73f9817c4570613039)

Constant-time utilities

Pull constant-time methods out to a separate header, add tests.

Reviewed-by: Bodo Moeller <bodo@openssl.org>
(cherry picked from commit e7169a5835390d20057de8a19785982bd6a9b8c8)

Conflicts:
ssl/Makefile
test/Makefile

Explicitly check for empty ASN.1 strings in d2i_ECPrivateKey

The old code implicitly relies on the ASN.1 code returning a \0-prefixed buffer
when the buffer length is 0. Change this to verify explicitly that the ASN.1 string
has positive length.

Reviewed-by: Dr Stephen Henson <steve@openssl.org>
(cherry picked from commit 82dc08de54ce443c2a9ac478faffe79e76157795)

RT3065: automatically generate a missing EC public key

When d2i_ECPrivateKey reads a private key with a missing (optional) public key,
generate one automatically from the group and private key.

Reviewed-by: Dr Stephen Henson <steve@openssl.org>
(cherry picked from commit ed383f847156940e93f256fed78599873a4a9b28)

Conflicts:
doc/crypto/EC_KEY_new.pod

RT3065: ec_private_key_dont_crash

This change saves several EC routines from crashing when an EC_KEY is
missing a public key. The public key is optional in the EC private key
format and, without this patch, running the following through `openssl
ec` causes a crash:

-----BEGIN EC PRIVATE KEY-----
MBkCAQEECAECAwQFBgcIoAoGCCqGSM49AwEH
-----END EC PRIVATE KEY-----

Reviewed-by: Dr Stephen Henson <steve@openssl.org>
(cherry picked from commit b391570bdeb386d4fd325917c248d593d3c43930)

RT2210: Add missing EVP_cleanup to example

I also removed some trailing whitespace and cleaned
up the "see also" list.

Reviewed-by: Emilia Kasper <emilia@openssl.org>
(cherry picked from commit 7b3e11c54466f1da8b707c932e308d345fd61101)

RT1744: SSL_CTX_set_dump_dh() doc feedback

The description of when the server creates a DH key is
confusing.  This cleans it up.
(rsalz: also removed trailing whitespace.)

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

RT1804: fix EXAMPLE in EVP_EncryptInit.pod

The EXAMPLE that used FILE and RC2 doesn't compile due to a
few minor errors.  Tweak to use IDEA and AES-128. Remove
examples about RC2 and RC5.

Reviewed-by: Emilia Kasper <emilia@openssl.org>

Typo fixes to evp documentation.

This patch was submitted by user "Kox" via the wiki

Reviewed-by: Tim Hudson <tjh@openssl.org>
(cherry picked from commit 2dd8cb3b9593f528d9537aa6a003d5c93df1e3c5)

RT3060: Limit the number of empty records.

Limit the number of empty records that will be processed consecutively
in order to prevent ssl3_get_record from never returning.

Reported by "oftc_must_be_destroyed" and George Kadianakis.

Reviewed-by: Bodo Moeller <bodo@openssl.org>
(cherry picked from commit 3aac17a82fbaf2bc23ee62f24611e5883d3e7b97)

RT3061: Don't SEGFAULT when trying to export a public DSA key as a private key.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit e19c93811f0db499c98d2888f1c0c0ab65e6238a)

Improve EVP_PKEY_sign documentation

Clarify the intended use of EVP_PKEY_sign. Make the code example compile.

Reviewed-by: Dr Stephen Henson <steve@openssl.org>
(cherry picked from commit d64c533a207f7b6d86c3bc8ffb053e5f4d0c1ca0)

define inline for Visual Studio

In Visual Studio, inline is available in C++ only, however __inline is available for C, see
http://msdn.microsoft.com/en-us/library/z8y1yy88.aspx

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Dr Stephen Henson <steve@openssl.org>
(cherry picked from commit f511b25a7370c775dc9fd6198dbacd1706cf242b)

Add volatile qualifications to two blocks of inline asm to stop GCC from
eliminating them as dead code.

Both volatile and "memory" are used because of some concern that the compiler
may still cache values across the asm block without it, and because this was
such a painful debugging session that I wanted to ensure that it's never
repeated.

(cherry picked from commit 7753a3a68431aa81b82beea4c3f5374b41454679)

Conflicts:
crypto/bn/asm/x86_64-gcc.c

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit a90b1e32d2f697d1aa39b49038469e2ea40d5e7b)

Fixed out-of-bounds read errors in ssl3_get_key_exchange.

PR#3450

Conflicts:
ssl/s3_clnt.c

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Further improve/fix ec_GFp_simple_points_make_affine (ecp_smpl.c) and
group_order_tests (ectest.c).  Also fix the EC_POINTs_mul documentation (ec.h).

Reviewed-by: emilia@openssl.org
Conflicts:
crypto/ec/ectest.c

Prepare for 1.0.0o-dev

Reviewed-by: Stephen Henson <steve@openssl.org>

Prepare for 1.0.0n release

Reviewed-by: Stephen Henson <steve@openssl.org>

make update

Reviewed-by: Stephen Henson <steve@openssl.org>

Updates to CHANGES and NEWS

Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Fix race condition in ssl_parse_serverhello_tlsext

CVE-2014-3509
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

Fix OID handling:

- Upon parsing, reject OIDs with invalid base-128 encoding.
- Always NUL-terminate the destination buffer in OBJ_obj2txt printing function.

CVE-2014-3508

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>

Fix DTLS anonymous EC(DH) denial of service

CVE-2014-3510

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>

Fix protocol downgrade bug in case of fragmented packets

CVE-2014-3511

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Bodo Möller <bodo@openssl.org>

Remove some duplicate DTLS code.

In a couple of functions, a sequence number would be calculated twice.

Additionally, in |dtls1_process_out_of_seq_message|, we know that
|frag_len| <= |msg_hdr->msg_len| so the later tests for |frag_len <
msg_hdr->msg_len| can be more clearly written as |frag_len !=
msg_hdr->msg_len|, since that's the only remaining case.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>

Applying same fix as in dtls1_process_out_of_seq_message. A truncated DTLS fragment would cause *ok to be clear, but the return value would still be the number of bytes read.

Problem identified by Emilia Käsper, based on previous issue/patch by Adam
Langley.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Fix return code for truncated DTLS fragment.

Previously, a truncated DTLS fragment in
|dtls1_process_out_of_seq_message| would cause *ok to be cleared, but
the return value would still be the number of bytes read. This would
cause |dtls1_get_message| not to consider it an error and it would
continue processing as normal until the calling function noticed that
*ok was zero.

I can't see an exploit here because |dtls1_get_message| uses
|s->init_num| as the length, which will always be zero from what I can
see.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>

Fix memory leak from zero-length DTLS fragments.

The |pqueue_insert| function can fail if one attempts to insert a
duplicate sequence number. When handling a fragment of an out of
sequence message, |dtls1_process_out_of_seq_message| would not call
|dtls1_reassemble_fragment| if the fragment's length was zero. It would
then allocate a fresh fragment and attempt to insert it, but ignore the
return value, leaking the fragment.

This allows an attacker to exhaust the memory of a DTLS peer.

Fixes CVE-2014-3507

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>

Fix DTLS handshake message size checks.

In |dtls1_reassemble_fragment|, the value of
|msg_hdr->frag_off+frag_len| was being checked against the maximum
handshake message size, but then |msg_len| bytes were allocated for the
fragment buffer. This means that so long as the fragment was within the
allowed size, the pending handshake message could consume 16MB + 2MB
(for the reassembly bitmap). Approx 10 outstanding handshake messages
are allowed, meaning that an attacker could consume ~180MB per DTLS
connection.

In the non-fragmented path (in |dtls1_process_out_of_seq_message|), no
check was applied.

Fixes CVE-2014-3506

Wholly based on patch by Adam Langley with one minor amendment.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Added comment for the frag->reassembly == NULL case as per feedback from Emilia

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Avoid double free when processing DTLS packets.

The |item| variable, in both of these cases, may contain a pointer to a
|pitem| structure within |s->d1->buffered_messages|. It was being freed
in the error case while still being in |buffered_messages|. When the
error later caused the |SSL*| to be destroyed, the item would be double
freed.

Thanks to Wah-Teh Chang for spotting that the fix in 1632ef74 was
inconsistent with the other error paths (but correct).

Fixes CVE-2014-3505

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>

Fix warnings about ignored return values.
(cherry picked from commit 27131fe8f7418bf22b1e3000ea6a5d7b1ec8ebd4)
Reviewed-by: Matt Caswell <matt@openssl.org>

Simplify and fix ec_GFp_simple_points_make_affine
(which didn't always handle value 0 correctly).

Reviewed-by: emilia@openssl.org
Conflicts:
CHANGES
crypto/ec/ectest.c

"EC_POINT_invert" was checking "dbl" function pointer instead of "invert".

PR#2569

Reviewed-by: Rich Salz <rsalz@openssl.org>
(cherry picked from commit cba11f57ce161fd301a72194827327128191de7e)