Build: Avoid SHA256_Init on FreeBSD and MINIX 3.

On FreeBSD 10 and older, SHA256_Init from libmd conflicts
with libcrypto from OpenSSL. The OpenSSL version has
different sizeof(SHA256_CTX) and it can cause weird
problems if wrong SHA256_Init gets used.

Looking at the source, MINIX 3 seems to have a similar issue but
I'm not sure. To be safe, I disabled SHA256_Init on MINIX 3 too.

NetBSD has SHA256_Init in libc and they had a similar problem,
but they already fixed it in 2009.

Thanks to Jim Wilcoxson for the bug report that helped
in finding the problem.

diff --git a/configure.ac b/configure.ac
index d64b0e085516edbd1dd191252c8c5b330ac4147e..32e68efa3413217f24aa328a61b461f42fd4d369 100644 (file)
--- a/configure.ac
+++ b/configure.ac
@@ -669,18 +669,32 @@ TUKLIB_PHYSMEM
 TUKLIB_CPUCORES
 TUKLIB_MBSTR
 
-# Check for system-provided SHA-256. At least the following is supported:
+# Check for system-provided SHA-256. The supported implementations are listed
+# below. The detection for the ones marked with [*] has been intentionally
+# disabled because they have symbol name conflicts with OpenSSL's libcrypto
+# which can cause weird problems (clean namespaces would make things too
+# boring, I guess).
 #
 # OS       Headers                     Library  Type           Function
-# FreeBSD  sys/types.h + sha256.h      libmd    SHA256_CTX     SHA256_Init
+# FreeBSD  sys/types.h + sha256.h      libmd    SHA256_CTX     SHA256_Init [*]
 # NetBSD   sys/types.h + sha2.h                 SHA256_CTX     SHA256_Init
 # OpenBSD  sys/types.h + sha2.h                 SHA2_CTX       SHA256Init
 # Solaris  sys/types.h + sha2.h        libmd    SHA256_CTX     SHA256Init
-# MINIX 3  sys/types.h + minix/sha2.h  libutil  SHA256_CTX     SHA256_Init
+# MINIX 3  sys/types.h + minix/sha2.h  libutil  SHA256_CTX     SHA256_Init [*]
 # Darwin   CommonCrypto/CommonDigest.h          CC_SHA256_CTX  CC_SHA256_Init
 #
-# Note that Darwin's CC_SHA256_Update takes buffer size as uint32_t instead
-# of size_t.
+# Notes:
+#
+#   - NetBSD's SHA256_Init doesn't conflict with libcrypto because
+#     libcrypto on NetBSD was made to use the libc implementation to avoid
+#     this exact symbol conflict problem:
+#     http://ftp.netbsd.org/pub/NetBSD/security/advisories/NetBSD-SA2009-012.txt.asc
+#
+#   - As of 2016-03-10, FreeBSD seems to have the issue fixed in SVN head
+#     but not in the FreeBSD 10 branch.
+#
+#   - Darwin's CC_SHA256_Update takes buffer size as uint32_t instead
+#     of size_t.
 #
 # We don't check for e.g. OpenSSL or libgcrypt because we don't want
 # to introduce dependencies to other packages by default. Maybe such
@@ -712,7 +726,8 @@ if test "x$enable_check_sha256" = "xyes"; then
                          #ifdef HAVE_MINIX_SHA2_H
                          # include <minix/sha2.h>
                          #endif]])
-               AC_SEARCH_LIBS([SHA256_Init], [md util])
+               dnl Omit detection of the FreeBSD and MINIX 3 versions:
+               dnl AC_SEARCH_LIBS([SHA256_Init], [md util])
                AC_SEARCH_LIBS([SHA256Init], [md])
                AC_CHECK_FUNCS([CC_SHA256_Init SHA256_Init SHA256Init],
                        [break])