#11454: Reduce email module load time, improve surrogate check efficiency.

The new _has_surrogates code was suggested by Serhiy Storchaka.  See
the issue for timings, but it is far faster than any other alternative,
and also removes the load time that we previously incurred from compiling
the complex regex this replaces.

diff --git a/Lib/email/utils.py b/Lib/email/utils.py
index 93a625c8b447d9742ff8a69258cbeec9e3ca56fa..b3b42bb379296f7768733e9ed54c8168f09a582e 100644 (file)
--- a/Lib/email/utils.py
+++ b/Lib/email/utils.py
@@ -54,10 +54,16 @@ TICK = "'"
 specialsre = re.compile(r'[][\\()<>@,:;".]')
 escapesre = re.compile(r'[\\"]')
 
-# How to figure out if we are processing strings that come from a byte
-# source with undecodable characters.
-_has_surrogates = re.compile(
-    '([^\ud800-\udbff]|\A)[\udc00-\udfff]([^\udc00-\udfff]|\Z)').search
+def _has_surrogates(s):
+    """Return True if s contains surrogate-escaped binary data."""
+    # This check is based on the fact that unless there are surrogates, utf8
+    # (Python's default encoding) can encode any string.  This is the fastest
+    # way to check for surrogates, see issue 11454 for timings.
+    try:
+        s.encode()
+        return False
+    except UnicodeEncodeError:
+        return True
 
 # How to deal with a string containing bytes before handing it to the
 # application through the 'normal' interface.