From: Yury Selivanov <yury@magic.io>
Date: Wed, 9 Nov 2016 14:43:08 +0000 (-0500)
Subject: Merge 3.5 (issue #27942)
X-Git-Tag: v3.6.0b4~115
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=6c1065061770e843f55be0506489c931a2cb7e96;p=python

Merge 3.5 (issue #27942)
---

6c1065061770e843f55be0506489c931a2cb7e96
diff --cc Misc/NEWS
index d787a29046,10ec9e0b38..6b77ffe38b
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@@ -356,390 -213,51 +356,392 @@@ Librar
  - Issue #25895: Enable WebSocket URL schemes in urllib.parse.urljoin.
    Patch by Gergely Imreh and Markus Holtermann.
  
 +- Issue #28114: Fix a crash in parse_envlist() when env contains byte strings.
 +  Patch by Eryk Sun.
 +
  - Issue #27599: Fixed buffer overrun in binascii.b2a_qp() and binascii.a2b_qp().
  
 -- Issue #19003:m email.generator now replaces only \r and/or \n line
 -  endings, per the RFC, instead of all unicode line endings.
 +- Issue #27906: Fix socket accept exhaustion during high TCP traffic.
 +  Patch by Kevin Conway.
  
 -- Issue #28019: itertools.count() no longer rounds non-integer step in range
 -  between 1.0 and 2.0 to 1.
 +- Issue #28174: Handle when SO_REUSEPORT isn't properly supported.
 +  Patch by Seth Michael Larson.
  
 -- Issue #25969: Update the lib2to3 grammar to handle the unpacking
 -  generalizations added in 3.5.
 +- Issue #26654: Inspect functools.partial in asyncio.Handle.__repr__.
 +  Patch by iceboy.
  
 -- Issue #14977: mailcap now respects the order of the lines in the mailcap
 -  files ("first match"), as required by RFC 1542.  Patch by Michael Lazar.
 +- Issue #26909: Fix slow pipes IO in asyncio.
 +  Patch by INADA Naoki.
  
 -- Issue #24594: Validates persist parameter when opening MSI database
 +- Issue #28176: Fix callbacks race in asyncio.SelectorLoop.sock_connect.
  
 -- Issue #17582: xml.etree.ElementTree nows preserves whitespaces in attributes
 -  (Patch by Duane Griffin.  Reviewed and approved by Stefan Behnel.)
 +- Issue #27759: Fix selectors incorrectly retain invalid file descriptors.
 +  Patch by Mark Williams.
  
 -- Issue #28047: Fixed calculation of line length used for the base64 CTE
 -  in the new email policies.
 +- Issue #28368: Refuse monitoring processes if the child watcher has no
 +  loop attached.
 +  Patch by Vincent Michel.
  
 -- Issue #27445: Don't pass str(_charset) to MIMEText.set_payload().
 -  Patch by Claude Paroz.
 +- Issue #28369: Raise RuntimeError when transport's FD is used with
 +  add_reader, add_writer, etc.
  
 -- Issue #22450: urllib now includes an "Accept: */*" header among the
 -  default headers.  This makes the results of REST API requests more
 -  consistent and predictable especially when proxy servers are involved.
 +- Issue #28370: Speedup asyncio.StreamReader.readexactly.
 +  Patch by Коренберг Марк.
  
 -- lib2to3.pgen3.driver.load_grammar() now creates a stable cache file
 -  between runs given the same Grammar.txt input regardless of the hash
 -  randomization setting.
 +- Issue #28371: Deprecate passing asyncio.Handles to run_in_executor.
  
 -- Issue #27570: Avoid zero-length memcpy() etc calls with null source
 -  pointers in the "ctypes" and "array" modules.
 +- Issue #28372: Fix asyncio to support formatting of non-python coroutines.
  
 -- Issue #22233: Break email header lines *only* on the RFC specified CR and LF
 -  characters, not on arbitrary unicode line breaks.  This also fixes a bug in
 -  HTTP header parsing.
 +- Issue #28399: Remove UNIX socket from FS before binding.
 +  Patch by Коренберг Марк.
  
 -- Issue 27988: Fix email iter_attachments incorrect mutation of payload list.
 +- Issue #27972: Prohibit Tasks to await on themselves.
  
 -- Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name
 +Windows
 +-------
 +
 +- Issue #28402: Adds signed catalog files for stdlib on Windows.
 +
 +- Issue #28333: Enables Unicode for ps1/ps2 and input() prompts. (Patch by
 +  Eryk Sun)
 +
 +- Issue #28251: Improvements to help manuals on Windows.
 +
 +- Issue #28110: launcher.msi has different product codes between 32-bit and
 +  64-bit
 +
 +- Issue #28161: Opening CON for write access fails
 +
 +- Issue #28162: WindowsConsoleIO readall() fails if first line starts with
 +  Ctrl+Z
 +
 +- Issue #28163: WindowsConsoleIO fileno() passes wrong flags to
 +  _open_osfhandle
 +
 +- Issue #28164: _PyIO_get_console_type fails for various paths
 +
 +- Issue #28137: Renames Windows path file to ._pth
 +
 +- Issue #28138: Windows ._pth file should allow import site
 +
 +C API
 +-----
 +
 +- Issue #28426: Deprecated undocumented functions PyUnicode_AsEncodedObject(),
 +  PyUnicode_AsDecodedObject(), PyUnicode_AsDecodedUnicode() and
 +  PyUnicode_AsEncodedUnicode().
 +
 +Build
 +-----
 +
 +- Issue #28258: Fixed build with Estonian locale (python-config and distclean
 +  targets in Makefile).  Patch by Arfrever Frehtes Taifersar Arahesis.
 +
 +- Issue #26661: setup.py now detects system libffi with multiarch wrapper.
 +
 +- Issue #15819: Remove redundant include search directory option for building
 +  outside the source tree.
 +
 +Tests
 +-----
 +
 +- Issue #28217: Adds _testconsole module to test console input.
 +
 +
 +What's New in Python 3.6.0 beta 1
 +=================================
 +
 +*Release date: 2016-09-12*
 +
 +Core and Builtins
 +-----------------
 +
 +- Issue #23722: The __class__ cell used by zero-argument super() is now
 +  initialized from type.__new__ rather than __build_class__, so class methods
 +  relying on that will now work correctly when called from metaclass methods
 +  during class creation. Patch by Martin Teichmann.
 +
 +- Issue #25221: Fix corrupted result from PyLong_FromLong(0) when Python
 +  is compiled with NSMALLPOSINTS = 0.
 +
 +- Issue #27080: Implement formatting support for PEP 515.  Initial patch
 +  by Chris Angelico.
 +
 +- Issue #27199: In tarfile, expose copyfileobj bufsize to improve throughput.
 +  Patch by Jason Fried.
 +
 +- Issue #27948: In f-strings, only allow backslashes inside the braces
 +  (where the expressions are).  This is a breaking change from the 3.6
 +  alpha releases, where backslashes are allowed anywhere in an
 +  f-string.  Also, require that expressions inside f-strings be
 +  enclosed within literal braces, and not escapes like
 +  f'\x7b"hi"\x7d'.
 +
 +- Issue #28046: Remove platform-specific directories from sys.path.
 +
 +- Issue #28071: Add early-out for differencing from an empty set.
 +
 +- Issue #25758: Prevents zipimport from unnecessarily encoding a filename
 +  (patch by Eryk Sun)
 +
 +- Issue #25856: The __module__ attribute of extension classes and functions
 +  now is interned. This leads to more compact pickle data with protocol 4.
 +
 +- Issue #27213: Rework CALL_FUNCTION* opcodes to produce shorter and more
 +  efficient bytecode. Patch by Demur Rumed, design by Serhiy Storchaka,
 +  reviewed by Serhiy Storchaka and Victor Stinner.
 +
 +- Issue #26331: Implement tokenizing support for PEP 515. Patch by Georg Brandl.
 +
 +- Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal.
 +  Patch by Ivan Levkivskyi.
 +
 +- Issue #28003: Implement PEP 525 -- Asynchronous Generators.
 +
 +- Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations.
 +  Patch by Ivan Levkivskyi.
 +
 +- Issue #26058: Add a new private version to the builtin dict type, incremented
 +  at each dictionary creation and at each dictionary change. Implementation of
 +  the PEP 509.
 +
 +- Issue #27364: A backslash-character pair that is not a valid escape sequence
 +  now generates a DeprecationWarning.  Patch by Emanuel Barry.
 +
 +- Issue #27350: `dict` implementation is changed like PyPy. It is more compact
 +  and preserves insertion order.
 +  (Concept developed by Raymond Hettinger and patch by Inada Naoki.)
 +
 +- Issue #27911: Remove unnecessary error checks in
 +  ``exec_builtin_or_dynamic()``.
 +
 +- Issue #27078: Added BUILD_STRING opcode.  Optimized f-strings evaluation.
 +
 +- Issue #17884: Python now requires systems with inttypes.h and stdint.h
 +
 +- Issue #27961: Require platforms to support ``long long``. Python hasn't
 +  compiled without ``long long`` for years, so this is basically a formality.
 +
 +- Issue #27355: Removed support for Windows CE.  It was never finished,
 +  and Windows CE is no longer a relevant platform for Python.
 +
 +- Implement PEP 523.
 +
 +- Issue #27870: A left shift of zero by a large integer no longer attempts
 +  to allocate large amounts of memory.
 +
 +- Issue #25402: In int-to-decimal-string conversion, improve the estimate
 +  of the intermediate memory required, and remove an unnecessarily strict
 +  overflow check. Patch by Serhiy Storchaka.
 +
 +- Issue #27214: In long_invert, be more careful about modifying object
 +  returned by long_add, and remove an unnecessary check for small longs.
 +  Thanks Oren Milman for analysis and patch.
 +
 +- Issue #27506: Support passing the bytes/bytearray.translate() "delete"
 +  argument by keyword.
 +
 +- Issue #27812: Properly clear out a generator's frame's backreference to the
 +  generator to prevent crashes in frame.clear().
 +
 +- Issue #27811: Fix a crash when a coroutine that has not been awaited is
 +  finalized with warnings-as-errors enabled.
 +
 +- Issue #27587: Fix another issue found by PVS-Studio: Null pointer check
 +  after use of 'def' in _PyState_AddModule().
 +  Initial patch by Christian Heimes.
 +
 +- Issue #27792: The modulo operation applied to ``bool`` and other
 +  ``int`` subclasses now always returns an ``int``. Previously
 +  the return type depended on the input values. Patch by Xiang Zhang.
 +
 +- Issue #26984: int() now always returns an instance of exact int.
 +
 +- Issue #25604: Fix a minor bug in integer true division; this bug could
 +  potentially have caused off-by-one-ulp results on platforms with
 +  unreliable ldexp implementations.
 +
 +- Issue #24254: Make class definition namespace ordered by default.
 +
 +- Issue #27662: Fix an overflow check in ``List_New``: the original code was
 +  checking against ``Py_SIZE_MAX`` instead of the correct upper bound of
 +  ``Py_SSIZE_T_MAX``. Patch by Xiang Zhang.
 +
 +- Issue #27782: Multi-phase extension module import now correctly allows the
 +  ``m_methods`` field to be used to add module level functions to instances
 +  of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang.
 +
 +- Issue #27936: The round() function accepted a second None argument
 +  for some types but not for others.  Fixed the inconsistency by
 +  accepting None for all numeric types.
 +
 +- Issue #27487: Warn if a submodule argument to "python -m" or
 +  runpy.run_module() is found in sys.modules after parent packages are
 +  imported, but before the submodule is executed.
 +
 +- Issue #27157: Make only type() itself accept the one-argument form.
 +  Patch by Eryk Sun and Emanuel Barry.
 +
 +- Issue #27558: Fix a SystemError in the implementation of "raise" statement.
 +  In a brand new thread, raise a RuntimeError since there is no active
 +  exception to reraise. Patch written by Xiang Zhang.
 +
 +- Issue #28008: Implement PEP 530 -- asynchronous comprehensions.
 +
++- Issue #27942: Fix memory leak in codeobject.c
++
 +Library
 +-------
 +
 +- Issue #28485: Always raise ValueError for negative
 +  compileall.compile_dir(workers=...) parameter, even when multithreading is
 +  unavailable.
 +
 +- Issue #28037: Use sqlite3_get_autocommit() instead of setting
 +  Connection->inTransaction manually.
 +
 +- Issue #25283: Attributes tm_gmtoff and tm_zone are now available on
 +  all platforms in the return values of time.localtime() and
 +  time.gmtime().
 +
 +- Issue #24454: Regular expression match object groups are now
 +  accessible using __getitem__.  "mo[x]" is equivalent to
 +  "mo.group(x)".
 +
 +- Issue #10740: sqlite3 no longer implicitly commit an open transaction
 +  before DDL statements.
 +
 +- Issue #17941: Add a *module* parameter to collections.namedtuple().
 +
 +- Issue #22493: Inline flags now should be used only at the start of the
 +  regular expression.  Deprecation warning is emitted if uses them in the
 +  middle of the regular expression.
 +
 +- Issue #26885: xmlrpc now supports unmarshalling additional data types used
 +  by Apache XML-RPC implementation for numerics and None.
 +
 +- Issue #28070: Fixed parsing inline verbose flag in regular expressions.
 +
 +- Issue #19500: Add client-side SSL session resumption to the ssl module.
 +
 +- Issue #28022: Deprecate ssl-related arguments in favor of SSLContext. The
 +  deprecation include manual creation of SSLSocket and certfile/keyfile
 +  (or similar) in ftplib, httplib, imaplib, smtplib, poplib and urllib.
 +
 +- Issue #28043: SSLContext has improved default settings: OP_NO_SSLv2,
 +  OP_NO_SSLv3, OP_NO_COMPRESSION, OP_CIPHER_SERVER_PREFERENCE,
 +  OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE and HIGH ciphers without MD5.
 +
 +- Issue #24693: Changed some RuntimeError's in the zipfile module to more
 +  appropriate types. Improved some error messages and debugging output.
 +
 +- Issue #17909: ``json.load`` and ``json.loads`` now support binary input
 +  encoded as UTF-8, UTF-16 or UTF-32. Patch by Serhiy Storchaka.
 +
 +- Issue #27137: the pure Python fallback implementation of ``functools.partial``
 +  now matches the behaviour of its accelerated C counterpart for subclassing,
 +  pickling and text representation purposes. Patch by Emanuel Barry and
 +  Serhiy Storchaka.
 +
 +- Fix possible integer overflows and crashes in the mmap module with unusual
 +  usage patterns.
 +
 +- Issue #1703178: Fix the ability to pass the --link-objects option to the
 +  distutils build_ext command.
 +
 +- Issue #28019: itertools.count() no longer rounds non-integer step in range
 +  between 1.0 and 2.0 to 1.
 +
 +- Issue #18401: Pdb now supports the 'readrc' keyword argument to control
 +  whether .pdbrc files should be read.  Patch by Martin Matusiak and
 +  Sam Kimbrel.
 +
 +- Issue #25969: Update the lib2to3 grammar to handle the unpacking
 +  generalizations added in 3.5.
 +
 +- Issue #14977: mailcap now respects the order of the lines in the mailcap
 +  files ("first match"), as required by RFC 1542.  Patch by Michael Lazar.
 +
 +- Issue #28025: Convert all ssl module constants to IntEnum and IntFlags.
 +  SSLContext properties now return flags and enums.
 +
 +- Issue #433028: Added support of modifier spans in regular expressions.
 +
 +- Issue #24594: Validates persist parameter when opening MSI database
 +
 +- Issue #17582: xml.etree.ElementTree nows preserves whitespaces in attributes
 +  (Patch by Duane Griffin.  Reviewed and approved by Stefan Behnel.)
 +
 +- Issue #28047: Fixed calculation of line length used for the base64 CTE
 +  in the new email policies.
 +
 +- Issue #27576: Fix call order in OrderedDict.__init__().
 +
 +- email.generator.DecodedGenerator now supports the policy keyword.
 +
 +- Issue #28027: Remove undocumented modules from ``Lib/plat-*``: IN, CDROM,
 +  DLFCN, TYPES, CDIO, and STROPTS.
 +
 +- Issue #27445: Don't pass str(_charset) to MIMEText.set_payload().
 +  Patch by Claude Paroz.
 +
 +- Issue #24277: The new email API is no longer provisional, and the docs
 +  have been reorganized and rewritten to emphasize the new API.
 +
 +- Issue #22450: urllib now includes an "Accept: */*" header among the
 +  default headers.  This makes the results of REST API requests more
 +  consistent and predictable especially when proxy servers are involved.
 +
 +- lib2to3.pgen3.driver.load_grammar() now creates a stable cache file
 +  between runs given the same Grammar.txt input regardless of the hash
 +  randomization setting.
 +
 +- Issue #28005: Allow ImportErrors in encoding implementation to propagate.
 +
 +- Issue #26667: Support path-like objects in importlib.util.
 +
 +- Issue #27570: Avoid zero-length memcpy() etc calls with null source
 +  pointers in the "ctypes" and "array" modules.
 +
 +- Issue #22233: Break email header lines *only* on the RFC specified CR and LF
 +  characters, not on arbitrary unicode line breaks.  This also fixes a bug in
 +  HTTP header parsing.
 +
 +- Issue 27331: The email.mime classes now all accept an optional policy keyword.
 +
 +- Issue 27988: Fix email iter_attachments incorrect mutation of payload list.
 +
 +- Issue #16113: Add SHA-3 and SHAKE support to hashlib module.
 +
 +- Eliminate a tautological-pointer-compare warning in _scproxy.c.
 +
 +- Issue #27776: The :func:`os.urandom` function does now block on Linux 3.17
 +  and newer until the system urandom entropy pool is initialized to increase
 +  the security. This change is part of the :pep:`524`.
 +
 +- Issue #27778: Expose the Linux ``getrandom()`` syscall as a new
 +  :func:`os.getrandom` function. This change is part of the :pep:`524`.
 +
 +- Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name
    fields in X.509 certs.
  
 +- Issue #18844: Add random.choices().
 +
 +- Issue #25761: Improved error reporting about truncated pickle data in
 +  C implementation of unpickler.  UnpicklingError is now raised instead of
 +  AttributeError and ValueError in some cases.
 +
 +- Issue #26798: Add BLAKE2 (blake2b and blake2s) to hashlib.
 +
 +- Issue #26032: Optimized globbing in pathlib by using os.scandir(); it is now
 +  about 1.5--4 times faster.
 +
 +- Issue #25596: Optimized glob() and iglob() functions in the
 +  glob module; they are now about 3--6 times faster.
 +
 +- Issue #27928: Add scrypt (password-based key derivation function) to
 +  hashlib module (requires OpenSSL 1.1.0).
 +
  - Issue #27850: Remove 3DES from ssl module's default cipher list to counter
    measure sweet32 attack (CVE-2016-2183).