--- /dev/null
+****************************
+ What's New In Python 3.5
+****************************
+
+:Release: |release|
+:Date: |today|
+
+.. Rules for maintenance:
+
+ * Anyone can add text to this document. Do not spend very much time
+ on the wording of your changes, because your text will probably
+ get rewritten to some degree.
+
+ * The maintainer will go through Misc/NEWS periodically and add
+ changes; it's therefore more important to add your changes to
+ Misc/NEWS than to this file.
+
+ * This is not a complete list of every single change; completeness
+ is the purpose of Misc/NEWS. Some changes I consider too small
+ or esoteric to include. If such a change is added to the text,
+ I'll just remove it. (This is another reason you shouldn't spend
+ too much time on writing your addition.)
+
+ * If you want to draw your new text to the attention of the
+ maintainer, add 'XXX' to the beginning of the paragraph or
+ section.
+
+ * It's OK to just add a fragmentary note about a change. For
+ example: "XXX Describe the transmogrify() function added to the
+ socket module." The maintainer will research the change and
+ write the necessary text.
+
+ * You can comment out your additions if you like, but it's not
+ necessary (especially when a final release is some months away).
+
+ * Credit the author of a patch or bugfix. Just the name is
+ sufficient; the e-mail address isn't necessary.
+
+ * It's helpful to add the bug/patch number as a comment:
+
+ XXX Describe the transmogrify() function added to the socket
+ module.
+ (Contributed by P.Y. Developer in :issue:`12345`.)
+
+ This saves the maintainer the effort of going through the Mercurial log
+ when researching a change.
+
+This article explains the new features in Python 3.5, compared to 3.4.
+
+For full details, see the :source:`Misc/NEWS` file.
+
+.. note:: Prerelease users should be aware that this document is currently in
+ draft form. It will be updated substantially as Python 3.5 moves towards
+ release, so it's worth checking back even after reading earlier versions.
+
+
+.. seealso::
+
+ :pep:`478` - Python 3.5 Release Schedule
+
+
+Summary -- Release highlights
+=============================
+
+.. This section singles out the most important changes in Python 3.3.
+ Brevity is key.
+
+New syntax features:
+
+* None yet.
+
+New library modules:
+
+* :mod:`zipapp`: :ref:`Improving Python ZIP Application Support
+ <whatsnew-zipapp>` (:pep:`441`).
+
+New built-in features:
+
+* None yet.
+
+Implementation improvements:
+
+* When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale),
+ :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the
+ ``surrogateescape`` error handler, instead of the ``strict`` error handler
+ (:issue:`19977`).
+
+Significantly Improved Library Modules:
+
+* None yet.
+
+Security improvements:
+
+* None yet.
+
+Please read on for a comprehensive list of user-facing changes.
+
+
+.. PEP-sized items next.
+
+.. _pep-4XX:
+
+.. PEP 4XX: Virtual Environments
+.. =============================
+
+
+.. (Implemented by Foo Bar.)
+
+.. .. seealso::
+
+ :pep:`4XX` - Python Virtual Environments
+ PEP written by Carl Meyer
+
+
+
+PEP 471 - os.scandir() function -- a better and faster directory iterator
+-------------------------------------------------------------------------
+
+:pep:`471` adds a new directory iteration function, :func:`os.scandir`,
+to the standard library. Additionally, :func:`os.walk` is now
+implemented using :func:`os.scandir`, which speeds it up by 3-5 times
+on POSIX systems and by 7-20 times on Windows systems.
+
+PEP and implementation written by Ben Hoyt with the help of Victor Stinner.
+
+.. seealso::
+
+ :pep:`471` -- os.scandir() function -- a better and faster directory
+ iterator
+
+
+PEP 475: Retry system calls failing with EINTR
+----------------------------------------------
+
+:pep:`475` adds support for automatic retry of system calls failing with EINTR:
+this means that user code doesn't have to deal with EINTR or InterruptedError
+manually, and should make it more robust against asynchronous signal reception.
+
+.. seealso::
+
+ :pep:`475` -- Retry system calls failing with EINTR
+
+
+PEP 486: Make the Python Launcher aware of virtual environments
+---------------------------------------------------------------
+
+:pep:`486` makes the Windows launcher (see :pep:`397`) aware of an active
+virtual environment. When the default interpreter would be used and the
+``VIRTUAL_ENV`` environment variable is set, the interpreter in the virtual
+environment will be used.
+
+.. seealso::
+
+ :pep:`486` -- Make the Python Launcher aware of virtual environments
+
+Other Language Changes
+======================
+
+Some smaller changes made to the core Python language are:
+
+* Added the ``'namereplace'`` error handlers. The ``'backslashreplace'``
+ error handlers now works with decoding and translating.
+ (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.)
+
+* The :option:`-b` option now affects comparisons of :class:`bytes` with
+ :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`)
+
+
+New Modules
+===========
+
+.. _whatsnew-zipapp:
+
+zipapp
+------
+
+The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and
+command line tool for creating executable Python Zip Applications, which
+were introduced in Python 2.6 in :issue:`1739468` but which were not well
+publicised, either at the time or since.
+
+With the new module, bundling your application is as simple as putting all
+the files, including a ``__main__.py`` file, into a directory ``myapp``
+and running::
+
+ $ python -m zipapp myapp
+ $ python myapp.pyz
+
+
+Improved Modules
+================
+
+argparse
+--------
+
+* :class:`~argparse.ArgumentParser` now allows to disable
+ :ref:`abbreviated usage <prefix-matching>` of long options by setting
+ :ref:`allow_abbrev` to ``False``.
+ (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.)
+
+cgi
+---
+
+* :class:`~cgi.FieldStorage` now supports the context management protocol.
+ (Contributed by Berker Peksag in :issue:`20289`.)
+
+code
+----
+
+* The :func:`code.InteractiveInterpreter.showtraceback` method now prints
+ the full chained traceback, just like the interactive interpreter.
+ (Contributed by Claudiu Popa in :issue:`17442`.)
+
+compileall
+----------
+
+* :func:`compileall.compile_dir` and :mod:`compileall`'s command-line interface
+ can now do parallel bytecode compilation.
+ (Contributed by Claudiu Popa in :issue:`16104`.)
+
+contextlib
+----------
+
+* The new :func:`contextlib.redirect_stderr` context manager(similar to
+ :func:`contextlib.redirect_stdout`) makes it easier for utility scripts to
+ handle inflexible APIs that write their output to :data:`sys.stderr` and
+ don't provide any options to redirect it.
+ (Contributed by Berker Peksag in :issue:`22389`.)
+
+difflib
+-------
+
+* The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file`
+ can now be customized by using *charset* keyword-only parameter. The default
+ charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``.
+ (Contributed by Berker Peksag in :issue:`2052`.)
+
+distutils
+---------
+
+* The ``build`` and ``build_ext`` commands now accept a ``-j``
+ option to enable parallel building of extension modules.
+ (Contributed by Antoine Pitrou in :issue:`5309`.)
+
+doctest
+-------
+
+* :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if
+ *module* contains no docstrings instead of raising :exc:`ValueError`.
+ (Contributed by Glenn Jones in :issue:`15916`.)
+
+glob
+----
+
+* :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in
+ subdirectories using the "``**``" pattern.
+ (Contributed by Serhiy Storchaka in :issue:`13968`.)
+
+imaplib
+-------
+
+* :class:`IMAP4` now supports the context management protocol. When used in a
+ :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called
+ automatically at the end of the block. (Contributed by Tarek Ziadé and
+ Serhiy Storchaka in :issue:`4972`.)
+
+imghdr
+------
+
+* :func:`~imghdr.what` now recognizes the `OpenEXR <http://www.openexr.com>`_
+ format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.)
+
+importlib
+---------
+
+* :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in
+ applications where startup time is paramount.
+ (Contributed by Brett Cannon in :issue:`17621`.)
+
+* :func:`importlib.abc.InspectLoader.source_to_code` is now a
+ static method to make it easier to work with source code in a string.
+ With a module object that you want to initialize you can then use
+ ``exec(code, module.__dict__)`` to execute the code in the module.
+
+* :func:`importlib.util.module_from_spec` is now the preferred way to create a
+ new module. Compared to :class:`types.ModuleType`, this new function will set
+ the various import-controlled attributes based on the passed-in spec object.
+
+inspect
+-------
+
+* :class:`inspect.Signature` and :class:`inspect.Parameter` are now
+ picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726`
+ and :issue:`20334`.)
+
+* New class method :meth:`inspect.Signature.from_callable`, which makes
+ subclassing of :class:`~inspect.Signature` easier. (Contributed
+ by Yury Selivanov and Eric Snow in :issue:`17373`.)
+
+ipaddress
+---------
+
+* :class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now
+ accept an ``(address, netmask)`` tuple argument, so as to easily construct
+ network objects from existing addresses. (Contributed by Peter Moody
+ and Antoine Pitrou in :issue:`16531`.)
+
+json
+----
+
+* The output of :mod:`json.tool` command line interface is now in the same
+ order as the input. Use the :option:`--sort-keys` option to sort the output
+ of dictionaries alphabetically by key. (Contributed by Berker Peksag in
+ :issue:`21650`.)
+
+* JSON decoder now raises :exc:`json.JSONDecodeError` instead of
+ :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.)
+
+os
+--
+
+* New :func:`os.scandir` function that exposes file information from
+ the operating system when listing a directory. :func:`os.scandir`
+ returns an iterator of :class:`os.DirEntry` objects corresponding to
+ the entries in the directory given by *path*. (Contributed by Ben
+ Hoyt with the help of Victor Stinner in :issue:`22524`.)
+
+* :class:`os.stat_result` now has a :attr:`~os.stat_result.st_file_attributes`
+ attribute on Windows. (Contributed by Ben Hoyt in :issue:`21719`.)
+
+re
+--
+
+* Number of capturing groups in regular expression is no longer limited by 100.
+ (Contributed by Serhiy Storchaka in :issue:`22437`.)
+
+* Now unmatched groups are replaced with empty strings in :func:`re.sub`
+ and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.)
+
+math
+----
+
+* :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark
+ Dickinson in :issue:`23185`.)
+
+shutil
+------
+
+* :func:`~shutil.move` now accepts a *copy_function* argument, allowing,
+ for example, :func:`~shutil.copy` to be used instead of the default
+ :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by
+ Claudiu Popa in :issue:`19840`.)
+
+signal
+------
+
+* Different constants of :mod:`signal` module are now enumeration values using
+ the :mod:`enum` module. This allows meaningful names to be printed during
+ debugging, instead of integer “magic numbers”. (Contributed by Giampaolo
+ Rodola' in :issue:`21076`.)
+
+smtpd
+-----
+
+* Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a
+ *decode_data* keyword to determine if the DATA portion of the SMTP
+ transaction is decoded using the ``utf-8`` codec or is instead provided to
+ :meth:`~smtpd.SMTPServer.process_message` as a byte string. The default
+ is ``True`` for backward compatibility reasons, but will change to ``False``
+ in Python 3.6. (Contributed by Maciej Szulik in :issue:`19662`.)
+
+* It is now possible to provide, directly or via name resolution, IPv6
+ addresses in the :class:`~smtpd.SMTPServer` constructor, and have it
+ successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.)
+
+* :mod:`~smtpd.SMTPServer` now supports :rfc:`6531` via the *enable_SMTPUTF8*
+ constructor argument and a user-provided
+ :meth:`~smtpd.SMTPServer.process_smtputf8_message` method.
+
+smtplib
+-------
+
+* A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to
+ implement custom authentication mechanisms.
+ (Contributed by Milan Oberkirch in :issue:`15014`.)
+
+sndhdr
+------
+
+* :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return
+ :func:`~collections.namedtuple`.
+ (Contributed by Claudiu Popa in :issue:`18615`.)
+
+socket
+------
+
+* New :meth:`socket.socket.sendfile` method allows to send a file over a socket
+ by using high-performance :func:`os.sendfile` function on UNIX resulting in
+ uploads being from 2x to 3x faster than when using plain
+ :meth:`socket.socket.send`.
+ (Contributed by Giampaolo Rodola' in :issue:`17552`.)
+
+sysconfig
+---------
+
+* The user scripts directory on Windows is now versioned.
+ (Contributed by Paul Moore in :issue:`23437`.)
+
+
+tarfile
+-------
+
+* The :func:`tarfile.open` function now supports ``'x'`` (exclusive creation)
+ mode. (Contributed by Berker Peksag in :issue:`21717`.)
+
+time
+----
+
+* The :func:`time.monotonic` function is now always available. (Contributed by
+ Victor Stinner in :issue:`22043`.)
+
+urllib
+------
+
+* A new :class:`urllib.request.HTTPBasicPriorAuthHandler` allows HTTP Basic
+ Authentication credentials to be sent unconditionally with the first HTTP
+ request, rather than waiting for a HTTP 401 Unauthorized response from the
+ server.
+ (Contributed by Matej Cepl in :issue:`19494`.)
+
+wsgiref
+-------
+
+* *headers* parameter of :class:`wsgiref.headers.Headers` is now optional.
+ (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.)
+
+xmlrpc
+------
+
+* :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`.
+ (Contributed by Claudiu Popa in :issue:`20627`.)
+
+faulthandler
+------------
+
+* :func:`~faulthandler.enable`, :func:`~faulthandler.register`,
+ :func:`~faulthandler.dump_traceback` and
+ :func:`~faulthandler.dump_traceback_later` functions now accept file
+ descriptors. (Contributed by Wei Wu in :issue:`23566`.)
+
+zipfile
+-------
+
+* Added support for writing ZIP files to unseekable streams.
+ (Contributed by Serhiy Storchaka in :issue:`23252`.)
+
+
+Optimizations
+=============
+
+The following performance enhancements have been added:
+
+* :func:`os.walk` has been sped up by 3-5x on POSIX systems and 7-20x
+ on Windows. This was done using the new :func:`os.scandir` function,
+ which exposes file information from the underlying ``readdir`` and
+ ``FindFirstFile``/``FindNextFile`` system calls. (Contributed by
+ Ben Hoyt with help from Victor Stinner in :issue:`23605`.)
+
+* Construction of ``bytes(int)`` (filled by zero bytes) is faster and use less
+ memory for large objects. ``calloc()`` is used instead of ``malloc()`` to
+ allocate memory for these objects.
+
+* Some operations on :class:`~ipaddress.IPv4Network` and
+ :class:`~ipaddress.IPv6Network` have been massively sped up, such as
+ :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`,
+ :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`.
+ The speed up can range from 3x to 15x.
+ (:issue:`21486`, :issue:`21487`, :issue:`20826`)
+
+* Many operations on :class:`io.BytesIO` are now 50% to 100% faster.
+ (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in
+ :issue:`22003`.)
+
+* :func:`marshal.dumps` is now faster (65%-85% with versions 3--4, 20-25% with
+ versions 0--2 on typical data, and up to 5x in best cases).
+ (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.)
+
+
+Build and C API Changes
+=======================
+
+Changes to Python's build process and to the C API include:
+
+* New ``calloc`` functions:
+
+ * :c:func:`PyMem_RawCalloc`
+ * :c:func:`PyMem_Calloc`
+ * :c:func:`PyObject_Calloc`
+ * :c:func:`_PyObject_GC_Calloc`
+
+
+Deprecated
+==========
+
+Unsupported Operating Systems
+-----------------------------
+
+* None yet.
+
+
+Deprecated Python modules, functions and methods
+------------------------------------------------
+
+* The :mod:`formatter` module has now graduated to full deprecation and is still
+ slated for removal in Python 3.6.
+
+* :mod:`smtpd` has in the past always decoded the DATA portion of email
+ messages using the ``utf-8`` codec. This can now be controlled by the new
+ *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is
+ ``True``, but this default is deprecated. Specify the *decode_data* keyword
+ with an appropriate value to avoid the deprecation warning.
+
+* :class:`~http.cookies.Morsel` has previously allowed for setting attributes
+ :attr:`~http.cookies.Morsel.key`, :attr:`~http.cookies.Morsel.value` and
+ :attr:`~http.cookies.Morsel.coded_value`. Use the preferred
+ :func:`~http.cookies.Morsel.set` method in order to avoid the deprecation
+ warning.
+
++* Passing a format string as keyword argument *format_string* to the
++ :meth:`~string.Formatter.format` method of the :class:`string.Formatter`
++ class has been deprecated.
++
+
+Deprecated functions and types of the C API
+-------------------------------------------
+
+* None yet.
+
+
+Deprecated features
+-------------------
+
+* None yet.
+
+
+Removed
+=======
+
+API and Feature Removals
+------------------------
+
+The following obsolete and previously deprecated APIs and features have been
+removed:
+
+* The ``__version__`` attribute has been dropped from the email package. The
+ email code hasn't been shipped separately from the stdlib for a long time,
+ and the ``__version__`` string was not updated in the last few releases.
+
+* The internal ``Netrc`` class in the :mod:`ftplib` module was deprecated in
+ 3.4, and has now been removed.
+ (Contributed by Matt Chaput in :issue:`6623`.)
+
+Porting to Python 3.5
+=====================
+
+This section lists previously described changes and other bugfixes
+that may require changes to your code.
+
+Changes in the Python API
+-------------------------
+
+* Before Python 3.5, a :class:`datetime.time` object was considered to be false
+ if it represented midnight in UTC. This behavior was considered obscure and
+ error-prone and has been removed in Python 3.5. See :issue:`13936` for full
+ details.
+
+* :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError`
+ or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation
+ would block. Previously, it would return 0. See :issue:`20951`.
+
+* The ``__name__`` attribute of generator is now set from the function name,
+ instead of being set from the code name. Use ``gen.gi_code.co_name`` to
+ retrieve the code name. Generators also have a new ``__qualname__``
+ attribute, the qualified name, which is now used for the representation
+ of a generator (``repr(gen)``). See :issue:`21205`.
+
+* The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`,
+ :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been
+ removed. (Contributed by Ezio Melotti in :issue:`15114`.)
+ The *convert_charrefs* argument of :class:`~html.parser.HTMLParser` is
+ now ``True`` by default. (Contributed by Berker Peksag in :issue:`21047`.)
+
+* Although it is not formally part of the API, it is worth noting for porting
+ purposes (ie: fixing tests) that error messages that were previously of the
+ form "'sometype' does not support the buffer protocol" are now of the form "a
+ bytes-like object is required, not 'sometype'". (Contributed by Ezio Melotti
+ in :issue:`16518`.)
+
+* If the current directory is set to a directory that no longer exists then
+ :exc:`FileNotFoundError` will no longer be raised and instead
+ :meth:`~importlib.machinery.FileFinder.find_spec` will return ``None``
+ **without** caching ``None`` in :data:`sys.path_importer_cache` which is
+ different than the typical case (:issue:`22834`).
+
+* HTTP status code and messages from :mod:`http.client` and :mod:`http.server`
+ were refactored into a common :class:`~http.HTTPStatus` enum. The values in
+ :mod:`http.client` and :mod:`http.server` remain available for backwards
+ compatibility. (Contributed by Demian Brecht in :issue:`21793`.)
+
+* When an import loader defines :meth:`~importlib.machinery.Loader.exec_module`
+ it is now expected to also define
+ :meth:`~importlib.machinery.Loader.create_module` (raises a
+ :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader
+ inherits from :class:`importlib.abc.Loader` then there is nothing to do, else
+ simply define :meth:`~importlib.machinery.Loader.create_module` to return
+ ``None`` (:issue:`23014`).
+
+* :func:`re.split` always ignored empty pattern matches, so the ``'x*'``
+ pattern worked the same as ``'x+'``, and the ``'\b'`` pattern never worked.
+ Now :func:`re.split` raises a warning if the pattern could match
+ an empty string. For compatibility use patterns that never match an empty
+ string (e.g. ``'x+'`` instead of ``'x*'``). Patterns that could only match
+ an empty string (such as ``'\b'``) now raise an error.
+
+Changes in the C API
+--------------------
+
+* The undocumented :c:member:`~PyMemoryViewObject.format` member of the
+ (non-public) :c:type:`PyMemoryViewObject` structure has been removed.
+
+ All extensions relying on the relevant parts in ``memoryobject.h``
+ must be rebuilt.
+
+* The :c:type:`PyMemAllocator` structure was renamed to
+ :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added.
+
+* Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references.
+ Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions
+ to format the :func:`repr` of the object.
+
+* Because the lack of the :attr:`__module__` attribute breaks pickling and
+ introspection, a deprecation warning now is raised for builtin type without
+ the :attr:`__module__` attribute. Would be an AttributeError in future.
+ (:issue:`20204`)
projects / python / commitdiff