From: Serhiy Storchaka Date: Mon, 23 Dec 2013 16:21:57 +0000 (+0200) Subject: Removed spaces before commas and periods. X-Git-Tag: v3.4.0b2~114 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=2a6145290be7aec586c170ce7e405fb3813f39fd;p=python Removed spaces before commas and periods. --- 2a6145290be7aec586c170ce7e405fb3813f39fd diff --cc Doc/reference/import.rst index e5cb8d9d16,eb497a10b2..a9723ecbff --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@@ -625,9 -473,8 +625,9 @@@ The Path Based Finde single: path based finder As mentioned previously, Python comes with several default meta path finders. -One of these, called the :term:`path based finder`, searches an :term:`import -path`, which contains a list of :term:`path entries `. Each path +One of these, called the :term:`path based finder` - (:class:`~importlib.machinery.PathFinder`) , searches an :term:`import path`, ++(:class:`~importlib.machinery.PathFinder`), searches an :term:`import path`, +which contains a list of :term:`path entries `. Each path entry names a location to search for modules. The path based finder itself doesn't know how to import anything. Instead, it diff --cc Doc/whatsnew/3.4.rst index 90175c58b3,0000000000..f1aa364c2c mode 100644,000000..100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@@ -1,1150 -1,0 +1,1150 @@@ +**************************** + What's New In Python 3.4 +**************************** + +.. :Author: Someone + (uncomment if there is a principal author) + +.. Rules for maintenance: + + * Anyone can add text to this document, but the maintainer reserves the + right to rewrite any additions. In particular, for obscure or esoteric + features, the maintainer may reduce any addition to a simple reference to + the new documentation rather than explaining the feature inline. + + * While the maintainer will periodically go through Misc/NEWS + and add changes, it's best not to rely on this. We know from experience + that any changes that aren't in the What's New documentation around the + time of the original release will remain largely unknown to the community + for years, even if they're added later. We also know from experience that + other priorities can arise, and the maintainer will run out of time to do + updates - in such cases, end users will be much better served by partial + notifications that at least give a hint about new features to + investigate. + + * This is not a complete list of every single change; completeness + is the purpose of Misc/NEWS. The What's New should focus on changes that + are visible to Python *users* and that *require* a feature release (i.e. + most bug fixes should only be recorded in Misc/NEWS) + + * PEPs should not be marked Final until they have an entry in What's New. + A placeholder entry that is just a section header and a link to the PEP + (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been + implemented and noted in What's New, don't forget to mark it as Final! + + * If you want to draw your new text to the attention of the + maintainer, add 'XXX' to the beginning of the paragraph or + section. + + * It's OK to add just a very brief note about a change. For + example: "The :ref:`~socket.transmogrify()` function was added to the + :mod:`socket` module." The maintainer will research the change and + write the necessary text (if appropriate). The advantage of doing this + is that even if no more descriptive text is ever added, readers will at + least have a notification that the new feature exists and a link to the + relevant documentation. + + * You can comment out your additions if you like, but it's not + necessary (especially when a final release is some months away). + + * Credit the author of a patch or bugfix. Just the name is + sufficient; the e-mail address isn't necessary. + + * It's helpful to add the bug/patch number as a comment: + + The :ref:`~socket.transmogrify()` function was added to the + :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.) + + This saves the maintainer the effort of going through the Mercurial log + when researching a change. + + * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``, + while :ref:`~mod.attr` will display as ``attr``. + +This article explains the new features in Python 3.4, compared to 3.3. + +.. Python 3.4 was released on TBD. + +For full details, see the +`changelog `_. + +.. note:: Prerelease users should be aware that this document is currently in + draft form. It will be updated substantially as Python 3.4 moves towards + release, so it's worth checking back even after reading earlier versions. + + +.. seealso:: + + :pep:`429` - Python 3.4 Release Schedule + + +Summary -- Release highlights +============================= + +.. This section singles out the most important changes in Python 3.4. + Brevity is key. + +New syntax features: + +* No new syntax features were added in Python 3.4. + +New expected features for Python implementations: + +* :ref:`Make newly created file descriptors non-inheritable ` + (:pep:`446`). +* command line option for :ref:`isolated mode `, + (:issue:`16499`). +* :ref:`improvements ` in the handling of + codecs that are not text encodings +* :ref:`A ModuleSpec Type ` for the Import System + (:pep:`451`). (Affects importer authors.) + +New library modules: + +* :mod:`asyncio`: New provisonal API for asynchronous IO (:pep:`3156`). +* :mod:`enum`: Support for enumeration types (:pep:`435`). +* :mod:`ensurepip`: Bootstrapping the pip installer (:pep:`453`). +* :mod:`pathlib`: Object-oriented filesystem paths (:pep:`428`). +* :mod:`selectors`: High-level and efficient I/O multiplexing, built upon the + :mod:`select` module primitives. +* :mod:`statistics`: A basic numerically stable statistics library (:pep:`450`). +* :mod:`tracemalloc`: Trace Python memory allocations (:pep:`454`). + +Significantly Improved Library Modules: + +* :ref:`Single-dispatch generic functions ` in + :mod:`functools` (:pep:`443`) +* New :mod:`pickle` :ref:`protocol 4 ` (:pep:`3154`) +* :ref:`SHA-3 (Keccak) support ` for :mod:`hashlib`. +* :ref:`TLSv1.1 and TLSv1.2 support ` for :mod:`ssl`. +* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork + on Unix ` (:issue:`8713`). +* :mod:`email` has a new submodule, :mod:`~email.contentmanager`, and + a new :mod:`~email.message.Message` subclass + (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME + handling `. + +CPython implementation improvements: + +* :ref:`Safe object finalization ` (:pep:`442`). +* Leveraging :pep:`442`, :ref:`module globals are no longer set to None + during finalization `, in most cases (:issue:`18214`). +* :ref:`Configurable memory allocators ` (:pep:`445`). +* :ref:`Secure and interchangeable hash algorithm ` + (:pep:`456`). +* :ref:`Argument Clinic ` (:pep:`436`). +* The :mod:`marshal` format has been made :ref:`more compact and efficient + ` (:issue:`16475`). + +Please read on for a comprehensive list of user-facing changes, including +sections on deprecations and porting issues. + + +New expected features for Python implementations +================================================ + +PEP 453: Explicit bootstrapping of pip in Python installations +-------------------------------------------------------------- + +The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard +cross-platform mechanism to boostrap the pip installer into Python +installations and virtual environments. + +The :mod:`venv` module and the :command:`pyvenv` utility make use of this +module to make ``pip`` readily available in virtual environments. When +using the command line interface, ``pip`` is installed by default, while +for the module API installation of ``pip`` must be requested explicitly. + +For CPython source builds on POSIX systems, the ``make install`` and +``make altinstall`` commands bootstrap ``pip`` by default. This behaviour +can be controlled through configure options, and overridden through +Makefile options. + +On Windows and Mac OS X, the CPython installers now offer the option to +install ``pip`` along with CPython itself. + +.. note:: + + The implementation of PEP 453 is still a work in progress. Refer to + :issue:`19347` for the progress on additional steps: + + * Having the binary installers install ``pip`` by default + * Recommending the use of ``pip`` in the "Installing Python Module" + documentation. + +.. seealso:: + + :pep:`453` - Explicit bootstrapping of pip in Python installations + PEP written by Donald Stufft and Nick Coghlan, implemented by + Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily. + + +.. _whatsnew-pep-446: + +PEP 446: Make newly created file descriptors non-inheritable +------------------------------------------------------------ + +:pep:`446` makes newly created file descriptors :ref:`non-inheritable +`. New functions and methods: + +* :func:`os.get_inheritable`, :func:`os.set_inheritable` +* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` +* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable` + +.. seealso:: + + :pep:`446` - Make newly created file descriptors non-inheritable + PEP written and implemented by Victor Stinner. + + +.. _codec-handling-improvements: + +Improvements to codec handling +------------------------------ + +Since it was first introduced, the :mod:`codecs` module has always been +intended to operate as a type-neutral dynamic encoding and decoding +system. However, its close coupling with the Python text model, especially +the type restricted convenience methods on the builtin :class:`str`, +:class:`bytes` and :class:`bytearray` types, has historically obscured that +fact. + +As a key step in clarifying the situation, the :meth:`codecs.encode` and +:meth:`codecs.decode` convenience functions are now properly documented in +Python 2.7, 3.3 and 3.4. These functions have existed in the :mod:`codecs` +module (and have been covered by the regression test suite) since Python 2.4, +but were previously only discoverable through runtime introspection. + +Unlike the convenience methods on :class:`str`, :class:`bytes` and +:class:`bytearray`, these convenience functions support arbitrary codecs +in both Python 2 and Python 3, rather than being limited to Unicode text +encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions +(in Python 2). + +In Python 3.4, the interpreter is able to identify the known non-text +encodings provided in the standard library and direct users towards these +general purpose convenience functions when appropriate:: + + >>> b"abcdef".decode("hex") + Traceback (most recent call last): + File "", line 1, in + LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs + + >>> "hello".encode("rot13") + Traceback (most recent call last): + File "", line 1, in + LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs + +In a related change, whenever it is feasible without breaking backwards +compatibility, exceptions raised during encoding and decoding operations +will be wrapped in a chained exception of the same type that mentions the +name of the codec responsible for producing the error:: + + >>> import codecs + + >>> codecs.decode(b"abcdefgh", "hex") + binascii.Error: Non-hexadecimal digit found + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 1, in + binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found) + + >>> codecs.encode("hello", "bz2") + TypeError: 'str' does not support the buffer interface + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 1, in + TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface) + +Finally, as the examples above show, these improvements have permitted +the restoration of the convenience aliases for the non-Unicode codecs that +were themselves restored in Python 3.2. This means that encoding binary data +to and from its hexadecimal representation (for example) can now be written +as:: + + >>> from codecs import encode, decode + >>> encode(b"hello", "hex") + b'68656c6c6f' + >>> decode(b"68656c6c6f", "hex") + b'hello' + +The binary and text transforms provided in the standard library are detailed +in :ref:`binary-transforms` and :ref:`text-transforms`. + - (Contributed by Nick Coghlan in :issue:`7475`, , :issue:`17827`, ++(Contributed by Nick Coghlan in :issue:`7475`, :issue:`17827`, +:issue:`17828` and :issue:`19619`) + +.. _whatsnew-pep-451: + +PEP 451: A ModuleSpec Type for the Import System +------------------------------------------------ + +:pep:`451` provides an encapsulation of the information about a module that the +import machinery will use to load it (that is, a module specification). This +helps simplify both the import implementation and several import-related APIs. +The change is also a stepping stone for `several future import-related +improvements`__. + +__ https://mail.python.org/pipermail/python-dev/2013-November/130111.html + +The public-facing changes from the PEP are entirely backward-compatible. +Furthermore, they should be transparent to everyone but importer authors. Key +finder and loader methods have been deprecated, but they will continue working. +New importers should use the new methods described in the PEP. Existing +importers should be updated to implement the new methods. + + +Other Language Changes +---------------------- + +Some smaller changes made to the core Python language are: + +* Unicode database updated to UCD version 6.3. + +* :func:`min` and :func:`max` now accept a *default* argument that can be used + to specify the value they return if the iterable they are evaluating has no + elements. Contributed by Julian Berman in :issue:`18111`. + +* Module objects are now :mod:`weakref`'able. + +* Module ``__file__`` attributes (and related values) should now always + contain absolute paths by default, with the sole exception of + ``__main__.__file__`` when a script has been executed directly using + a relative path (Contributed by Brett Cannon in :issue:`18416`). + +* Now all the UTF-\* codecs (except UTF-7) reject surrogates during both + encoding and decoding unless the ``surrogatepass`` error handler is used, + with the exception of the UTF-16 decoder that accepts valid surrogate pairs, + and the UTF-16 encoder that produces them while encoding non-BMP characters. + Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in + :issue:`12892`. + + +New Modules +=========== + + +asyncio +------- + +The new :mod:`asyncio` module (defined in :pep:`3156`) provides a standard +pluggable event loop model for Python, providing solid asynchronous IO +support in the standard library, and making it easier for other event loop +implementations to interoperate with the standard library and each other. + +For Python 3.4, this module is considered a :term:`provisional API`. + +.. seealso:: + + :pep:`3156` - Asynchronous IO Support Rebooted: the "asyncio" Module + PEP written and implementation led by Guido van Rossum. + +enum +---- + +The new :mod:`enum` module (defined in :pep:`435`) provides a standard +implementation of enumeration types, allowing other modules (such as +:mod:`socket`) to provide more informative error messages and better +debugging support by replacing opaque integer constants with backwards +compatible enumeration values. + +.. seealso:: + + :pep:`435` - Adding an Enum type to the Python standard library + PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman, + implemented by Ethan Furman. + + +pathlib +------- + +The new :mod:`pathlib` module offers classes representing filesystem paths +with semantics appropriate for different operating systems. Path classes are +divided between *pure paths*, which provide purely computational operations +without I/O, and *concrete paths*, which inherit from pure paths but also +provide I/O operations. + +For Python 3.4, this module is considered a :term:`provisional API`. + +.. seealso:: + + :pep:`428` - The pathlib module -- object-oriented filesystem paths + PEP written and implemented by Antoine Pitrou. + + +selectors +--------- + +The new :mod:`selectors` module (created as part of implementing :pep:`3156`) +allows high-level and efficient I/O multiplexing, built upon the +:mod:`select` module primitives. + + +statistics +---------- + +The new :mod:`statistics` module (defined in :pep:`450`) offers some core +statistics functionality directly in the standard library. This module +supports calculation of the mean, median, mode, variance and standard +deviation of a data series. + +.. seealso:: + + :pep:`450` - Adding A Statistics Module To The Standard Library + PEP written and implemented by Steven D'Aprano + + +tracemalloc +----------- + +The new :mod:`tracemalloc` module (defined in :pep:`454`) is a debug tool to +trace memory blocks allocated by Python. It provides the following information: + +* Traceback where an object was allocated +* Statistics on allocated memory blocks per filename and per line number: + total size, number and average size of allocated memory blocks +* Compute the differences between two snapshots to detect memory leaks + +.. seealso:: + + :pep:`454` - Add a new tracemalloc module to trace Python memory allocations + PEP written and implemented by Victor Stinner + + +Improved Modules +================ + +aifc +---- + +The :meth:`~aifc.getparams` method now returns a namedtuple rather than a +plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.) + + +audioop +------- + +Added support for 24-bit samples (:issue:`12866`). + +Added the :func:`~audioop.byteswap` function to convert big-endian samples +to little-endian and vice versa (:issue:`19641`). + + +base64 +------ + +The encoding and decoding functions in :mod:`base64` now accept any +:term:`bytes-like object` in cases where it previously required a +:class:`bytes` or :class:`bytearray` instance (:issue:`17839`). + + +colorsys +-------- + +The number of digits in the coefficients for the RGB --- YIQ conversions have +been expanded so that they match the FCC NTSC versions. The change in +results should be less than 1% and may better match results found elsewhere. + + +contextlib +---------- + +The new :class:`contextlib.suppress` context manager helps to clarify the +intent of code that deliberately suppresses exceptions from a single +statement. (Contributed by Raymond Hettinger in :issue:`15806` and +Zero Piraeus in :issue:`19266`) + +The new :func:`contextlib.redirect_stdout` context manager makes it easier +for utility scripts to handle inflexible APIs that don't provide any +options to retrieve their output as a string or direct it to somewhere +other than :data:`sys.stdout`. In conjunction with :class:`io.StringIO`, +this context manager is also useful for checking expected output from +command line utilities. (Contribute by Raymond Hettinger in :issue:`15805`) + +The :mod:`contextlib` documentation has also been updated to include a +:ref:`discussion ` of the +differences between single use, reusable and reentrant context managers. + + +dis +--- + +The :mod:`dis` module is now built around an :class:`~dis.Instruction` class +that provides details of individual bytecode operations and a +:func:`~dis.get_instructions` iterator that emits the Instruction stream for a +given piece of Python code. The various display tools in the :mod:`dis` +module have been updated to be based on these new components. + +The new :class:`dis.Bytecode` class provides an object-oriented API for +inspecting bytecode, both in human-readable form and for iterating over +instructions. + +(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816` +and Claudiu Popa in :issue:`17916`) + + +doctest +------- + +Added :data:`~doctest.FAIL_FAST` flag to halt test running as soon as the first +failure is detected. (Contributed by R. David Murray and Daniel Urban in +:issue:`16522`.) + +Updated the doctest command line interface to use :mod:`argparse`, and added +``-o`` and ``-f`` options to the interface. ``-o`` allows doctest options to +be specified on the command line, and ``-f`` is a shorthand for ``-o +FAIL_FAST`` (to parallel the similar option supported by the :mod:`unittest` +CLI). (Contributed by R. David Murray in :issue:`11390`.) + + +email +----- + +:meth:`~email.message.Message.as_string` now accepts a *policy* argument to +override the default policy of the message when generating a string +representation of it. This means that ``as_string`` can now be used in more +circumstances, instead of having to create and use a :mod:`~email.generator` in +order to pass formatting parameters to its ``flatten`` method. + +New method :meth:`~email.message.Message.as_bytes` added to produce a bytes +representation of the message in a fashion similar to how ``as_string`` +produces a string representation. It does not accept the *maxheaderlen* +argument, but does accept the *unixfrom* and *policy* arguments. The +:class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method +calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive +result: a bytes object containing the fully formatted message. + +(Contributed by R. David Murray in :issue:`18600`.) + +.. _whatsnew_email_contentmanager: + +A pair of new subclasses of :class:`~email.message.Message` have been added, +along with a new sub-module, :mod:`~email.contentmanager`. All documentation +is currently in the new module, which is being added as part of the new +:term:`provisional ` email API. These classes provide a +number of new methods that make extracting content from and inserting content +into email messages much easier. See the :mod:`~email.contentmanager` +documentation for details. + +These API additions complete the bulk of the work that was planned as part of +the email6 project. The currently provisional API is scheduled to become final +in Python 3.5 (possibly with a few minor additions in the area of error +handling). + +(Contributed by R. David Murray in :issue:`18891`.) + + +functools +--------- + +The new :func:`~functools.partialmethod` descriptor bring partial argument +application to descriptors, just as :func:`~functools.partial` provides +for normal callables. The new descriptor also makes it easier to get +arbitrary callables (including :func:`~functools.partial` instances) +to behave like normal instance methods when included in a class definition. + +(Contributed by Alon Horev and Nick Coghlan in :issue:`4331`) + +.. _whatsnew-singledispatch: + +The new :func:`~functools.singledispatch` decorator brings support for +single-dispatch generic functions to the Python standard library. Where +object oriented programming focuses on grouping multiple operations on a +common set of data into a class, a generic function focuses on grouping +multiple implementations of an operation that allows it to work with +*different* kinds of data. + +.. seealso:: + + :pep:`443` - Single-dispatch generic functions + PEP written and implemented by Łukasz Langa. + + +hashlib +------- + +New :func:`hashlib.pbkdf2_hmac` function. +(Contributed by Christian Heimes in :issue:`18582`) + +.. _whatsnew-sha3: + +New :ref:`hash algorithms ` ``sah3_224()``, ``sha3_256()``, +``sha3_384()``, and ``sha3_512()``. (Contributed by Christian Heimes in +:issue:`16113`.) + + +html +---- + +Added a new :func:`html.unescape` function that converts HTML5 character +references to the corresponding Unicode characters. +(Contributed by Ezio Melotti in :issue:`2927`) + +Added a new *convert_charrefs* keyword argument to +:class:`~html.parser.HTMLParser` that, when ``True``, automatically converts +all character references. For backward-compatibility, its value defaults +to ``False``, but it will change to ``True`` in future versions, so you +are invited to set it explicitly and update your code to use this new feature. +(Contributed by Ezio Melotti in :issue:`13633`) + +The *strict* argument of :class:`~html.parser.HTMLParser` is now deprecated. +(Contributed by Ezio Melotti in :issue:`15114`) + + +inspect +------- + + +The inspect module now offers a basic :ref:`command line interface +` to quickly display source code and other +information for modules, classes and functions. (Contributed by Claudiu Popa +and Nick Coghlan in :issue:`18626`) + +:func:`~inspect.unwrap` makes it easy to unravel wrapper function chains +created by :func:`functools.wraps` (and any other API that sets the +``__wrapped__`` attribute on a wrapper function). (Contributed by +Daniel Urban, Aaron Iles and Nick Coghlan in :issue:`13266`) + +As part of the implementation of the new :mod:`enum` module, the +:mod:`inspect` module now has substantially better support for custom +``__dir__`` methods and dynamic class attributes provided through +metaclasses (Contributed by Ethan Furman in :issue:`18929` and +:issue:`19030`) + + +.. _whatsnew-marshal-3: + +marshal +------- + +The default :mod:`marshal` version has been bumped to 3. The code implementing +the new version restores the Python2 behavior of recording only one copy of +interned strings and preserving the interning on deserialization, and extends +this "one copy" ability to any object type (including handling recursive +references). This reduces both the size of ``.pyc`` files and the amount of +memory a module occupies in memory when it is loaded from a ``.pyc`` (or +``.pyo``) file. (Contributed by Kristján Valur Jónsson in :issue:`16475`.) + + +mmap +---- + +mmap objects can now be weakref'ed. +(Contributed by Valerie Lambert in :issue:`4885`.) + + +multiprocessing +--------------- + +.. _whatsnew-multiprocessing-no-fork: + +On Unix, two new :ref:`start methods ` +(``spawn`` and ``forkserver``) have been added for starting processes using +:mod:`multiprocessing`. These make the mixing of processes with threads more +robust, and the ``spawn`` method matches the semantics that multiprocessing has +always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`). + +Also, except when using the old *fork* start method, child processes +will no longer inherit unneeded handles/file descriptors from their parents +(part of :issue:`8713`). + +:mod:`multiprocessing` now relies on :mod:`runpy` (which implements the +``-m`` switch) to initialise ``__main__`` appropriately in child processes +when using the ``spawn`` or ``forkserver`` start methods. This resolves some +edge cases where combining multiprocessing, the ``-m`` command line switch +and explicit relative imports could cause obscure failures in child +processes. (Contributed by Nick Coghlan in :issue:`19946`) + + +os +-- + +New functions to get and set the :ref:`inheritable flag ` of a file +descriptors or a Windows handle: + +* :func:`os.get_inheritable`, :func:`os.set_inheritable` +* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` + + +pdb +--- + +The ``print`` command has been removed from :mod:`pdb`, restoring access to the +``print`` function. + +Rationale: Python2's ``pdb`` did not have a ``print`` command; instead, +entering ``print`` executed the ``print`` statement. In Python3 ``print`` was +mistakenly made an alias for the pdb :pdbcmd:`p` command. ``p``, however, +prints the ``repr`` of its argument, not the ``str`` like the Python2 ``print`` +command did. Worse, the Python3 ``pdb print`` command shadowed the Python3 +``print`` function, making it inaccessible at the ``pdb`` prompt. + +(Contributed by Connor Osborn in :issue:`18764`.) + + +.. _whatsnew-protocol-4: + +Pickle +------ + +protocol 4 + +:mod:`pickle` now supports (but does not use by default) a new pickle protocol, +protocol 4. This new protocol addresses a number of issues that were present +in previous protocols, such as the serialization of nested classes, very large +strings and containers, or classes whose :meth:`__new__` method takes +keyword-only arguments. It also provides some efficiency improvements. + +.. seealso:: + + :pep:`3154` - Pickle protocol 4 + PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti. + + +poplib +------ + +New :meth:`~poplib.POP3.stls` method to switch a clear-text POP3 session into +an encrypted POP3 session. + +New :meth:`~poplib.POP3.capa` method to query the capabilities advertised by the +POP3 server. + +(Contributed by Lorenzo Catucci in :issue:`4473`.) + + +pprint +------ + +The :mod:`pprint` module now supports *compact* mode for formatting long +sequences (:issue:`19132`). + + +pydoc +----- + +While significant changes have not been made to :mod:`pydoc` directly, +its handling of custom ``__dir__`` methods and various descriptor +behaviours has been improved substantially by the underlying changes in +the :mod:`inspect` module. + + +re +-- + +Added :func:`re.fullmatch` function and :meth:`regex.fullmatch` method, +which anchor the pattern at both ends of the string to match. +(Contributed by Matthew Barnett in :issue:`16203`.) + +The repr of :ref:`regex objects ` now includes the pattern +and the flags; the repr of :ref:`match objects ` now +includes the start, end, and the part of the string that matched. +(Contributed by Serhiy Storchaka in :issue:`13592` and :issue:`17087`.) + + +resource +-------- + +New :func:`resource.prlimit` function and Linux specific constants. +(Contributed by Christian Heimes in :issue:`16595` and :issue:`19324`.) + +smtplib +------- + +:exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows +both socket level errors and SMTP protocol level errors to be caught in one +try/except statement by code that only cares whether or not an error occurred. +(:issue:`2118`). + + +socket +------ + +Socket objects have new methods to get or set their :ref:`inheritable flag +`: + +* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable` + +The ``socket.AF_*`` and ``socket.SOCK_*`` constants are enumeration values, +using the new :mod:`enum` module. This allows descriptive reporting during +debugging, instead of seeing integer "magic numbers". + +ssl +--- + +.. _whatsnew-tls-11-12: + +:data:`~ssl.PROTOCOL_TLSv1_1` and :data:`~ssl.PROTOCOL_TLSv1_2` (TLSv1.1 and +TLSv1.2 support) have been added; support for these protocols is only available if +Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orrù and +Antoine Pitrou in :issue:`16692`) + +New diagnostic functions :func:`~ssl.get_default_verify_paths`, +:meth:`~ssl.SSLContext.cert_store_stats` and +:meth:`~ssl.SSLContext.get_ca_certs` (Contributed by Christian Heimes +in :issue:`18143` and :issue:`18147`) + +Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows' +cert store. (Contributed by Christian Heimes in :issue:`17134`.) + +Support for server-side SNI using the new +:meth:`ssl.SSLContext.set_servername_callback` method. +(Contributed by Daniel Black in :issue:`8109`.) + + +stat +---- + +The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C +implementation is required as most of the values aren't standardized and +platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) + +The module supports new file types: door, event port and whiteout. + + +struct +------ + +Streaming struct unpacking using :func:`struct.iter_unpack`. +(Contributed by Antoine Pitrou in :issue:`17804`.) + + +sunau +----- + +The :meth:`~sunau.getparams` method now returns a namedtuple rather than a +plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.) + +:meth:`sunau.open` now supports the context manager protocol (:issue:`18878`). + + +traceback +--------- + +A new :func:`traceback.clear_frames` function takes a traceback object +and clears the local variables in all of the frames it references, +reducing the amount of memory consumed (:issue:`1565525`). + + +urllib +------ + +Add support.for ``data:`` URLs in :mod:`urllib.request`. +(Contributed by Mathias Panzenböck in :issue:`16423`.) + + +unittest +-------- + +Support for easy dynamically-generated subtests using the +:meth:`~unittest.TestCase.subTest` context manager. +(Contributed by Antoine Pitrou in :issue:`16997`.) + + +wave +---- + +The :meth:`~wave.getparams` method now returns a namedtuple rather than a +plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.) + +:meth:`wave.open` now supports the context manager protocol. (Contributed +by Claudiu Popa in :issue:`17616`.) + + +weakref +------- + +New :class:`~weakref.WeakMethod` class simulates weak references to bound +methods. (Contributed by Antoine Pitrou in :issue:`14631`.) + +New :class:`~weakref.finalize` class makes it possible to register a callback +to be invoked when an object is garbage collected, without needing to +carefully manage the lifecycle of the weak reference itself. (Contributed by +Richard Oudkerk in :issue:`15528`) + + +xml.etree +--------- + +Add an event-driven parser for non-blocking applications, +:class:`~xml.etree.ElementTree.XMLPullParser`. +(Contributed by Antoine Pitrou in :issue:`17741`.) + + +zipfile.PyZipfile +----------------- + +Add a filter function to ignore some packages (tests for instance), +:meth:`~zipfile.PyZipFile.writepy`. +(Contributed by Christian Tismer in :issue:`19274`.) + + +Other improvements +================== + +Tab-completion is now enabled by default in the interactive interpreter. +(Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.) + +Python invocation changes +========================= + +Invoking the Python interpreter with ``--version`` now outputs the version to +standard output instead of standard error (:issue:`18338`). Similar changes +were made to :mod:`argparse` (:issue:`18920`) and other modules that have +script-like invocation capabilities (:issue:`18922`). + +Optimizations +============= + +Major performance enhancements have been added: + +* The UTF-32 decoder is now 3x to 4x faster. + +* The cost of hash collisions for sets is now reduced. Each hash table + probe now checks a series of consecutive, adjacent key/hash pairs before + continuing to make random probes through the hash table. This exploits + cache locality to make collision resolution less expensive. + + The collision resolution scheme can be described as a hybrid of linear + probing and open addressing. The number of additional linear probes + defaults to nine. This can be changed at compile-time by defining + LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off + linear probing entirely. + + (Contributed by Raymond Hettinger in :issue:`18771`.) + +* The interpreter starts about 30% faster. A couple of measures lead to the + speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`, + :mod:`collections` and :mod:`locale` modules and their dependencies are no + longer imported by default. The marshal module has been improved to load + compiled Python code faster. + + (Contributed by Antoine Pitrou, Christian Heimes and Victor Stinner in + :issue:`19219`, :issue:`19218`, :issue:`19209`, :issue:`19205` and + :issue:`9548`) + + +CPython Implementation Changes +============================== + + +.. _whatsnew-pep-445: + +PEP 445: Customization of CPython memory allocators +--------------------------------------------------- + +:pep:`445` adds new C level interfaces to customize memory allocation in +the CPython interpreter. + +.. seealso:: + + :pep:`445` - Add new APIs to customize Python memory allocators + PEP written and implemented by Victor Stinner. + + +.. _whatsnew-pep-442: + +PEP 442: Safe object finalization +--------------------------------- + +:pep:`442` removes the current limitations and quirks of object finalization +in CPython. With it, objects with :meth:`__del__` methods, as well as +generators with :keyword:`finally` clauses, can be finalized when they are +part of a reference cycle. + +As part of this change, module globals are no longer forcibly set to +:const:`None` during interpreter shutdown in most cases, instead relying +on the normal operation of the cyclic garbage collector. This avoids a +whole class of interpreter-shutdown-time errors, usually involving +``__del__`` methods, that have plagued Python since the cyclic GC +was first introduced. + +.. seealso:: + + :pep:`442` - Safe object finalization + PEP written and implemented by Antoine Pitrou. + + +.. _whatsnew-pep-456: + +PEP 456: Secure and Interchangeable Hash Algorithm +-------------------------------------------------- + +:pep:`456` follows up on earlier security fix work done on Python's hash +algorithm to address certain DOS attacks to which public facing APIs backed by +dictionary lookups may be subject. (See :issue:`14621` for the start of the +current round of improvements.) The PEP unifies CPython's hash code to make it +easier for a packager to substitute a different hash algorithm, and switches +Python's default implementation to a SipHash implementation on platforms that +have a 64 bit data type. Any performance differences in comparison with the +older FNV algorithm are trivial. + +The PEP adds additional fields to the :func:`sys.hash_info` struct sequence to +describe the hash algorithm in use by the currently executing binary. Otherwise, +the PEP does not alter any existing CPython APIs. + + +Other build and C API changes +----------------------------- + +Changes to Python's build process and to the C API include: + +* The new :c:func:`Py_SetStandardStreamEncoding` pre-initialization API + allows applications embedding the CPython interpreter to reliably force + a particular encoding and error handler for the standard streams + (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`) + +* Most Python C APIs that don't mutate string arguments are now correctly + marked as accepting ``const char *`` rather than ``char *`` (Contributed + by Serhiy Storchaka in :issue:`1772673`). + +.. _whatsnew-pep-436: + +* "Argument Clinic" (:pep:`436`) is now part of the CPython build process + and can be used to simplify the process of defining and maintaining + accurate signatures for builtins and standard library extension modules + implemented in C. + + .. note:: + The Argument Clinic PEP is not fully up to date with the state of the + implementation. This has been deemed acceptable by the release manager + and core development team in this case, as Argument Clinic will not + be made available as a public API for third party use in Python 3.4. + + +Deprecated +========== + +Unsupported Operating Systems +----------------------------- + +* OS/2 +* Windows 2000 +* VMS + + +Deprecated Python modules, functions and methods +------------------------------------------------ + +* :meth:`difflib.SequenceMatcher.isbjunk` and + :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and + ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object. + +* :func:`importlib.util.module_for_loader` is pending deprecation. Using + :func:`importlib.util.module_to_load` and + :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader + to more easily customize module loading. + +* The :mod:`imp` module is pending deprecation. To keep compatibility with + Python 2/3 code bases, the module's removal is currently not scheduled. + +* The :mod:`formatter` module is pending deprecation and is slated for removal + in Python 3.6. + +* MD5 as default digestmod for :mod:`hmac` is deprecated. Python 3.6 will + require an explicit digest name or constructor as *digestmod* argument. + + +Deprecated functions and types of the C API +------------------------------------------- + +* The ``PyThreadState.tick_counter`` field has been removed: its value was + meaningless since Python 3.2 ("new GIL"). + + +Deprecated features +------------------- + +* The site module adding a "site-python" directory to sys.path, if it + exists, is deprecated (:issue:`19375`). + + +Porting to Python 3.4 +===================== + +Changes in the Python API +------------------------- + +This section lists previously described changes and other bugfixes +that may require changes to your code. + +* The ABCs defined in :mod:`importlib.abc` now either raise the appropriate + exception or return a default value instead of raising + :exc:`NotImplementedError` blindly. This will only affect code calling + :func:`super` and falling through all the way to the ABCs. For compatibility, + catch both :exc:`NotImplementedError` or the appropriate exception as needed. + +* The module type now initializes the :attr:`__package__` and :attr:`__loader__` + attributes to ``None`` by default. To determine if these attributes were set + in a backwards-compatible fashion, use e.g. + ``getattr(module, '__loader__', None) is not None``. + +* :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and + ``__package__`` unconditionally to properly support reloading. If this is not + desired then you will need to set these attributes manually. You can use + :func:`importlib.util.module_to_load` for module management. + +* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``, + ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading. + +* Frozen packages no longer set ``__path__`` to a list containing the package + name but an empty list instead. Determing if a module is a package should be + done using ``hasattr(module, '__path__')``. + +* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path + it would write to is a symlink or a non-regular file. This is to act as a + warning that import will overwrite those files with a regular file regardless + of what type of file path they were originally. + +* :meth:`importlib.abc.SourceLoader.get_source` no longer raises + :exc:`ImportError` when the source code being loaded triggers a + :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is + meant to be raised only when source code cannot be found but it should, it was + felt to be over-reaching/overloading of that meaning when the source code is + found but improperly structured. If you were catching ImportError before and + wish to continue to ignore syntax or decoding issues, catch all three + exceptions now. + +* :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly + set the ``__wrapped__`` attribute to the function being wrapper, even if + that function also had its ``__wrapped__`` attribute set. This means + ``__wrapped__`` attributes now correctly link a stack of decorated + functions rather than every ``__wrapped__`` attribute in the chain + referring to the innermost function. Introspection libraries that + assumed the previous behaviour was intentional can use + :func:`inspect.unwrap` to access the first function in the chain that has + no ``__wrapped__`` attribute. + +* :class:`importlib.machinery.PathFinder` now passes on the current working + directory to objects in :data:`sys.path_hooks` for the empty string. This + results in :data:`sys.path_importer_cache` never containing ``''``, thus + iterating through :data:`sys.path_importer_cache` based on :data:`sys.path` + will not find all keys. A module's ``__file__`` when imported in the current + working directory will also now have an absolute path, including when using + ``-m`` with the interpreter (this does not influence when the path to a file + is specified on the command-line). + +Changes in the C API +-------------------- + +* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** + argument is not set. Previously only ``NULL`` was returned with no exception + set. + +* The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must + now be a string allocated by :c:func:`PyMem_RawMalloc` or + :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a + string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. + +* :c:func:`PyThread_set_key_value` now always set the value. In Python + 3.3, the function did nothing if the key already exists (if the current + value is a non-NULL pointer). + +* The ``f_tstate`` (thread state) field of the :c:type:`PyFrameObject` + structure has been removed to fix a bug: see :issue:`14432` for the + rationale. +