From: Serhiy Storchaka Date: Sun, 13 Oct 2013 20:12:09 +0000 (+0300) Subject: Issue #18758: Fixed and improved cross-references. X-Git-Tag: v3.4.0a4~116 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=98b28fddd8e63e1a9410facee3e41925eed8ac46;p=python Issue #18758: Fixed and improved cross-references. --- 98b28fddd8e63e1a9410facee3e41925eed8ac46 diff --cc Doc/library/enum.rst index f09c60c00d,0000000000..c2030faf8b mode 100644,000000..100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@@ -1,688 -1,0 +1,688 @@@ +:mod:`enum` --- Support for enumerations +======================================== + +.. module:: enum + :synopsis: Implementation of an enumeration class. + +.. :moduleauthor:: Ethan Furman +.. :sectionauthor:: Barry Warsaw , +.. :sectionauthor:: Eli Bendersky , +.. :sectionauthor:: Ethan Furman + +**Source code:** :source:`Lib/enum.py` + +---------------- + +An enumeration is a set of symbolic names (members) bound to unique, constant +values. Within an enumeration, the members can be compared by identity, and +the enumeration itself can be iterated over. + +This module defines two enumeration classes that can be used to define unique +sets of names and values: :class:`Enum` and :class:`IntEnum`. It also defines +one decorator, :func:`unique`, that ensures only unique member values are +present in an enumeration. + + +Creating an Enum +---------------- + +Enumerations are created using the :keyword:`class` syntax, which makes them +easy to read and write. An alternative creation method is described in +`Functional API`_. To define an enumeration, subclass :class:`Enum` as +follows:: + + >>> from enum import Enum + >>> class Color(Enum): + ... red = 1 + ... green = 2 + ... blue = 3 + ... + +.. note:: Nomenclature + + - The class :class:`Color` is an *enumeration* (or *enum*) + - The attributes :attr:`Color.red`, :attr:`Color.green`, etc., are + *enumeration members* (or *enum members*). + - The enum members have *names* and *values* (the name of + :attr:`Color.red` is ``red``, the value of :attr:`Color.blue` is + ``3``, etc.) + +.. note:: + + Even though we use the :keyword:`class` syntax to create Enums, Enums + are not normal Python classes. See `How are Enums different?`_ for + more details. + +Enumeration members have human readable string representations:: + + >>> print(Color.red) + Color.red + +...while their ``repr`` has more information:: + + >>> print(repr(Color.red)) + + +The *type* of an enumeration member is the enumeration it belongs to:: + + >>> type(Color.red) + + >>> isinstance(Color.green, Color) + True + >>> + +Enum members also have a property that contains just their item name:: + + >>> print(Color.red.name) + red + +Enumerations support iteration, in definition order:: + + >>> class Shake(Enum): + ... vanilla = 7 + ... chocolate = 4 + ... cookies = 9 + ... mint = 3 + ... + >>> for shake in Shake: + ... print(shake) + ... + Shake.vanilla + Shake.chocolate + Shake.cookies + Shake.mint + +Enumeration members are hashable, so they can be used in dictionaries and sets:: + + >>> apples = {} + >>> apples[Color.red] = 'red delicious' + >>> apples[Color.green] = 'granny smith' + >>> apples == {Color.red: 'red delicious', Color.green: 'granny smith'} + True + + +Programmatic access to enumeration members and their attributes +--------------------------------------------------------------- + +Sometimes it's useful to access members in enumerations programmatically (i.e. +situations where ``Color.red`` won't do because the exact color is not known +at program-writing time). ``Enum`` allows such access:: + + >>> Color(1) + + >>> Color(3) + + +If you want to access enum members by *name*, use item access:: + + >>> Color['red'] + + >>> Color['green'] + + +If have an enum member and need its :attr:`name` or :attr:`value`:: + + >>> member = Color.red + >>> member.name + 'red' + >>> member.value + 1 + + +Duplicating enum members and values +----------------------------------- + +Having two enum members with the same name is invalid:: + + >>> class Shape(Enum): + ... square = 2 + ... square = 3 + ... + Traceback (most recent call last): + ... + TypeError: Attempted to reuse key: 'square' + +However, two enum members are allowed to have the same value. Given two members +A and B with the same value (and A defined first), B is an alias to A. By-value +lookup of the value of A and B will return A. By-name lookup of B will also +return A:: + + >>> class Shape(Enum): + ... square = 2 + ... diamond = 1 + ... circle = 3 + ... alias_for_square = 2 + ... + >>> Shape.square + + >>> Shape.alias_for_square + + >>> Shape(2) + + +.. note:: + + Attempting to create a member with the same name as an already + defined attribute (another member, a method, etc.) or attempting to create + an attribute with the same name as a member is not allowed. + + +Ensuring unique enumeration values +---------------------------------- + +By default, enumerations allow multiple names as aliases for the same value. +When this behavior isn't desired, the following decorator can be used to +ensure each value is used only once in the enumeration: + +.. decorator:: unique + +A :keyword:`class` decorator specifically for enumerations. It searches an +enumeration's :attr:`__members__` gathering any aliases it finds; if any are +found :exc:`ValueError` is raised with the details:: + + >>> from enum import Enum, unique + >>> @unique + ... class Mistake(Enum): + ... one = 1 + ... two = 2 + ... three = 3 + ... four = 3 + ... + Traceback (most recent call last): + ... + ValueError: duplicate values found in : four -> three + + +Iteration +--------- + +Iterating over the members of an enum does not provide the aliases:: + + >>> list(Shape) + [, , ] + +The special attribute ``__members__`` is an ordered dictionary mapping names +to members. It includes all names defined in the enumeration, including the +aliases:: + + >>> for name, member in Shape.__members__.items(): + ... name, member + ... + ('square', ) + ('diamond', ) + ('circle', ) + ('alias_for_square', ) + +The ``__members__`` attribute can be used for detailed programmatic access to +the enumeration members. For example, finding all the aliases:: + + >>> [name for name, member in Shape.__members__.items() if member.name != name] + ['alias_for_square'] + + +Comparisons +----------- + +Enumeration members are compared by identity:: + + >>> Color.red is Color.red + True + >>> Color.red is Color.blue + False + >>> Color.red is not Color.blue + True + +Ordered comparisons between enumeration values are *not* supported. Enum +members are not integers (but see `IntEnum`_ below):: + + >>> Color.red < Color.blue + Traceback (most recent call last): + File "", line 1, in + TypeError: unorderable types: Color() < Color() + +Equality comparisons are defined though:: + + >>> Color.blue == Color.red + False + >>> Color.blue != Color.red + True + >>> Color.blue == Color.blue + True + +Comparisons against non-enumeration values will always compare not equal +(again, :class:`IntEnum` was explicitly designed to behave differently, see +below):: + + >>> Color.blue == 2 + False + + +Allowed members and attributes of enumerations +---------------------------------------------- + +The examples above use integers for enumeration values. Using integers is +short and handy (and provided by default by the `Functional API`_), but not +strictly enforced. In the vast majority of use-cases, one doesn't care what +the actual value of an enumeration is. But if the value *is* important, +enumerations can have arbitrary values. + +Enumerations are Python classes, and can have methods and special methods as +usual. If we have this enumeration:: + + >>> class Mood(Enum): + ... funky = 1 + ... happy = 3 + ... + ... def describe(self): + ... # self is the member here + ... return self.name, self.value + ... + ... def __str__(self): + ... return 'my custom str! {0}'.format(self.value) + ... + ... @classmethod + ... def favorite_mood(cls): + ... # cls here is the enumeration + ... return cls.happy + ... + +Then:: + + >>> Mood.favorite_mood() + + >>> Mood.happy.describe() + ('happy', 3) + >>> str(Mood.funky) + 'my custom str! 1' + +The rules for what is allowed are as follows: _sunder_ names (starting and +ending with a single underscore) are reserved by enum and cannot be used; +all other attributes defined within an enumeration will become members of this +enumeration, with the exception of *__dunder__* names and descriptors (methods +are also descriptors). + +Note: if your enumeration defines :meth:`__new__` and/or :meth:`__init__` then +whatever value(s) were given to the enum member will be passed into those +methods. See `Planet`_ for an example. + + +Restricted subclassing of enumerations +-------------------------------------- + +Subclassing an enumeration is allowed only if the enumeration does not define +any members. So this is forbidden:: + + >>> class MoreColor(Color): + ... pink = 17 + ... + Traceback (most recent call last): + ... + TypeError: Cannot extend enumerations + +But this is allowed:: + + >>> class Foo(Enum): + ... def some_behavior(self): + ... pass + ... + >>> class Bar(Foo): + ... happy = 1 + ... sad = 2 + ... + +Allowing subclassing of enums that define members would lead to a violation of +some important invariants of types and instances. On the other hand, it makes +sense to allow sharing some common behavior between a group of enumerations. +(See `OrderedEnum`_ for an example.) + + +Pickling +-------- + +Enumerations can be pickled and unpickled:: + + >>> from test.test_enum import Fruit + >>> from pickle import dumps, loads + >>> Fruit.tomato is loads(dumps(Fruit.tomato)) + True + +The usual restrictions for pickling apply: picklable enums must be defined in +the top level of a module, since unpickling requires them to be importable +from that module. + +.. warning:: + + In order to support the singleton nature of enumeration members, pickle + protocol version 2 or higher must be used. + + +Functional API +-------------- + +The :class:`Enum` class is callable, providing the following functional API:: + + >>> Animal = Enum('Animal', 'ant bee cat dog') + >>> Animal + + >>> Animal.ant + + >>> Animal.ant.value + 1 + >>> list(Animal) + [, , , ] + - The semantics of this API resemble :class:`namedtuple`. The first argument - of the call to :class:`Enum` is the name of the enumeration. ++The semantics of this API resemble :class:`~collections.namedtuple`. The first ++argument of the call to :class:`Enum` is the name of the enumeration. + +The second argument is the *source* of enumeration member names. It can be a +whitespace-separated string of names, a sequence of names, a sequence of +2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to +values. The last two options enable assigning arbitrary values to +enumerations; the others auto-assign increasing integers starting with 1. A +new class derived from :class:`Enum` is returned. In other words, the above +assignment to :class:`Animal` is equivalent to:: + + >>> class Animals(Enum): + ... ant = 1 + ... bee = 2 + ... cat = 3 + ... dog = 4 + ... + +The reason for defaulting to ``1`` as the starting number and not ``0`` is +that ``0`` is ``False`` in a boolean sense, but enum members all evaluate +to ``True``. + +Pickling enums created with the functional API can be tricky as frame stack +implementation details are used to try and figure out which module the +enumeration is being created in (e.g. it will fail if you use a utility +function in separate module, and also may not work on IronPython or Jython). +The solution is to specify the module name explicitly as follows:: + + >>> Animals = Enum('Animals', 'ant bee cat dog', module=__name__) + +Derived Enumerations +-------------------- + +IntEnum +^^^^^^^ + +A variation of :class:`Enum` is provided which is also a subclass of +:class:`int`. Members of an :class:`IntEnum` can be compared to integers; +by extension, integer enumerations of different types can also be compared +to each other:: + + >>> from enum import IntEnum + >>> class Shape(IntEnum): + ... circle = 1 + ... square = 2 + ... + >>> class Request(IntEnum): + ... post = 1 + ... get = 2 + ... + >>> Shape == 1 + False + >>> Shape.circle == 1 + True + >>> Shape.circle == Request.post + True + +However, they still can't be compared to standard :class:`Enum` enumerations:: + + >>> class Shape(IntEnum): + ... circle = 1 + ... square = 2 + ... + >>> class Color(Enum): + ... red = 1 + ... green = 2 + ... + >>> Shape.circle == Color.red + False + +:class:`IntEnum` values behave like integers in other ways you'd expect:: + + >>> int(Shape.circle) + 1 + >>> ['a', 'b', 'c'][Shape.circle] + 'b' + >>> [i for i in range(Shape.square)] + [0, 1] + +For the vast majority of code, :class:`Enum` is strongly recommended, +since :class:`IntEnum` breaks some semantic promises of an enumeration (by +being comparable to integers, and thus by transitivity to other +unrelated enumerations). It should be used only in special cases where +there's no other choice; for example, when integer constants are +replaced with enumerations and backwards compatibility is required with code +that still expects integers. + + +Others +^^^^^^ + +While :class:`IntEnum` is part of the :mod:`enum` module, it would be very +simple to implement independently:: + + class IntEnum(int, Enum): + pass + +This demonstrates how similar derived enumerations can be defined; for example +a :class:`StrEnum` that mixes in :class:`str` instead of :class:`int`. + +Some rules: + +1. When subclassing :class:`Enum`, mix-in types must appear before + :class:`Enum` itself in the sequence of bases, as in the :class:`IntEnum` + example above. +2. While :class:`Enum` can have members of any type, once you mix in an + additional type, all the members must have values of that type, e.g. + :class:`int` above. This restriction does not apply to mix-ins which only + add methods and don't specify another data type such as :class:`int` or + :class:`str`. +3. When another data type is mixed in, the :attr:`value` attribute is *not the + same* as the enum member itself, although it is equivalant and will compare + equal. +4. %-style formatting: `%s` and `%r` call :class:`Enum`'s :meth:`__str__` and + :meth:`__repr__` respectively; other codes (such as `%i` or `%h` for + IntEnum) treat the enum member as its mixed-in type. +5. :meth:`str.__format__` (or :func:`format`) will use the mixed-in + type's :meth:`__format__`. If the :class:`Enum`'s :func:`str` or + :func:`repr` is desired use the `!s` or `!r` :class:`str` format codes. + + +Interesting examples +-------------------- + +While :class:`Enum` and :class:`IntEnum` are expected to cover the majority of +use-cases, they cannot cover them all. Here are recipes for some different +types of enumerations that can be used directly, or as examples for creating +one's own. + + +AutoNumber +^^^^^^^^^^ + +Avoids having to specify the value for each enumeration member:: + + >>> class AutoNumber(Enum): + ... def __new__(cls): + ... value = len(cls.__members__) + 1 + ... obj = object.__new__(cls) + ... obj._value_ = value + ... return obj + ... + >>> class Color(AutoNumber): + ... red = () + ... green = () + ... blue = () + ... + >>> Color.green.value == 2 + True + +.. note:: + + The :meth:`__new__` method, if defined, is used during creation of the Enum + members; it is then replaced by Enum's :meth:`__new__` which is used after + class creation for lookup of existing members. Due to the way Enums are + supposed to behave, there is no way to customize Enum's :meth:`__new__`. + + +OrderedEnum +^^^^^^^^^^^ + +An ordered enumeration that is not based on :class:`IntEnum` and so maintains +the normal :class:`Enum` invariants (such as not being comparable to other +enumerations):: + + >>> class OrderedEnum(Enum): + ... def __ge__(self, other): + ... if self.__class__ is other.__class__: + ... return self.value >= other.value + ... return NotImplemented + ... def __gt__(self, other): + ... if self.__class__ is other.__class__: + ... return self.value > other.value + ... return NotImplemented + ... def __le__(self, other): + ... if self.__class__ is other.__class__: + ... return self.value <= other.value + ... return NotImplemented + ... def __lt__(self, other): + ... if self.__class__ is other.__class__: + ... return self.value < other.value + ... return NotImplemented + ... + >>> class Grade(OrderedEnum): + ... A = 5 + ... B = 4 + ... C = 3 + ... D = 2 + ... F = 1 + ... + >>> Grade.C < Grade.A + True + + +DuplicateFreeEnum +^^^^^^^^^^^^^^^^^ + +Raises an error if a duplicate member name is found instead of creating an +alias:: + + >>> class DuplicateFreeEnum(Enum): + ... def __init__(self, *args): + ... cls = self.__class__ + ... if any(self.value == e.value for e in cls): + ... a = self.name + ... e = cls(self.value).name + ... raise ValueError( + ... "aliases not allowed in DuplicateFreeEnum: %r --> %r" + ... % (a, e)) + ... + >>> class Color(DuplicateFreeEnum): + ... red = 1 + ... green = 2 + ... blue = 3 + ... grene = 2 + ... + Traceback (most recent call last): + ... + ValueError: aliases not allowed in DuplicateFreeEnum: 'grene' --> 'green' + +.. note:: + + This is a useful example for subclassing Enum to add or change other + behaviors as well as disallowing aliases. If the only desired change is + disallowing aliases, the :func:`unique` decorator can be used instead. + + +Planet +^^^^^^ + +If :meth:`__new__` or :meth:`__init__` is defined the value of the enum member +will be passed to those methods:: + + >>> class Planet(Enum): + ... MERCURY = (3.303e+23, 2.4397e6) + ... VENUS = (4.869e+24, 6.0518e6) + ... EARTH = (5.976e+24, 6.37814e6) + ... MARS = (6.421e+23, 3.3972e6) + ... JUPITER = (1.9e+27, 7.1492e7) + ... SATURN = (5.688e+26, 6.0268e7) + ... URANUS = (8.686e+25, 2.5559e7) + ... NEPTUNE = (1.024e+26, 2.4746e7) + ... def __init__(self, mass, radius): + ... self.mass = mass # in kilograms + ... self.radius = radius # in meters + ... @property + ... def surface_gravity(self): + ... # universal gravitational constant (m3 kg-1 s-2) + ... G = 6.67300E-11 + ... return G * self.mass / (self.radius * self.radius) + ... + >>> Planet.EARTH.value + (5.976e+24, 6378140.0) + >>> Planet.EARTH.surface_gravity + 9.802652743337129 + + +How are Enums different? +------------------------ + +Enums have a custom metaclass that affects many aspects of both derived Enum +classes and their instances (members). + + +Enum Classes +^^^^^^^^^^^^ + +The :class:`EnumMeta` metaclass is responsible for providing the +:meth:`__contains__`, :meth:`__dir__`, :meth:`__iter__` and other methods that +allow one to do things with an :class:`Enum` class that fail on a typical +class, such as `list(Color)` or `some_var in Color`. :class:`EnumMeta` is +responsible for ensuring that various other methods on the final :class:`Enum` +class are correct (such as :meth:`__new__`, :meth:`__getnewargs__`, +:meth:`__str__` and :meth:`__repr__`) + + +Enum Members (aka instances) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The most interesting thing about Enum members is that they are singletons. +:class:`EnumMeta` creates them all while it is creating the :class:`Enum` +class itself, and then puts a custom :meth:`__new__` in place to ensure +that no new ones are ever instantiated by returning only the existing +member instances. + + +Finer Points +^^^^^^^^^^^^ + +Enum members are instances of an Enum class, and even though they are +accessible as `EnumClass.member`, they are not accessible directly from +the member:: + + >>> Color.red + + >>> Color.red.blue + Traceback (most recent call last): + ... + AttributeError: 'Color' object has no attribute 'blue' + +Likewise, the :attr:`__members__` is only available on the class. + +If you give your :class:`Enum` subclass extra methods, like the `Planet`_ +class above, those methods will show up in a :func:`dir` of the member, +but not of the class:: + + >>> dir(Planet) + ['EARTH', 'JUPITER', 'MARS', 'MERCURY', 'NEPTUNE', 'SATURN', 'URANUS', 'VENUS', '__class__', '__doc__', '__members__', '__module__'] + >>> dir(Planet.EARTH) + ['__class__', '__doc__', '__module__', 'name', 'surface_gravity', 'value'] + +A :meth:`__new__` method will only be used for the creation of the +:class:`Enum` members -- after that it is replaced. This means if you wish to +change how :class:`Enum` members are looked up you either have to write a +helper function or a :func:`classmethod`. diff --cc Doc/library/importlib.rst index 443756f36e,efd027b316..4cc4d10972 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@@ -115,73 -109,6 +115,73 @@@ Function .. versionadded:: 3.3 +.. function:: reload(module) + + Reload a previously imported *module*. The argument must be a module object, + so it must have been successfully imported before. This is useful if you + have edited the module source file using an external editor and want to try + out the new version without leaving the Python interpreter. The return value + is the module object (the same as the *module* argument). + + When :func:`.reload` is executed: + + * Python modules' code is recompiled and the module-level code re-executed, + defining a new set of objects which are bound to names in the module's + dictionary by reusing the :term:`loader` which originally loaded the + module. The ``init`` function of extension modules is not called a second + time. + + * As with all other objects in Python the old objects are only reclaimed + after their reference counts drop to zero. + + * The names in the module namespace are updated to point to any new or + changed objects. + + * Other references to the old objects (such as names external to the module) are + not rebound to refer to the new objects and must be updated in each namespace + where they occur if that is desired. + + There are a number of other caveats: + + If a module is syntactically correct but its initialization fails, the first + :keyword:`import` statement for it does not bind its name locally, but does + store a (partially initialized) module object in ``sys.modules``. To reload + the module you must first :keyword:`import` it again (this will bind the name + to the partially initialized module object) before you can :func:`reload` it. + + When a module is reloaded, its dictionary (containing the module's global + variables) is retained. Redefinitions of names will override the old + definitions, so this is generally not a problem. If the new version of a + module does not define a name that was defined by the old version, the old + definition remains. This feature can be used to the module's advantage if it + maintains a global table or cache of objects --- with a :keyword:`try` + statement it can test for the table's presence and skip its initialization if + desired:: + + try: + cache + except NameError: + cache = {} + + It is legal though generally not very useful to reload built-in or + dynamically loaded modules (this is not true for e.g. :mod:`sys`, - :mod:`__main__`, :mod:`__builtin__` and other key modules where reloading is ++ :mod:`__main__`, :mod:`builtins` and other key modules where reloading is + frowned upon). In many cases, however, extension modules are not designed to + be initialized more than once, and may fail in arbitrary ways when reloaded. + + If a module imports objects from another module using :keyword:`from` ... + :keyword:`import` ..., calling :func:`reload` for the other module does not + redefine the objects imported from it --- one way around this is to + re-execute the :keyword:`from` statement, another is to use :keyword:`import` + and qualified names (*module.name*) instead. + + If a module instantiates instances of a class, reloading the module that + defines the class does not affect the method definitions of the instances --- + they continue to use the old class definition. The same is true for derived + classes. + + .. versionadded:: 3.4 + :mod:`importlib.abc` -- Abstract base classes related to import --------------------------------------------------------------- diff --cc Doc/library/smtpd.rst index 2c2df8ada1,2ca71ffc3c..fe7258a0fc --- a/Doc/library/smtpd.rst +++ b/Doc/library/smtpd.rst @@@ -27,8 -27,7 +27,8 @@@ SMTPServer Object ------------------ - .. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432, -.. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432) ++.. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432,\ + map=None) Create a new :class:`SMTPServer` object, which binds to local address *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. It @@@ -96,8 -90,7 +96,8 @@@ MailmanProxy Object SMTPChannel Objects ------------------- - .. class:: SMTPChannel(server, conn, addr, data_size_limit=33554432, -.. class:: SMTPChannel(server, conn, addr) ++.. class:: SMTPChannel(server, conn, addr, data_size_limit=33554432,\ + map=None)) Create a new :class:`SMTPChannel` object which manages the communication between the server and a single SMTP client. diff --cc Doc/library/unittest.mock.rst index 4a7d647f71,949486170b..d7efcf4f84 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@@ -1989,12 -1969,8 +1989,13 @@@ mock_ope default) then a `MagicMock` will be created for you, with the API limited to methods or attributes available on standard file handles. - `read_data` is a string for the `read`, `readline`, and `readlines` methods - `read_data` is a string for the `~io.IOBase.read` method of the file handle - to return. This is an empty string by default. ++ `read_data` is a string for the :meth:`~io.IOBase.read`, ++ :meth:`~io.IOBase.readline`, and :meth:`~io.IOBase.readlines` methods + of the file handle to return. Calls to those methods will take data from + `read_data` until it is depleted. The mock of these methods is pretty + simplistic. If you need more control over the data that you are feeding to + the tested code you will need to customize this mock for yourself. + `read_data` is an empty string by default. Using `open` as a context manager is a great way to ensure your file handles are closed properly and is becoming common:: diff --cc Doc/whatsnew/3.4.rst index 5e26630028,0000000000..d809c1239c mode 100644,000000..100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@@ -1,656 -1,0 +1,656 @@@ +**************************** + What's New In Python 3.4 +**************************** + +.. :Author: Someone + (uncomment if there is a principal author) + +.. Rules for maintenance: + + * Anyone can add text to this document, but the maintainer reserves the + right to rewrite any additions. In particular, for obscure or esoteric + features, the maintainer may reduce any addition to a simple reference to + the new documentation rather than explaining the feature inline. + + * While the maintainer will periodically go through Misc/NEWS + and add changes, it's best not to rely on this. We know from experience + that any changes that aren't in the What's New documentation around the + time of the original release will remain largely unknown to the community + for years, even if they're added later. We also know from experience that + other priorities can arise, and the maintainer will run out of time to do + updates - in such cases, end users will be much better served by partial + notifications that at least give a hint about new features to + investigate. + + * This is not a complete list of every single change; completeness + is the purpose of Misc/NEWS. The What's New should focus on changes that + are visible to Python *users* and that *require* a feature release (i.e. + most bug fixes should only be recorded in Misc/NEWS) + + * PEPs should not be marked Final until they have an entry in What's New. + A placeholder entry that is just a section header and a link to the PEP + (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been + implemented and noted in What's New, don't forget to mark it as Final! + + * If you want to draw your new text to the attention of the + maintainer, add 'XXX' to the beginning of the paragraph or + section. + + * It's OK to add just a very brief note about a change. For + example: "The :ref:`~socket.transmogrify()` function was added to the + :mod:`socket` module." The maintainer will research the change and + write the necessary text (if appropriate). The advantage of doing this + is that even if no more descriptive text is ever added, readers will at + least have a notification that the new feature exists and a link to the + relevant documentation. + + * You can comment out your additions if you like, but it's not + necessary (especially when a final release is some months away). + + * Credit the author of a patch or bugfix. Just the name is + sufficient; the e-mail address isn't necessary. + + * It's helpful to add the bug/patch number as a comment: + + The :ref:`~socket.transmogrify()` function was added to the + :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.) + + This saves the maintainer the effort of going through the Mercurial log + when researching a change. + + * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``, + while :ref:`~mod.attr` will display as ``attr``. + +This article explains the new features in Python 3.4, compared to 3.3. + +.. Python 3.4 was released on TBD. + +For full details, see the +`changelog `_. + +.. note:: Prerelease users should be aware that this document is currently in + draft form. It will be updated substantially as Python 3.4 moves towards + release, so it's worth checking back even after reading earlier versions. + + +.. seealso:: + + .. :pep:`4XX` - Python 3.4 Release Schedule + + +Summary -- Release highlights +============================= + +.. This section singles out the most important changes in Python 3.4. + Brevity is key. + +New syntax features: + +* None yet. + +New library modules: + +* :mod:`enum`: Implementation of the :pep:`435`. +* :mod:`selectors`: High-level and efficient I/O multiplexing, built upon the + :mod:`select` module primitives. + +New built-in features: + +* :ref:`PEP 442: Safe object finalization `. +* :ref:`PEP 445: Configurable memory allocators `. +* :ref:`PEP 446: Make newly created file descriptors non-inheritable `. + +Implementation improvements: + +* A more efficient :mod:`marshal` format (:issue:`16475`). +* Improve finalization of Python modules to avoid setting their globals + to None, in most cases (:issue:`18214`). + +Significantly Improved Library Modules: + +* Single-dispatch generic functions (:pep:`443`) +* SHA-3 (Keccak) support for :mod:`hashlib`. +* TLSv1.1 and TLSv1.2 support for :mod:`ssl`. +* :mod:`multiprocessing` now has option to avoid using :func:`os.fork` + on Unix (:issue:`8713`). + +Security improvements: + +* command line option for :ref:`isolated mode `, + :issue:`16499`. + +Please read on for a comprehensive list of user-facing changes. + +.. _pep-446: + +PEP 446: Make newly created file descriptors non-inheritable +============================================================ + +The :pep:`446` makes newly created file descriptors :ref:`non-inheritable +`. New functions and methods: + +* :func:`os.get_inheritable`, :func:`os.set_inheritable` +* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` +* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable` + + +.. _pep-445: + +PEP 445: Add new APIs to customize Python memory allocators +=========================================================== + +The :pep:`445` adds new Application Programming Interfaces (API) to customize +Python memory allocators. + + +.. _pep-442: + +PEP 442: Safe object finalization +================================= + +This PEP removes the current limitations and quirks of object finalization. +With it, objects with :meth:`__del__` methods, as well as generators +with :keyword:`finally` clauses, can be finalized when they are part of a +reference cycle. + +As part of this change, module globals are no longer forcibly set to +:const:`None` during interpreter shutdown, instead relying on the normal +operation of the cyclic garbage collector. + +.. seealso:: + + :pep:`442` - Safe object finalization + PEP written and implemented by Antoine Pitrou + + +Other Language Changes +====================== + +Some smaller changes made to the core Python language are: + +* Unicode database updated to UCD version 6.3. + +* :func:`min` and :func:`max` now accept a *default* argument that can be used + to specify the value they return if the iterable they are evaluating has no + elements. Contributed by Julian Berman in :issue:`18111`. + +* Module objects are now :mod:`weakref`'able. + + +New Modules +=========== + +selectors +--------- + +The new :mod:`selectors` module allows high-level and efficient I/O +multiplexing, built upon the :mod:`select` module primitives. + + +Improved Modules +================ + +aifc +---- + +The :meth:`~aifc.getparams` method now returns a namedtuple rather than a +plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.) + + +codecs +------ + +The :meth:`codecs.encode` and :meth:`codecs.decode` convenience functions are +now properly documented. These functions have existed in the :mod:`codecs` +module since ~2004, but were previously only discoverable through runtime +introspection. + +Unlike the convenience methods on :class:`str`, :class:`bytes` and +:class:`bytearray`, these convenience functions support arbitrary codecs, +rather than being limited to Unicode text encodings. + + +colorsys +-------- + +The number of digits in the coefficients for the RGB --- YIQ conversions have +been expanded so that they match the FCC NTSC versions. The change in +results should be less than 1% and may better match results found elsewhere. + + +contextlib +---------- + +The new :class:`contextlib.ignore` context manager helps to clarify the +intent of code that deliberately ignores failures from a particular +operation. + +The new :class:`contextlib.redirect_stdio` context manager makes it easier +for utility scripts to handle inflexible APIs that don't provide any +options to retrieve their output as a string or direct it to somewhere +other than :data:`sys.stdout`. + + +dis +--- + - The :mod:`dis` module is now built around an :class:`Instruction` class that - provides details of individual bytecode operations and a - :func:`get_instructions` iterator that emits the Instruction stream for a ++The :mod:`dis` module is now built around an :class:`~dis.Instruction` class ++that provides details of individual bytecode operations and a ++:func:`~dis.get_instructions` iterator that emits the Instruction stream for a +given piece of Python code. The various display tools in the :mod:`dis` +module have been updated to be based on these new components. + +The new :class:`dis.Bytecode` class provides an object-oriented API for +inspecting bytecode, both in human-readable form and for iterating over +instructions. + +(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816`) + + +doctest +------- + +Added :data:`~doctest.FAIL_FAST` flag to halt test running as soon as the first +failure is detected. (Contributed by R. David Murray and Daniel Urban in +:issue:`16522`.) + +Updated the doctest command line interface to use :mod:`argparse`, and added +``-o`` and ``-f`` options to the interface. ``-o`` allows doctest options to +be specified on the command line, and ``-f`` is a shorthand for ``-o +FAIL_FAST`` (to parallel the similar option supported by the :mod:`unittest` +CLI). (Contributed by R. David Murray in :issue:`11390`.) + + +email +----- + +:meth:`~email.message.Message.as_string` now accepts a *policy* argument to +override the default policy of the message when generating a string +representation of it. This means that ``as_string`` can now be used in more +circumstances, instead of having to create and use a :mod:`~email.generator` in +order to pass formatting parameters to its ``flatten`` method. + +New method :meth:`~email.message.Message.as_bytes` added to produce a bytes +representation of the message in a fashion similar to how ``as_string`` +produces a string representation. It does not accept the *maxheaderlen* +argument, but does accept the *unixfrom* and *policy* arguments. The +:class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method +calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive +result: a bytes object containing the fully formatted message. + +(Contributed by R. David Murray in :issue:`18600`.) + + +functools +--------- + +New :func:`functools.singledispatch` decorator: see the :pep:`443`. + + +hashlib +------- + +New :func:`hashlib.pbkdf2_hmac` function. + +(Contributed by Christian Heimes in :issue:`18582`) + + +inspect +------- + + +The inspect module now offers a basic command line interface to quickly +display source code and other information for modules, classes and +functions. + +:func:`~inspect.unwrap` makes it easy to unravel wrapper function chains +created by :func:`functools.wraps` (and any other API that sets the +``__wrapped__`` attribute on a wrapper function). + +mmap +---- + +mmap objects can now be weakref'ed. + +(Contributed by Valerie Lambert in :issue:`4885`.) + + +multiprocessing +--------------- + +On Unix two new *start methods* have been added for starting processes +using :mod:`multiprocessing`. These make the mixing of processes with +threads more robust. See :issue:`8713`. + +Also, except when using the old *fork* start method, child processes +will no longer inherit unneeded handles/file descriptors from their parents. + + +os +-- + +New functions to get and set the :ref:`inheritable flag ` of a file +descriptors or a Windows handle: + +* :func:`os.get_inheritable`, :func:`os.set_inheritable` +* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` + + +pdb +--- + +The ``print`` command has been removed from :mod:`pdb`, restoring access to the +``print`` function. + +Rationale: Python2's ``pdb`` did not have a ``print`` command; instead, +entering ``print`` executed the ``print`` statement. In Python3 ``print`` was +mistakenly made an alias for the pdb :pdbcmd:`p` command. ``p``, however, +prints the ``repr`` of its argument, not the ``str`` like the Python2 ``print`` +command did. Worse, the Python3 ``pdb print`` command shadowed the Python3 +``print`` function, making it inaccessible at the ``pdb`` prompt. + +(Contributed by Connor Osborn in :issue:`18764`.) + + +poplib +------ + +New :meth:`~poplib.POP3.stls` method to switch a clear-text POP3 session into +an encrypted POP3 session. + +New :meth:`~poplib.POP3.capa` method to query the capabilities advertised by the +POP3 server. + +(Contributed by Lorenzo Catucci in :issue:`4473`.) + + +pprint +------ + +The :mod::`pprint` module now supports *compact* mode for formatting long +sequences (:issue:`19132`). + + +smtplib +------- + +:exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows +both socket level errors and SMTP protocol level errors to be caught in one +try/except statement by code that only cares whether or not an error occurred. +(:issue:`2118`). + + +socket +------ + +Socket objects have new methods to get or set their :ref:`inheritable flag +`: + +* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable` + +The ``socket.AF_*`` and ``socket.SOCK_*`` constants are enumeration values, +using the new :mod:`enum` module. This allows descriptive reporting during +debugging, instead of seeing integer "magic numbers". + +ssl +--- + +TLSv1.1 and TLSv1.2 support. + +(Contributed by Michele Orrù and Antoine Pitrou in :issue:`16692`) + +* New diagnostic functions :func:`~ssl.get_default_verify_paths`, + :meth:`~ssl.SSLContext.cert_store_stats` and + :meth:`~ssl.SSLContext.get_ca_certs` + +* Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows' + cert store. + +(Contributed by Christian Heimes in :issue:`18143`, :issue:`18147` and + :issue:`17134`.) + +Support for server-side SNI using the new +:meth:`ssl.SSLContext.set_servername_callback` method. + +(Contributed by Daniel Black in :issue:`8109`.) + + +stat +---- + +The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C +implementation is required as most of the values aren't standardized and +platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) + +The module supports new file types: door, event port and whiteout. + + +struct +------ + +Streaming struct unpacking using :func:`struct.iter_unpack`. + +(Contributed by Antoine Pitrou in :issue:`17804`.) + + +sunau +----- + +The :meth:`~sunau.getparams` method now returns a namedtuple rather than a +plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.) + +:meth:`sunau.open` now supports the context manager protocol (:issue:`18878`). + + +traceback +--------- + +A new :func:`traceback.clear_frames` function takes a traceback object +and clears the local variables in all of the frames it references, +reducing the amount of memory consumed (:issue:`1565525`). + + +urllib +------ + +Add support.for ``data:`` URLs in :mod:`urllib.request`. + +(Contributed by Mathias Panzenböck in :issue:`16423`.) + + +unittest +-------- + +Support for easy dynamically-generated subtests using the +:meth:`~unittest.TestCase.subTest` context manager. + +(Contributed by Antoine Pitrou in :issue:`16997`.) + + +wave +---- + +The :meth:`~wave.getparams` method now returns a namedtuple rather than a +plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.) + +:meth:`wave.open` now supports the context manager protocol. (Contributed +by Claudiu Popa in :issue:`17616`.) + + +weakref +------- + +New :class:`~weakref.WeakMethod` class simulates weak references to bound +methods. (Contributed by Antoine Pitrou in :issue:`14631`.) + +New :class:`~weakref.finalize` class makes it possible to register a callback +to be invoked when an object is garbage collected, without needing to +carefully manage the lifecycle of the weak reference itself. (Contributed by +Richard Oudkerk in :issue:`15528`) + + +xml.etree +--------- + +Add an event-driven parser for non-blocking applications, +:class:`~xml.etree.ElementTree.XMLPullParser`. + +(Contributed by Antoine Pitrou in :issue:`17741`.) + +Other improvements +================== + +Tab-completion is now enabled by default in the interactive interpreter. + +(Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.) + +Python invocation changes +========================= + +Invoking the Python interpreter with ``--version`` now outputs the version to +standard output instead of standard error (:issue:`18338`). Similar changes +were made to :mod:`argparse` (:issue:`18920`) and other modules that have +script-like invocation capabilities (:issue:`18922`). + +Optimizations +============= + +Major performance enhancements have been added: + +* The UTF-32 decoder is now 3x to 4x faster. + +* The cost of hash collisions for sets is now reduced. Each hash table + probe now checks a series of consecutive, adjacent key/hash pairs before + continuing to make random probes through the hash table. This exploits + cache locality to make collision resolution less expensive. + + The collision resolution scheme can be described as a hybrid of linear + probing and open addressing. The number of additional linear probes + defaults to nine. This can be changed at compile-time by defining + LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off + linear probing entirely. + + (Contributed by Raymond Hettinger in :issue"`18771`.) + +* The interpreter starts about 30% faster. A couple of measures lead to the + speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`, + :mod:`collections` and :mod:`locale` modules and their dependencies are no + longer imported by default. The marshal module has been improved to load + compiled Python code faster. + + (Contributed by Antoine Pitrou, Christian Heimes and Victor Stinner in + :issue:`19219`, :issue:`19218`, :issue:`19209`, :issue:`19205` and + :issue:`9548`) + + + +Build and C API Changes +======================= + +Changes to Python's build process and to the C API include: + +* None yet. + + +Deprecated +========== + +Unsupported Operating Systems +----------------------------- + +* OS/2 +* Windows 2000 + + +Deprecated Python modules, functions and methods +------------------------------------------------ + +* :meth:`difflib.SequenceMatcher.isbjunk` and + :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and + ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object. + +* :func:`importlib.util.module_for_loader` is pending deprecation. Using + :func:`importlib.util.module_to_load` and + :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader + to more easily customize module loading. + +* The :mod:`imp` module is pending deprecation. To keep compatibility with + Python 2/3 code bases, the module's removal is currently not scheduled. + +* The :mod:`formatter` module is pending deprecation and is slated for removal + in Python 3.6. + + +Deprecated functions and types of the C API +------------------------------------------- + +* The ``PyThreadState.tick_counter`` field has been value: its value was meaningless + since Python 3.2 ("new GIL"). + + +Deprecated features +------------------- + +* None yet. + + +Porting to Python 3.4 +===================== + +This section lists previously described changes and other bugfixes +that may require changes to your code. + +* The ABCs defined in :mod:`importlib.abc` now either raise the appropriate + exception or return a default value instead of raising + :exc:`NotImplementedError` blindly. This will only affect code calling + :func:`super` and falling through all the way to the ABCs. For compatibility, + catch both :exc:`NotImplementedError` or the appropriate exception as needed. + +* The module type now initializes the :attr:`__package__` and :attr:`__loader__` + attributes to ``None`` by default. To determine if these attributes were set + in a backwards-compatible fashion, use e.g. + ``getattr(module, '__loader__', None) is not None``. + +* :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and + ``__package__`` unconditionally to properly support reloading. If this is not + desired then you will need to set these attributes manually. You can use + :func:`importlib.util.module_to_load` for module management. + +* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``, + ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading. + +* Frozen packages no longer set ``__path__`` to a list containing the package + name but an empty list instead. Determing if a module is a package should be + done using ``hasattr(module, '__path__')``. + +* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** + argument is not set. Previously only ``NULL`` was returned with no exception + set. + +* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path + it would write to is a symlink or a non-regular file. This is to act as a + warning that import will overwrite those files with a regular file regardless + of what type of file path they were originally. + +* :meth:`importlib.abc.SourceLoader.get_source` no longer raises + :exc:`ImportError` when the source code being loaded triggers a + :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is + meant to be raised only when source code cannot be found but it should, it was + felt to be over-reaching/overloading of that meaning when the source code is + found but improperly structured. If you were catching ImportError before and + wish to continue to ignore syntax or decoding issues, catch all three + exceptions now. + +* :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly + set the ``__wrapped__`` attribute even if the wrapped function had a + wrapped attribute set. This means ``__wrapped__`` attributes now correctly + link a stack of decorated functions rather than every ``__wrapped__`` + attribute in the chain referring to the innermost function. Introspection + libraries that assumed the previous behaviour was intentional can use + :func:`inspect.unwrap` to gain equivalent behaviour. + +* (C API) The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must + now be a string allocated by :c:func:`PyMem_RawMalloc` or + :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a + string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. + diff --cc Misc/NEWS index a56339ce82,636f29581e..2d47040598 --- a/Misc/NEWS +++ b/Misc/NEWS @@@ -111,42 -117,6 +111,44 @@@ Librar - Issue #4366: Fix building extensions on all platforms when --enable-shared is used. +Tests +----- + +- Issue #18919: Unified and extended tests for audio modules: aifc, sunau and + wave. + +- Issue #18714: Added tests for ``pdb.find_function()``. + +Documentation +------------- + ++- Issue #18758: Fixed and improved cross-references. ++ +- Issue #18972: Modernize email examples and use the argparse module in them. + +Build +----- + +- Issue #19130: Correct PCbuild/readme.txt, Python 3.3 and 3.4 require VS 2010. + + +What's New in Python 3.4.0 Alpha 3? +=================================== + +Release date: 2013-09-29 + + +Core and Builtins +----------------- + +- Issue #18818: The "encodingname" part of PYTHONIOENCODING is now optional. + +- Issue #19098: Prevent overflow in the compiler when the recursion limit is set + absurdly high. + +Library +------- + - Issue #18950: Fix miscellaneous bugs in the sunau module. Au_read.readframes() now updates current file position and reads correct number of frames from multichannel stream. Au_write.writeframesraw() now