nesting from left to right, and evaluating the expression to produce an element
each time the innermost block is reached.
-Note that the comprehension is executed in a separate scope, so names assigned
-to in the target list don't "leak" into the enclosing scope.
+However, aside from the iterable expression in the leftmost :keyword:`for` clause,
+the comprehension is executed in a separate implicitly nested scope. This ensures
+that names assigned to in the target list don't "leak" into the enclosing scope.
+
+The iterable expression in the leftmost :keyword:`for` clause is evaluated
+directly in the enclosing scope and then passed as an argument to the implictly
+nested scope. Subsequent :keyword:`for` clauses and any filter condition in the
+leftmost :keyword:`for` clause cannot be evaluated in the enclosing scope as
+they may depend on the values obtained from the leftmost iterable. For example:
+``[x*y for x in range(10) for y in range(x, x+10)]``.
+
+To ensure the comprehension always results in a container of the appropriate
+type, ``yield`` and ``yield from`` expressions are prohibited in the implicitly
+nested scope (in Python 3.7, such expressions emit :exc:`DeprecationWarning`
+when compiled, in Python 3.8+ they will emit :exc:`SyntaxError`).
Since Python 3.6, in an :keyword:`async def` function, an :keyword:`async for`
clause may be used to iterate over a :term:`asynchronous iterator`.
suspend the execution of the coroutine function in which it appears.
See also :pep:`530`.
+.. versionadded:: 3.6
+ Asynchronous comprehensions were introduced.
+
+.. deprecated:: 3.7
+ ``yield`` and ``yield from`` deprecated in the implicitly nested scope.
+
+
.. _lists:
List displays
Variables used in the generator expression are evaluated lazily when the
:meth:`~generator.__next__` method is called for the generator object (in the same
-fashion as normal generators). However, the leftmost :keyword:`for` clause is
-immediately evaluated, so that an error produced by it can be seen before any
-other possible error in the code that handles the generator expression.
-Subsequent :keyword:`for` clauses cannot be evaluated immediately since they
-may depend on the previous :keyword:`for` loop. For example: ``(x*y for x in
-range(10) for y in bar(x))``.
+fashion as normal generators). However, the iterable expression in the
+leftmost :keyword:`for` clause is immediately evaluated, so that an error
+produced by it will be emitted at the point where the generator expression
+is defined, rather than at the point where the first value is retrieved.
+Subsequent :keyword:`for` clauses and any filter condition in the leftmost
+:keyword:`for` clause cannot be evaluated in the enclosing scope as they may
+depend on the values obtained from the leftmost iterable. For example:
+``(x*y for x in range(10) for y in range(x, x+10))``.
The parentheses can be omitted on calls with only one argument. See section
:ref:`calls` for details.
+To avoid interfering with the expected operation of the generator expression
+itself, ``yield`` and ``yield from`` expressions are prohibited in the
+implicitly defined generator (in Python 3.7, such expressions emit
+:exc:`DeprecationWarning` when compiled, in Python 3.8+ they will emit
+:exc:`SyntaxError`).
+
If a generator expression contains either :keyword:`async for`
clauses or :keyword:`await` expressions it is called an
:dfn:`asynchronous generator expression`. An asynchronous generator
expression returns a new asynchronous generator object,
which is an asynchronous iterator (see :ref:`async-iterators`).
+.. versionadded:: 3.6
+ Asynchronous generator expressions were introduced.
+
.. versionchanged:: 3.7
Prior to Python 3.7, asynchronous generator expressions could
only appear in :keyword:`async def` coroutines. Starting
with 3.7, any function can use asynchronous generator expressions.
+.. deprecated:: 3.7
+ ``yield`` and ``yield from`` deprecated in the implicitly nested scope.
+
+
.. _yieldexpr:
Yield expressions
async def agen(): # defines an asynchronous generator function (PEP 525)
yield 123
+Due to their side effects on the containing scope, ``yield`` expressions
+are not permitted as part of the implicitly defined scopes used to
+implement comprehensions and generator expressions (in Python 3.7, such
+expressions emit :exc:`DeprecationWarning` when compiled, in Python 3.8+
+they will emit :exc:`SyntaxError`)..
+
+.. deprecated:: 3.7
+ Yield expressions deprecated in the implicitly nested scopes used to
+ implement comprehensions and generator expressions.
+
Generator functions are described below, while asynchronous generator
functions are described separately in section
:ref:`asynchronous-generator-functions`.
Deprecated
==========
+* Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated
+ in comprehensions and generator expressions (aside from the iterable expression
+ in the leftmost :keyword:`for` clause). This ensures that comprehensions
+ always immediately return a container of the appropriate type (rather than
+ potentially returning a :term:`generator iterator` object), while generator
+ expressions won't attempt to interleave their implicit output with the output
+ from any explicit yield expressions.
+
+ In Python 3.7, such expressions emit :exc:`DeprecationWarning` when compiled,
+ in Python 3.8+ they will emit :exc:`SyntaxError`. (Contributed by Serhiy
+ Storchaka in :issue:`10544`.)
+
- Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with
a macro if ``Py_LIMITED_API`` is not set or set to the value between
``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or
[None]
-
-An obscene abuse of a yield expression within a generator expression:
-
->>> list((yield 21) for i in range(4))
-[21, None, 21, None, 21, None, 21, None]
-
-And a more sane, but still weird usage:
+Yield is allowed only in the outermost iterable in generator expression:
>>> def f(): list(i for i in [(yield 26)])
>>> type(f())
>>> type(f())
<class 'generator'>
->>> def f(): x=(i for i in (yield) if (yield))
->>> type(f())
-<class 'generator'>
-
>>> def f(d): d[(yield "a")] = d[(yield "b")] = 27
>>> data = [1,2]
>>> g = f(data)
# Check annotation refleak on SyntaxError
check_syntax_error(self, "def g(a:(yield)): pass")
+ def test_yield_in_comprehensions(self):
+ # Check yield in comprehensions
+ def g(): [x for x in [(yield 1)]]
+ def g(): [x for x in [(yield from ())]]
+
+ def check(code, warntext):
+ with self.assertWarnsRegex(DeprecationWarning, warntext):
+ compile(code, '<test string>', 'exec')
+ import warnings
+ with warnings.catch_warnings():
+ warnings.filterwarnings('error', category=DeprecationWarning)
+ with self.assertRaisesRegex(SyntaxError, warntext):
+ compile(code, '<test string>', 'exec')
+
+ check("def g(): [(yield x) for x in ()]",
+ "'yield' inside list comprehension")
+ check("def g(): [x for x in () if not (yield x)]",
+ "'yield' inside list comprehension")
+ check("def g(): [y for x in () for y in [(yield x)]]",
+ "'yield' inside list comprehension")
+ check("def g(): {(yield x) for x in ()}",
+ "'yield' inside set comprehension")
+ check("def g(): {(yield x): x for x in ()}",
+ "'yield' inside dict comprehension")
+ check("def g(): {x: (yield x) for x in ()}",
+ "'yield' inside dict comprehension")
+ check("def g(): ((yield x) for x in ())",
+ "'yield' inside generator expression")
+ check("def g(): [(yield from x) for x in ()]",
+ "'yield' inside list comprehension")
+ check("class C: [(yield x) for x in ()]",
+ "'yield' inside list comprehension")
+ check("[(yield x) for x in ()]",
+ "'yield' inside list comprehension")
+
def test_raise(self):
# 'raise' test [',' test]
try: raise RuntimeError('just testing')
--- /dev/null
+Yield expressions are now deprecated in comprehensions and generator
+expressions. They are still permitted in the definition of the outermost
+iterable, as that is evaluated directly in the enclosing scope.
e->lineno, e->col_offset)) {
return 0;
}
- st->st_cur->ste_generator = is_generator;
if (outermost->is_async) {
st->st_cur->ste_coroutine = 1;
}
if (value)
VISIT(st, expr, value);
VISIT(st, expr, elt);
+ if (st->st_cur->ste_generator) {
+ PyObject *msg = PyUnicode_FromString(
+ (e->kind == ListComp_kind) ? "'yield' inside list comprehension" :
+ (e->kind == SetComp_kind) ? "'yield' inside set comprehension" :
+ (e->kind == DictComp_kind) ? "'yield' inside dict comprehension" :
+ "'yield' inside generator expression");
+ if (msg == NULL) {
+ symtable_exit_block(st, (void *)e);
+ return 0;
+ }
+ if (PyErr_WarnExplicitObject(PyExc_DeprecationWarning,
+ msg, st->st_filename, st->st_cur->ste_lineno,
+ NULL, NULL) == -1)
+ {
+ if (PyErr_ExceptionMatches(PyExc_DeprecationWarning)) {
+ /* Replace the DeprecationWarning exception with a SyntaxError
+ to get a more accurate error report */
+ PyErr_Clear();
+ PyErr_SetObject(PyExc_SyntaxError, msg);
+ PyErr_SyntaxLocationObject(st->st_filename,
+ st->st_cur->ste_lineno,
+ st->st_cur->ste_col_offset);
+ }
+ Py_DECREF(msg);
+ symtable_exit_block(st, (void *)e);
+ return 0;
+ }
+ Py_DECREF(msg);
+ }
+ st->st_cur->ste_generator |= is_generator;
return symtable_exit_block(st, (void *)e);
}
projects / python / commitdiff