]> granicus.if.org Git - python/commit
Issue #24619: Simplify async/await tokenization.
authorYury Selivanov <yselivanov@sprymix.com>
Thu, 23 Jul 2015 12:01:58 +0000 (15:01 +0300)
committerYury Selivanov <yselivanov@sprymix.com>
Thu, 23 Jul 2015 12:01:58 +0000 (15:01 +0300)
commit96ec934e755355cfc5af036db8641646b7ddb45e
treea6fd6a4cbef1b75ab0cc10db01fd91ecf2e99976
parentf315c1c01676bfabb5b1c6628642668f1ef436a6
Issue #24619: Simplify async/await tokenization.

This commit simplifies async/await tokenization in tokenizer.c,
tokenize.py & lib2to3/tokenize.py.  Previous solution was to keep
a stack of async-def & def blocks, whereas the new approach is just
to remember position of the outermost async-def block.

This change won't bring any parsing performance improvements, but
it makes the code much easier to read and validate.
Lib/lib2to3/pgen2/tokenize.py
Lib/lib2to3/tests/test_parser.py
Lib/test/test_coroutines.py
Lib/test/test_tokenize.py
Lib/tokenize.py
Parser/tokenizer.c
Parser/tokenizer.h