From: Terry Jan Reedy Date: Mon, 14 Apr 2014 20:17:09 +0000 (-0400) Subject: Closes #20956: 2.7 tokenize does not produce named tuples. Patch by Sam Kimbrel. X-Git-Tag: v2.7.7rc1~63 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=f7568e0a8a7902b303f442ab6544bdd009cfb3c9;p=python Closes #20956: 2.7 tokenize does not produce named tuples. Patch by Sam Kimbrel. --- diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index d0374521cd..0fc0d7ef1b 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -17,9 +17,10 @@ for on-screen displays. To simplify token stream handling, all :ref:`operators` and :ref:`delimiters` tokens are returned using the generic :data:`token.OP` token type. The exact -type can be determined by checking the token ``string`` field on the -:term:`named tuple` returned from :func:`tokenize.tokenize` for the character -sequence that identifies a specific operator token. +type can be determined by checking the second field (containing the actual +token string matched) of the tuple returned from +:func:`tokenize.generate_tokens` for the character sequence that identifies a +specific operator token. The primary entry point is a :term:`generator`: diff --git a/Misc/ACKS b/Misc/ACKS index a6180020f9..5814320287 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -670,6 +670,7 @@ Mads Kiilerich Jason Killen Jan Kim Taek Joo Kim +Sam Kimbrel W. Trevor King Paul Kippes Steve Kirsch