From f7568e0a8a7902b303f442ab6544bdd009cfb3c9 Mon Sep 17 00:00:00 2001 From: Terry Jan Reedy Date: Mon, 14 Apr 2014 16:17:09 -0400 Subject: [PATCH] Closes #20956: 2.7 tokenize does not produce named tuples. Patch by Sam Kimbrel. --- Doc/library/tokenize.rst | 7 ++++--- Misc/ACKS | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index d0374521cd..0fc0d7ef1b 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -17,9 +17,10 @@ for on-screen displays. To simplify token stream handling, all :ref:`operators` and :ref:`delimiters` tokens are returned using the generic :data:`token.OP` token type. The exact -type can be determined by checking the token ``string`` field on the -:term:`named tuple` returned from :func:`tokenize.tokenize` for the character -sequence that identifies a specific operator token. +type can be determined by checking the second field (containing the actual +token string matched) of the tuple returned from +:func:`tokenize.generate_tokens` for the character sequence that identifies a +specific operator token. The primary entry point is a :term:`generator`: diff --git a/Misc/ACKS b/Misc/ACKS index a6180020f9..5814320287 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -670,6 +670,7 @@ Mads Kiilerich Jason Killen Jan Kim Taek Joo Kim +Sam Kimbrel W. Trevor King Paul Kippes Steve Kirsch -- 2.50.1