From: Brett Cannon Date: Tue, 22 Feb 2011 03:35:18 +0000 (+0000) Subject: Merged revisions 88498 via svnmerge from X-Git-Tag: v3.2.1b1~393 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=45b96d373e6025ce0c16d6d60943da1bf4d13620;p=python Merged revisions 88498 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/py3k ........ r88498 | brett.cannon | 2011-02-21 19:25:12 -0800 (Mon, 21 Feb 2011) | 8 lines Issue #11074: Make 'tokenize' so it can be reloaded. The module stored away the 'open' object as found in the global namespace (which fell through to the built-in namespace) since it defined its own 'open'. Problem is that if you reloaded the module it then grabbed the 'open' defined in the previous load, leading to code that infinite recursed. Switched to simply call builtins.open directly. ........ --- diff --git a/Lib/tokenize.py b/Lib/tokenize.py index 506aa6a42f..f575e9bc23 100644 --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -24,6 +24,7 @@ __author__ = 'Ka-Ping Yee ' __credits__ = ('GvR, ESR, Tim Peters, Thomas Wouters, Fred Drake, ' 'Skip Montanaro, Raymond Hettinger, Trent Nelson, ' 'Michael Foord') +import builtins import re import sys from token import * @@ -335,13 +336,11 @@ def detect_encoding(readline): return default, [first, second] -_builtin_open = open - def open(filename): """Open a file in read only mode using the encoding detected by detect_encoding(). """ - buffer = _builtin_open(filename, 'rb') + buffer = builtins.open(filename, 'rb') encoding, lines = detect_encoding(buffer.readline) buffer.seek(0) text = TextIOWrapper(buffer, encoding, line_buffering=True) diff --git a/Misc/NEWS b/Misc/NEWS index 39abedcd83..a5ac7c104e 100644 --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,6 +15,8 @@ Core and Builtins Library ------- +- Issue #11074: Make 'tokenize' so it can be reloaded. + - Issue #4681: Allow mmap() to work on file sizes and offsets larger than 4GB, even on 32-bit builds. Initial patch by Ross Lagerwall, adapted for 32-bit Windows.