From: Antoine Pitrou Date: Sat, 23 Jul 2011 19:50:21 +0000 (+0200) Subject: Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without X-Git-Tag: v3.3.0a1~1851 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=d42c1d09e99d74f6b411bdb1d3c8b9a648dd6a79;p=python Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without a read1() method), and add a *write_through* parameter to mandate unbuffered writes. --- d42c1d09e99d74f6b411bdb1d3c8b9a648dd6a79 diff --cc Lib/_pyio.py index 265edab113,78c6d95c23..a9c31d59ee --- a/Lib/_pyio.py +++ b/Lib/_pyio.py @@@ -1521,7 -1515,7 +1521,8 @@@ class TextIOWrapper(TextIOBase) self._decoded_chars_used = 0 # offset into _decoded_chars for read() self._snapshot = None # info for reconstructing decoder state self._seekable = self._telling = self.buffer.seekable() + self._has_read1 = hasattr(self.buffer, 'read1') + self._b2cratio = 0.0 if self._seekable and self.writable(): position = self.buffer.tell() @@@ -1687,14 -1681,12 +1688,17 @@@ # len(dec_buffer) bytes ago with decoder state (b'', dec_flags). # Read a chunk, decode it, and put the result in self._decoded_chars. - input_chunk = self.buffer.read1(self._CHUNK_SIZE) + if self._has_read1: + input_chunk = self.buffer.read1(self._CHUNK_SIZE) + else: + input_chunk = self.buffer.read(self._CHUNK_SIZE) eof = not input_chunk - self._set_decoded_chars(self._decoder.decode(input_chunk, eof)) + decoded_chars = self._decoder.decode(input_chunk, eof) + self._set_decoded_chars(decoded_chars) + if decoded_chars: + self._b2cratio = len(input_chunk) / len(self._decoded_chars) + else: + self._b2cratio = 0.0 if self._telling: # At the snapshot point, len(dec_buffer) bytes before the read, diff --cc Misc/NEWS index 282425d81b,2eea1bedc8..089b07a624 --- a/Misc/NEWS +++ b/Misc/NEWS @@@ -237,92 -181,8 +237,96 @@@ Core and Builtin Library ------- ++- Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without ++ a read1() method), and add a *write_through* parameter to mandate ++ unbuffered writes. ++ +- Issue #10883: Fix socket leaks in urllib.request when using FTP. + +- Issue #12592: Make Python build on OpenBSD 5 (and future major releases). + +- Issue #12372: POSIX semaphores are broken on AIX: don't use them. + +- Issue #12551: Provide a get_channel_binding() method on SSL sockets so as + to get channel binding data for the current SSL session (only the + "tls-unique" channel binding is implemented). This allows the implementation + of certain authentication mechanisms such as SCRAM-SHA-1-PLUS. Patch by + Jacek Konieczny. + +- Issue #665194: email.utils now has format_datetime and parsedate_to_datetime + functions, allowing for round tripping of RFC2822 format dates. + +- Issue #12571: Add a plat-linux3 directory mirroring the plat-linux2 + directory, so that "import DLFCN" and other similar imports work on + Linux 3.0. + +- Issue #7484: smtplib no longer puts <> around addresses in VRFY and EXPN + commands; they aren't required and in fact postfix doesn't support that form. + +- Issue #12273: Remove ast.__version__. AST changes can be accounted for by + checking sys.version_info or sys._mercurial. + +- Silence spurious "broken pipe" tracebacks when shutting down a + ProcessPoolExecutor. + +- Fix potential resource leaks in concurrent.futures.ProcessPoolExecutor + by joining all queues and processes when shutdown() is called. + +- Issue #11603: Fix a crash when __str__ is rebound as __repr__. Patch by + Andreas Stührk. + +- Issue #11321: Fix a crash with multiple imports of the _pickle module when + embedding Python. Patch by Andreas Stührk. + +- Issue #6755: Add get_wch() method to curses.window class. Patch by Iñigo + Serna. + +- Add cgi.closelog() function to close the log file. + +- Issue #12502: asyncore: fix polling loop with AF_UNIX sockets. + +- Issue #4376: ctypes now supports nested structures in a endian different than + the parent structure. Patch by Vlad Riscutia. + +- Raise ValueError when attempting to set the _CHUNK_SIZE attribute of a + TextIOWrapper to a huge value, not TypeError. + +- Issue #12504: Close file handles in a timely manner in packaging.database. + This fixes a bug with the remove (uninstall) feature on Windows. + +- Issues #12169 and #10510: Factor out code used by various packaging commands + to make HTTP POST requests, and make sure it uses CRLF. + +- Issue #12016: Multibyte CJK decoders now resynchronize faster. They only + ignore the first byte of an invalid byte sequence. For example, + b'\xff\n'.decode('gb2312', 'replace') gives '\ufffd\n' instead of '\ufffd'. + +- Issue #12459: time.sleep() now raises a ValueError if the sleep length is + negative, instead of an infinite sleep on Windows or raising an IOError on + Linux for example, to have the same behaviour on all platforms. + +- Issue #12451: pydoc: html_getfile() now uses tokenize.open() to support + Python scripts using a encoding different than UTF-8 (read the coding cookie + of the script). + +- Issue #12493: subprocess: Popen.communicate() now also handles EINTR errors + if the process has only one pipe. + +- Issue #12467: warnings: fix a race condition if a warning is emitted at + shutdown, if globals()['__file__'] is None. + +- Issue #12451: pydoc: importfile() now opens the Python script in binary mode, + instead of text mode using the locale encoding, to avoid encoding issues. + +- Issue #12451: runpy: run_path() now opens the Python script in binary mode, + instead of text mode using the locale encoding, to support other encodings + than UTF-8 (scripts using the coding cookie). + +- Issue #12451: xml.dom.pulldom: parse() now opens files in binary mode instead + of the text mode (using the locale encoding) to avoid encoding issues. + - Issue #12147: Adjust the new-in-3.2 smtplib.send_message method for better - conformance to the RFCs: correctly handle Sender and Resent headers. + conformance to the RFCs: correctly handle Sender and Resent- headers. - Issue #12352: Fix a deadlock in multiprocessing.Heap when a block is freed by the garbage collector while the Heap lock is held.