From: Benjamin Peterson Date: Sat, 27 Jun 2015 18:56:46 +0000 (-0500) Subject: merge 3.3 X-Git-Tag: v3.5.0b3~41^2 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=ff0f322edbb200c1dadd287c3355a5bc006d7fa6;p=python merge 3.3 --- ff0f322edbb200c1dadd287c3355a5bc006d7fa6 diff --cc Misc/NEWS index 7205e93841,29914fc9ea..ee3bcf7865 --- a/Misc/NEWS +++ b/Misc/NEWS @@@ -10,283 -10,281 +10,285 @@@ Release date: tb Core and Builtins ----------------- -- Issue #24096: Make warnings.warn_explicit more robust against mutation of the - warnings.filters list. - -- Issue #24044: Fix possible null pointer dereference in list.sort in out of - memory conditions. - -- Issue #23055: Fixed a buffer overflow in PyUnicode_FromFormatV. Analysis - and fix by Guido Vranken. +- Issue #24115: Update uses of PyObject_IsTrue(), PyObject_Not(), + PyObject_IsInstance(), PyObject_RichCompareBool() and _PyDict_Contains() + to check for and handle errors correctly. -Library -------- +- Issue #24257: Fixed system error in the comparison of faked + types.SimpleNamespace. -- Fix possible integer overflows in the pickle module. +- Issue #22939: Fixed integer overflow in iterator object. Patch by + Clement Rouault. -- Issue #22931: Allow '[' and ']' in cookie values. +- Issue #23985: Fix a possible buffer overrun when deleting a slice from + the front of a bytearray and then appending some other bytes data. -- Issue #24094: Fix possible crash in json.encode with poorly behaved dict - subclasses. +- Issue #24102: Fixed exception type checking in standard error handlers. -- Issue #23367: Fix possible overflows in the unicodedata module. +- Issue #23757: PySequence_Tuple() incorrectly called the concrete list API + when the data was a list subclass. -- Issue #23361: Fix possible overflow in Windows subprocess creation code. +- Issue #24096: Make warnings.warn_explicit more robust against mutation of the + warnings.filters list. -- Issue #23363: Fix possible overflow in itertools.permutations. +- Issue #23996: Avoid a crash when a delegated generator raises an + unnormalized StopIteration exception. Patch by Stefan Behnel. -- Issue #23364: Fix possible overflow in itertools.product. +- Issue #24022: Fix tokenizer crash when processing undecodable source code. -- Issue #23369: Fixed possible integer overflow in - _json.encode_basestring_ascii. +- Issue #23309: Avoid a deadlock at shutdown if a daemon thread is aborted + while it is holding a lock to a buffered I/O object, and the main thread + tries to use the same I/O object (typically stdout or stderr). A fatal + error is emitted instead. -- Issue #23366: Fixed possible integer overflow in itertools.combinations. +- Issue #22977: Fixed formatting Windows error messages on Wine. + Patch by Martin Panter. -- Issue #23365: Fixed possible integer overflow in - itertools.combinations_with_replacement. +- Issue #23803: Fixed str.partition() and str.rpartition() when a separator + is wider then partitioned string. -C API ------ +- Issue #23192: Fixed generator lambdas. Patch by Bruno Cauet. -- Issue #23998: PyImport_ReInitLock() now checks for lock allocation error +- Issue #23629: Fix the default __sizeof__ implementation for variable-sized + objects. +- Issue #24044: Fix possible null pointer dereference in list.sort in out of + memory conditions. -What's New in Python 3.3.6? -=========================== +- Issue #21354: PyCFunction_New function is exposed by python DLL again. -*Release date: 11-Oct-2014* +Library +------- -Core and Builtins ------------------ +- Issue #24489: ensure a previously set C errno doesn't disturb cmath.polar(). -- Issue #22643: Fix integer overflow in Unicode case operations (upper, lower, - title, swapcase, casefold). +- Issue #5633: Fixed timeit when the statement is a string and the setup is not. -- Issue #22518: Fixed integer overflow issues in "backslashreplace", - "xmlcharrefreplace", and "surrogatepass" error handlers. +- Issue #24326: Fixed audioop.ratecv() with non-default weightB argument. + Original patch by David Moore. -- Issue #22520: Fix overflow checking when generating the repr of a unicode - object. +- Issue #23840: tokenize.open() now closes the temporary binary file on error + to fix a resource warning. -- Issue #22519: Fix overflow checking in PyBytes_Repr. +- Issue #24257: Fixed segmentation fault in sqlite3.Row constructor with faked + cursor type. -- Issue #22518: Fix integer overflow issues in latin-1 encoding. +- Issue #22107: tempfile.gettempdir() and tempfile.mkdtemp() now try again + when a directory with the chosen name already exists on Windows as well as + on Unix. tempfile.mkstemp() now fails early if parent directory is not + valid (not exists or is a file) on Windows. -- Issue #23165: Perform overflow checks before allocating memory in the - _Py_char2wchar function. +- Issue #6598: Increased time precision and random number range in + email.utils.make_msgid() to strengthen the uniqueness of the message ID. -Library -------- +- Issue #24091: Fixed various crashes in corner cases in C implementation of + ElementTree. -- Issue #16043: Add a default limit for the amount of data xmlrpclib.gzip_decode - will return. This resolves CVE-2013-1753. +- Issue #21931: msilib.FCICreate() now raises TypeError in the case of a bad + argument instead of a ValueError with a bogus FCI error number. + Patch by Jeffrey Armstrong. -- Issue #22517: When a io.BufferedRWPair object is deallocated, clear its - weakrefs. +- Issue #23796: peek and read1 methods of BufferedReader now raise ValueError + if they called on a closed object. Patch by John Hergenroeder. -- Issue #22419: Limit the length of incoming HTTP request in wsgiref server to - 65536 bytes and send a 414 error code for higher lengths. Patch contributed - by Devin Cook. ++- Fix possible integer overflows in the pickle module. + -- Lax cookie parsing in http.cookies could be a security issue when combined - with non-standard cookie handling in some Web browsers. Reported by - Sergey Bobrov. +- Issue #22931: Allow '[' and ']' in cookie values. -- Issue #21766: Prevent a security hole in CGIHTTPServer by URL unquoting paths - before checking for a CGI script at that path. +- Issue #20274: Remove ignored and erroneous "kwargs" parameters from three + METH_VARARGS methods on _sqlite.Connection. -- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second - parameter. Bug reported by Guido Vranken. +- Issue #24094: Fix possible crash in json.encode with poorly behaved dict + subclasses. -- Issue #20633: Replace relative import by absolute import. +- Asyncio issue 222 / PR 231 (Victor Stinner) -- fix @coroutine + functions without __name__. -- Issue #21082: In os.makedirs, do not set the process-wide umask. Note this - changes behavior of makedirs when exist_ok=True. +- Issue #9246: On POSIX, os.getcwd() now supports paths longer than 1025 bytes. + Patch written by William Orr. -- Issue #20875: Prevent possible gzip "'read' is not defined" NameError. - Patch by Claudiu Popa. +- The keywords attribute of functools.partial is now always a dictionary. -- Issue #11599: When an external command (e.g. compiler) fails, distutils now - prints out the whole command line (instead of just the command name) if the - environment variable DISTUTILS_DEBUG is set. +- Issues #24099, #24100, and #24101: Fix free-after-use bug in heapq's siftup + and siftdown functions. -- Issue #4931: distutils should not produce unhelpful "error: None" messages - anymore. distutils.util.grok_environment_error is kept but doc-deprecated. +- Backport collections.deque fixes from Python 3.5. Prevents reentrant badness + during deletion by deferring the decref until the container has been restored + to a consistent state. -- Issue #20283: RE pattern methods now accept the string keyword parameters - as documented. The pattern and source keyword parameters are left as - deprecated aliases. +- Issue #23008: Fixed resolving attributes with boolean value is False in pydoc. -- Issue #21323: Fix http.server to again handle scripts in CGI subdirectories, - broken by the fix for security issue #19435. Patch by Zach Byrne. +- Fix asyncio issue 235: LifoQueue and PriorityQueue's put didn't + increment unfinished tasks (this bug was introduced in 3.4.3 when + JoinableQueue was merged with Queue). -Tests ------ +- Issue #23908: os functions now reject paths with embedded null character + on Windows instead of silently truncate them. -- Issue #17752: Fix distutils tests when run from the installed location. +- Issue #23728: binascii.crc_hqx() could return an integer outside of the range + 0-0xffff for empty data. -- Issue #20946: Correct alignment assumptions of some ctypes tests. +- Issue #23811: Add missing newline to the PyCompileError error message. + Patch by Alex Shkop. -- Issue #20939: Fix test_geturl failure in test_urllibnet due to - new redirect of http://www.python.org/ to https://www.python.org. +- Issue #17898: Fix exception in gettext.py when parsing certain plural forms. +- Issue #22982: Improve BOM handling when seeking to multiple positions of + a writable text file. -What's New in Python 3.3.5? -=========================== +- Issue #23865: close() methods in multiple modules now are idempotent and more + robust at shutdown. If they need to release multiple resources, all are + released even if errors occur. -*Release date: 09-Mar-2014* +- Issue #23881: urllib.request.ftpwrapper constructor now closes the socket if + the FTP connection failed to fix a ResourceWarning. -No changes from release candidate 2. +- Issue #23400: Raise same exception on both Python 2 and 3 if sem_open is not + available. Patch by Davin Potts. +- Issue #15133: _tkinter.tkapp.getboolean() now supports Tcl_Obj and always + returns bool. tkinter.BooleanVar now validates input values (accepted bool, + int, str, and Tcl_Obj). tkinter.BooleanVar.get() now always returns bool. -What's New in Python 3.3.5 release candidate 2? -=============================================== +- Issue #23338: Fixed formatting ctypes error messages on Cygwin. + Patch by Makoto Kato. -*Release date: 02-Mar-2014* +- Issue #16840: Tkinter now supports 64-bit integers added in Tcl 8.4 and + arbitrary precision integers added in Tcl 8.5. -Core and Builtins ------------------ +- Issue #23834: Fix socket.sendto(), use the C Py_ssize_t type to store the + result of sendto() instead of the C int type. -- Issue #20731: Properly position in source code files even if they - are opened in text mode. Patch by Serhiy Storchaka. +- Issue #21526: Tkinter now supports new boolean type in Tcl 8.5. -- Issue #19619: str.encode, bytes.decode and bytearray.decode now use an - internal API to throw LookupError for known non-text encodings, rather - than attempting the encoding or decoding operation and then throwing a - TypeError for an unexpected output type. (The latter mechanism remains - in place for third party non-text encodings) +- Issue #23838: linecache now clears the cache and returns an empty result on + MemoryError. -Library -------- +- Issue #18473: Fixed 2to3 and 3to2 compatible pickle mappings. Fixed + ambigious reverse mappings. Added many new mappings. Import mapping is no + longer applied to modules already mapped with full name mapping. -- Issue #20778: Fix modulefinder to work with bytecode-only modules. +- Issue #23745: The new email header parser now handles duplicate MIME + parameter names without error, similar to how get_param behaves. -- Issue #20791: copy.copy() now doesn't make a copy when the input is - a bytes object. Initial patch by Peter Otten. +- Issue #23792: Ignore KeyboardInterrupt when the pydoc pager is active. + This mimics the behavior of the standard unix pagers, and prevents + pipepager from shutting down while the pager itself is still running. -- Issue #20621: Fixes a zipimport bug introduced in 3.3.4 that could cause - spurious crashes or SystemErrors when importing modules or packages from a - zip file. The change causing the problem was reverted. +- Issue #23742: ntpath.expandvars() no longer loses unbalanced single quotes. -- Issue #20404: io.TextIOWrapper (and hence the open() builtin) now uses the - internal codec marking system added for issue #19619 to throw LookupError - for known non-text encodings at stream construction time. The existing - output type checks remain in place to deal with unmarked third party - codecs. +- Issue #21802: The reader in BufferedRWPair now is closed even when closing + writer failed in BufferedRWPair.close(). -Tests ------ +- Issue #23671: string.Template now allows to specify the "self" parameter as + keyword argument. string.Formatter now allows to specify the "self" and + the "format_string" parameters as keyword arguments. -- Issue #20743: Fix a reference leak in test_tcl. +- Issue #21560: An attempt to write a data of wrong type no longer cause + GzipFile corruption. Original patch by Wolfgang Maier. -Tools/Demos ------------ +- Issue #23647: Increase impalib's MAXLINE to accommodate modern mailbox sizes. -- Issue #20535: PYTHONWARNING no longer affects the run_tests.py script. - Patch by Arfrever Frehtes Taifersar Arahesis. +- Issue #23539: If body is None, http.client.HTTPConnection.request now sets + Content-Length to 0 for PUT, POST, and PATCH headers to avoid 411 errors from + some web servers. +- Issue #22351: The nntplib.NNTP constructor no longer leaves the connection + and socket open until the garbage collector cleans them up. Patch by + Martin Panter. -What's New in Python 3.3.5 release candidate 1? -=============================================== +- Issue #23136: _strptime now uniformly handles all days in week 0, including + Dec 30 of previous year. Based on patch by Jim Carroll. -*Release date: 23-Feb-2014* +- Issue #23700: Iterator of NamedTemporaryFile now keeps a reference to + NamedTemporaryFile instance. Patch by Bohuslav Kabrda. -Core and Builtins ------------------ +- Issue #22903: The fake test case created by unittest.loader when it fails + importing a test module is now picklable. -- Issue #20588: Make Python-ast.c C89 compliant. +- Issue #23568: Add rdivmod support to MagicMock() objects. + Patch by Håkan Lövdahl. -- Issue #20437: Fixed 21 potential bugs when deleting objects references. +- Issue #23138: Fixed parsing cookies with absent keys or values in cookiejar. + Patch by Demian Brecht. -- Issue #20538: UTF-7 incremental decoder produced inconsistant string when - input was truncated in BASE64 section. +- Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now + handle exceptions raised by an iterator. Patch by Alon Diamant and Davin + Potts. -Library -------- +- Issue #22928: Disabled HTTP header injections in http.client. + Original patch by Demian Brecht. -- Issue #20635: Fixed grid_columnconfigure() and grid_rowconfigure() methods of - Tkinter widgets to work in wantobjects=True mode. +- Issue #23615: Modules bz2, tarfile and tokenize now can be reloaded with + imp.reload(). Patch by Thomas Kluyver. -- Issue #19612: On Windows, subprocess.Popen.communicate() now ignores - OSError(22, 'Invalid argument') when writing input data into stdin, whereas - the process already exited. +- Issue #23476: In the ssl module, enable OpenSSL's X509_V_FLAG_TRUSTED_FIRST + flag on certificate stores when it is available. -- Issue #6815: os.path.expandvars() now supports non-ASCII environment - variables names and values. +- Issue #23576: Avoid stalling in SSL reads when EOF has been reached in the + SSL layer but the underlying connection hasn't been closed. -- Issue #17671: Fixed a crash when use non-initialized io.BufferedRWPair. - Based on patch by Stephen Tu. +- Issue #23504: Added an __all__ to the types module. -- Issue #8478: Untokenizer.compat processes first token from iterator input. - Patch based on lines from Georg Brandl, Eric Snow, and Gareth Rees. +- Issue #20204: Added the __module__ attribute to _tkinter classes. -- Issue #20594: Avoid name clash with the libc function posix_close. +- Issue #23521: Corrected pure python implementation of timedelta division. -- Issue #19856: shutil.move() failed to move a directory to other directory - on Windows if source name ends with os.altsep. + * Eliminated OverflowError from timedelta * float for some floats; + * Corrected rounding in timedlta true division. -- Issue #14983: email.generator now always adds a line end after each MIME - boundary marker, instead of doing so only when there is an epilogue. This - fixes an RFC compliance bug and solves an issue with signed MIME parts. +- Issue #21619: Popen objects no longer leave a zombie after exit in the with + statement if the pipe was broken. Patch by Martin Panter. -- Issue #20540: Fix a performance regression (vs. Python 3.2) when layering - a multiprocessing Connection over a TCP socket. For small payloads, Nagle's - algorithm would introduce idle delays before the entire transmission of a - message. +- Issue #6639: Module-level turtle functions no longer raise TclError after + closing the window. -- Issue #16983: the new email header parsing code will now decode encoded words - that are (incorrectly) surrounded by quotes, and register a defect. +- Issues #814253, #9179: Warnings now are raised when group references and + conditional group references are used in lookbehind assertions in regular + expressions. -- Issue #19772: email.generator no longer mutates the message object when - doing a down-transform from 8bit to 7bit CTEs. +- Issue #23215: Multibyte codecs with custom error handlers that ignores errors + consumed too much memory and raised SystemError or MemoryError. + Original patch by Aleksi Torhamo. -- Issue #18805: the netmask/hostmask parsing in ipaddress now more reliably - filters out illegal values and correctly allows any valid prefix length. +- Issue #5700: io.FileIO() called flush() after closing the file. + flush() was not called in close() if closefd=False. -- Issue #17369: get_filename was raising an exception if the filename - parameter's RFC2231 encoding was broken in certain ways. This was - a regression relative to python2. +- Issue #23374: Fixed pydoc failure with non-ASCII files when stdout encoding + differs from file system encoding (e.g. on Mac OS). -- Issue #20013: Some imap servers disconnect if the current mailbox is - deleted, and imaplib did not handle that case gracefully. Now it - handles the 'bye' correctly. +- Issue #23481: Remove RC4 from the SSL module's default cipher list. -- Issue #19920: TarFile.list() no longer fails when outputs a listing - containing non-encodable characters. Based on patch by Vajrasky Kok. +- Issue #21548: Fix pydoc.synopsis() and pydoc.apropos() on modules with empty + docstrings. -- Issue #20515: Fix NULL pointer dereference introduced by issue #20368. +- Issue #22885: Fixed arbitrary code execution vulnerability in the dbm.dumb + module. Original patch by Claudiu Popa. -- Issue #19186: Restore namespacing of expat symbols inside the pyexpat module. +- Issue #23146: Fix mishandling of absolute Windows paths with forward + slashes in pathlib. -- Issue #20426: When passing the re.DEBUG flag, re.compile() displays the - debug output every time it is called, regardless of the compilation cache. +- Issue #23421: Fixed compression in tarfile CLI. Patch by wdv4758h. -- Issue #20368: The null character now correctly passed from Tcl to Python. - Improved error handling in variables-related commands. +- Issue #23367: Fix possible overflows in the unicodedata module. -- Issue #20435: Fix _pyio.StringIO.getvalue() to take into account newline - translation settings. +- Issue #23361: Fix possible overflow in Windows subprocess creation code. -- Issue #20288: fix handling of invalid numeric charrefs in HTMLParser. +- Issue #23801: Fix issue where cgi.FieldStorage did not always ignore the + entire preamble to a multipart body. -- Issue #20424: Python implementation of io.StringIO now supports lone surrogates. +- Issue #23310: Fix MagicMock's initializer to work with __methods__, just + like configure_mock(). Patch by Kasia Jachim. -- Issue #19456: ntpath.join() now joins relative paths correctly when a drive - is present. +- asyncio: New event loop APIs: set_task_factory() and get_task_factory(). -- Issue #19077: tempfile.TemporaryDirectory cleanup is now most likely - successful when called during nulling out of modules during shutdown. - Misleading exception no longer raised when resource warning is emitted - during shutdown. +- asyncio: async() function is deprecated in favour of ensure_future(). -- Issue #20367: Fix behavior of concurrent.futures.as_completed() for - duplicate arguments. Patch by Glenn Langford. +- Issue #23898: Fix inspect.classify_class_attrs() to support attributes + with overloaded __eq__ and __bool__. Patch by Mike Bayer. -- Issue #8260: The read(), readline() and readlines() methods of - codecs.StreamReader returned incomplete data when were called after - readline() or read(size). Based on patch by Amaury Forgeot d'Arc. +- Issue #24298: Fix inspect.signature() to correctly unwrap wrappers + around bound methods. IDLE ---- diff --cc Modules/_pickle.c index 435cbaea9d,ec8bd6c3ba..19a1c88bd1 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@@ -660,10 -431,9 +658,10 @@@ PyMemoTable_Copy(PyMemoTable *self /* The table we get from _New() is probably smaller than we wanted. Free it and allocate one that's the right size. */ PyMem_FREE(new->mt_table); - new->mt_table = PyMem_MALLOC(self->mt_allocated * sizeof(PyMemoEntry)); + new->mt_table = PyMem_NEW(PyMemoEntry, self->mt_allocated); if (new->mt_table == NULL) { PyMem_FREE(new); + PyErr_NoMemory(); return NULL; } for (i = 0; i < self->mt_allocated; i++) { @@@ -755,9 -525,9 +753,9 @@@ _PyMemoTable_ResizeTable(PyMemoTable *s /* Allocate new table. */ oldtable = self->mt_table; - self->mt_table = PyMem_MALLOC(new_size * sizeof(PyMemoEntry)); + self->mt_table = PyMem_NEW(PyMemoEntry, new_size); if (self->mt_table == NULL) { - PyMem_FREE(oldtable); + self->mt_table = oldtable; PyErr_NoMemory(); return -1; } @@@ -1314,11 -1099,9 +1310,11 @@@ _Unpickler_MemoPut(UnpicklerObject *sel static PyObject ** _Unpickler_NewMemo(Py_ssize_t new_size) { - PyObject **memo = PyMem_MALLOC(new_size * sizeof(PyObject *)); + PyObject **memo = PyMem_NEW(PyObject *, new_size); - if (memo == NULL) + if (memo == NULL) { + PyErr_NoMemory(); return NULL; + } memset(memo, 0, new_size * sizeof(PyObject *)); return memo; }