From: Antoine Pitrou Date: Tue, 16 Sep 2014 22:25:57 +0000 (+0200) Subject: Lax cookie parsing in http.cookies could be a security issue when combined X-Git-Tag: v3.4.2rc1~23 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=637e4544afda57d52c81bddba5486bda9574e6b2;p=python Lax cookie parsing in http.cookies could be a security issue when combined with non-standard cookie handling in some Web browsers. Reported by Sergey Bobrov. --- 637e4544afda57d52c81bddba5486bda9574e6b2 diff --cc Misc/NEWS index 9ba754a773,1f389f8714..e411cdbaf4 --- a/Misc/NEWS +++ b/Misc/NEWS @@@ -32,673 -13,665 +32,677 @@@ Core and Builtin Library ------- + - Lax cookie parsing in http.cookies could be a security issue when combined + with non-standard cookie handling in some Web browsers. Reported by + Sergey Bobrov. + -- Issue #21766: Prevent a security hole in CGIHTTPServer by URL unquoting paths - before checking for a CGI script at that path. +- Issue #22384: An exception in Tkinter callback no longer crashes the program + when it is run with pythonw.exe. -- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second - parameter. Bug reported by Guido Vranken. +- Issue #22168: Prevent turtle AttributeError with non-default Canvas on OS X. -- Issue #20633: Replace relative import by absolute import. +- Issue #21147: sqlite3 now raises an exception if the request contains a null + character instead of truncate it. Based on patch by Victor Stinner. -- Issue #21082: In os.makedirs, do not set the process-wide umask. Note this - changes behavior of makedirs when exist_ok=True. +- Issue #21951: Fixed a crash in Tkinter on AIX when called Tcl command with + empty string or tuple argument. -- Issue #20875: Prevent possible gzip "'read' is not defined" NameError. - Patch by Claudiu Popa. +- Issue #21951: Tkinter now most likely raises MemoryError instead of crash + if the memory allocation fails. -- Issue #11599: When an external command (e.g. compiler) fails, distutils now - prints out the whole command line (instead of just the command name) if the - environment variable DISTUTILS_DEBUG is set. +- Issue #22338: Fix a crash in the json module on memory allocation failure. -- Issue #4931: distutils should not produce unhelpful "error: None" messages - anymore. distutils.util.grok_environment_error is kept but doc-deprecated. +- Issue #22226: First letter no longer is stripped from the "status" key in + the result of Treeview.heading(). -- Issue #20283: RE pattern methods now accept the string keyword parameters - as documented. The pattern and source keyword parameters are left as - deprecated aliases. +- Issue #19524: Fixed resource leak in the HTTP connection when an invalid + response is received. Patch by Martin Panter. -- Issue #21323: Fix http.server to again handle scripts in CGI subdirectories, - broken by the fix for security issue #19435. Patch by Zach Byrne. +- Issue #22051: turtledemo no longer reloads examples to re-run them. + Initialization of variables and gui setup should be done in main(), + which is called each time a demo is run, but not on import. -Tests ------ +- Issue #21933: Turtledemo users can change the code font size with a menu + selection or control(command) '-' or '+' or control-mousewheel. + Original patch by Lita Cho. -- Issue #17752: Fix distutils tests when run from the installed location. +_ Issue #21597: The separator between the turtledemo text pane and the drawing + canvas can now be grabbed and dragged with a mouse. The code text pane can + be widened to easily view or copy the full width of the text. The canvas + can be widened on small screens. Original patches by Jan Kanis and Lita Cho. -- Issue #20946: Correct alignment assumptions of some ctypes tests. +- Issue #18132: Turtledemo buttons no longer disappear when the window is + shrunk. Original patches by Jan Kanis and Lita Cho. -- Issue #20939: Fix test_geturl failure in test_urllibnet due to - new redirect of http://www.python.org/ to https://www.python.org. +- Issue #22216: smtplib now resets its state more completely after a quit. The + most obvious consequence of the previous behavior was a STARTTLS failure + during a connect/starttls/quit/connect/starttls sequence. +- Issue #22185: Fix an occasional RuntimeError in threading.Condition.wait() + caused by mutation of the waiters queue without holding the lock. Patch + by Doug Zongker. -What's New in Python 3.3.5? -=========================== +- Issue #22182: Use e.args to unpack exceptions correctly in + distutils.file_util.move_file. Patch by Claudiu Popa. -*Release date: 09-Mar-2014* +- The webbrowser module now uses subprocess's start_new_session=True rather + than a potentially risky preexec_fn=os.setsid call. -No changes from release candidate 2. +- Issue #22236: Fixed Tkinter images copying operations in NoDefaultRoot mode. +- Issue #22191: Fix warnings.__all__. -What's New in Python 3.3.5 release candidate 2? -=============================================== +- Issue #15696: Add a __sizeof__ implementation for mmap objects on Windows. -*Release date: 02-Mar-2014* +- Issue #22068: Avoided reference loops with Variables and Fonts in Tkinter. -Core and Builtins ------------------ +- Issue #22165: SimpleHTTPRequestHandler now supports undecodable file names. -- Issue #20731: Properly position in source code files even if they - are opened in text mode. Patch by Serhiy Storchaka. +- Issue #8797: Raise HTTPError on failed Basic Authentication immediately. + Initial patch by Sam Bull. -- Issue #19619: str.encode, bytes.decode and bytearray.decode now use an - internal API to throw LookupError for known non-text encodings, rather - than attempting the encoding or decoding operation and then throwing a - TypeError for an unexpected output type. (The latter mechanism remains - in place for third party non-text encodings) +- Issue #20729: Restored the use of lazy iterkeys()/itervalues()/iteritems() + in the mailbox module. -Library -------- +- Issue #21448: Changed FeedParser feed() to avoid O(N**2) behavior when + parsing long line. Original patch by Raymond Hettinger. -- Issue #20778: Fix modulefinder to work with bytecode-only modules. +- Issue #22184: The functools LRU Cache decorator factory now gives an earlier + and clearer error message when the user forgets the required parameters. -- Issue #20791: copy.copy() now doesn't make a copy when the input is - a bytes object. Initial patch by Peter Otten. +- Issue #17923: glob() patterns ending with a slash no longer match non-dirs on + AIX. Based on patch by Delhallt. -- Issue #20621: Fixes a zipimport bug introduced in 3.3.4 that could cause - spurious crashes or SystemErrors when importing modules or packages from a - zip file. The change causing the problem was reverted. +- Issue #21121: Don't force 3rd party C extensions to be built with + -Werror=declaration-after-statement. -- Issue #20404: io.TextIOWrapper (and hence the open() builtin) now uses the - internal codec marking system added for issue #19619 to throw LookupError - for known non-text encodings at stream construction time. The existing - output type checks remain in place to deal with unmarked third party - codecs. +- Issue #21975: Fixed crash when using uninitialized sqlite3.Row (in particular + when unpickling pickled sqlite3.Row). sqlite3.Row is now initialized in the + __new__() method. -Tests ------ +- Issue #21580: Now Tkinter correctly handles bytes arguments passed to Tk. + In particular this allows to initialize images from binary data. -- Issue #20743: Fix a reference leak in test_tcl. +- Issue #17172: Make turtledemo start as active on OS X even when run with + subprocess. Patch by Lita Cho. -Tools/Demos ------------ +- Issue #21704: Fix build error for _multiprocessing when semaphores + are not available. Patch by Arfrever Frehtes Taifersar Arahesis. -- Issue #20535: PYTHONWARNING no longer affects the run_tests.py script. - Patch by Arfrever Frehtes Taifersar Arahesis. +- Fix repr(_socket.socket) on Windows 64-bit: don't fail with OverflowError + on closed socket. repr(socket.socket) already works fine. +- Issue #16133: The asynchat.async_chat.handle_read() method now ignores + BlockingIOError exceptions. -What's New in Python 3.3.5 release candidate 1? -=============================================== +- Issue #22044: Fixed premature DECREF in call_tzinfo_method. + Patch by Tom Flanagan. -*Release date: 23-Feb-2014* +- Issue #19884: readline: Disable the meta modifier key if stdout is not + a terminal to not write the ANSI sequence "\033[1034h" into stdout. This + sequence is used on some terminal (ex: TERM=xterm-256color") to enable + support of 8 bit characters. -Core and Builtins ------------------ +- Issue #21888: plistlib's load() and loads() now work if the fmt parameter is + specified. -- Issue #20588: Make Python-ast.c C89 compliant. +- Issue #21044: tarfile.open() now handles fileobj with an integer 'name' + attribute. Based on patch by Antoine Pietri. -- Issue #20437: Fixed 21 potential bugs when deleting objects references. +- Issue #21867: Prevent turtle crash due to invalid undo buffer size. -- Issue #20538: UTF-7 incremental decoder produced inconsistant string when - input was truncated in BASE64 section. +- Issue #19076: Don't pass the redundant 'file' argument to self.error(). -Library -------- +- Issue #21942: Fixed source file viewing in pydoc's server mode on Windows. -- Issue #20635: Fixed grid_columnconfigure() and grid_rowconfigure() methods of - Tkinter widgets to work in wantobjects=True mode. +- Issue #11259: asynchat.async_chat().set_terminator() now raises a ValueError + if the number of received bytes is negative. -- Issue #19612: On Windows, subprocess.Popen.communicate() now ignores - OSError(22, 'Invalid argument') when writing input data into stdin, whereas - the process already exited. +- Issue #12523: asynchat.async_chat.push() now raises a TypeError if it doesn't + get a bytes string -- Issue #6815: os.path.expandvars() now supports non-ASCII environment - variables names and values. +- Issue #21707: Add missing kwonlyargcount argument to + ModuleFinder.replace_paths_in_code(). -- Issue #17671: Fixed a crash when use non-initialized io.BufferedRWPair. - Based on patch by Stephen Tu. +- Issue #20639: calling Path.with_suffix('') allows removing the suffix + again. Patch by July Tikhonov. -- Issue #8478: Untokenizer.compat processes first token from iterator input. - Patch based on lines from Georg Brandl, Eric Snow, and Gareth Rees. +- Issue #21714: Disallow the construction of invalid paths using + Path.with_name(). Original patch by Antony Lee. -- Issue #20594: Avoid name clash with the libc function posix_close. +- Issue #21897: Fix a crash with the f_locals attribute with closure + variables when frame.clear() has been called. -- Issue #19856: shutil.move() failed to move a directory to other directory - on Windows if source name ends with os.altsep. +- Issue #21151: Fixed a segfault in the winreg module when ``None`` is passed + as a ``REG_BINARY`` value to SetValueEx. Patch by John Ehresman. -- Issue #14983: email.generator now always adds a line end after each MIME - boundary marker, instead of doing so only when there is an epilogue. This - fixes an RFC compliance bug and solves an issue with signed MIME parts. +- Issue #21090: io.FileIO.readall() does not ignore I/O errors anymore. Before, + it ignored I/O errors if at least the first C call read() succeed. -- Issue #20540: Fix a performance regression (vs. Python 3.2) when layering - a multiprocessing Connection over a TCP socket. For small payloads, Nagle's - algorithm would introduce idle delays before the entire transmission of a - message. +- Issue #21781: ssl.RAND_add() now supports strings longer than 2 GB. -- Issue #16983: the new email header parsing code will now decode encoded words - that are (incorrectly) surrounded by quotes, and register a defect. +- Issue #11453: asyncore: emit a ResourceWarning when an unclosed file_wrapper + object is destroyed. The destructor now closes the file if needed. The + close() method can now be called twice: the second call does nothing. -- Issue #19772: email.generator no longer mutates the message object when - doing a down-transform from 8bit to 7bit CTEs. +- Issue #21858: Better handling of Python exceptions in the sqlite3 module. -- Issue #18805: the netmask/hostmask parsing in ipaddress now more reliably - filters out illegal values and correctly allows any valid prefix length. +- Issue #21476: Make sure the email.parser.BytesParser TextIOWrapper is + discarded after parsing, so the input file isn't unexpectedly closed. -- Issue #17369: get_filename was raising an exception if the filename - parameter's RFC2231 encoding was broken in certain ways. This was - a regression relative to python2. +- Issue #21729: Used the "with" statement in the dbm.dumb module to ensure + files closing. Patch by Claudiu Popa. -- Issue #20013: Some imap servers disconnect if the current mailbox is - deleted, and imaplib did not handle that case gracefully. Now it - handles the 'bye' correctly. +- Issue #21491: socketserver: Fix a race condition in child processes reaping. -- Issue #19920: TarFile.list() no longer fails when outputs a listing - containing non-encodable characters. Based on patch by Vajrasky Kok. +- Issue #21832: Require named tuple inputs to be exact strings. -- Issue #20515: Fix NULL pointer dereference introduced by issue #20368. +- Issue #19145: The times argument for itertools.repeat now handles + negative values the same way for keyword arguments as it does for + positional arguments. -- Issue #19186: Restore namespacing of expat symbols inside the pyexpat module. +- Issue #21812: turtle.shapetransform did not tranform the turtle on the + first call. (Issue identified and fixed by Lita Cho.) -- Issue #20426: When passing the re.DEBUG flag, re.compile() displays the - debug output every time it is called, regardless of the compilation cache. +- Issue #21635: The difflib SequenceMatcher.get_matching_blocks() method + cache didn't match the actual result. The former was a list of tuples + and the latter was a list of named tuples. -- Issue #20368: The null character now correctly passed from Tcl to Python. - Improved error handling in variables-related commands. +- Issue #21722: The distutils "upload" command now exits with a non-zero + return code when uploading fails. Patch by Martin Dengler. -- Issue #20435: Fix _pyio.StringIO.getvalue() to take into account newline - translation settings. +- Issue #21723: asyncio.Queue: support any type of number (ex: float) for the + maximum size. Patch written by Vajrasky Kok. -- Issue #20288: fix handling of invalid numeric charrefs in HTMLParser. +- Issue #21326: Add a new is_closed() method to asyncio.BaseEventLoop. + run_forever() and run_until_complete() methods of asyncio.BaseEventLoop now + raise an exception if the event loop was closed. -- Issue #20424: Python implementation of io.StringIO now supports lone surrogates. +- Issue #21774: Fixed NameError for an incorrect variable reference in the + XML Minidom code for creating processing instructions. + (Found and fixed by Claudiu Popa.) -- Issue #19456: ntpath.join() now joins relative paths correctly when a drive - is present. +- Issue #21766: Prevent a security hole in CGIHTTPServer by URL unquoting paths + before checking for a CGI script at that path. -- Issue #19077: tempfile.TemporaryDirectory cleanup is now most likely - successful when called during nulling out of modules during shutdown. - Misleading exception no longer raised when resource warning is emitted - during shutdown. +- Issue #21310: Fixed possible resource leak in failed open(). -- Issue #20367: Fix behavior of concurrent.futures.as_completed() for - duplicate arguments. Patch by Glenn Langford. +- Issue #21677: Fixed chaining nonnormalized exceptions in io close() methods. -- Issue #8260: The read(), readline() and readlines() methods of - codecs.StreamReader returned incomplete data when were called after - readline() or read(size). Based on patch by Amaury Forgeot d'Arc. +- Issue #11709: Fix the pydoc.help function to not fail when sys.stdin is not a + valid file. -IDLE ----- +- Issue #13223: Fix pydoc.writedoc so that the HTML documentation for methods + that use 'self' in the example code is generated correctly. -- Issue #20406: Use Python application icons for Idle window title bars. - Patch mostly by Serhiy Storchaka. +- Issue #21463: In urllib.request, fix pruning of the FTP cache. -- Update the python.gif icon for the Idle classbrowser and pathbowser - from the old green snake to the new new blue and yellow snakes. +- Issue #21618: The subprocess module could fail to close open fds that were + inherited by the calling process and already higher than POSIX resource + limits would otherwise allow. On systems with a functioning /proc/self/fd + or /dev/fd interface the max is now ignored and all fds are closed. -- Issue #17721: Remove non-functional configuration dialog help button until we - make it actually gives some help when clicked. Patch by Guilherme Simões. +- Issue #21552: Fixed possible integer overflow of too long string lengths in + the tkinter module on 64-bit platforms. -Tests ------ +- Issue #14315: The zipfile module now ignores extra fields in the central + directory that are too short to be parsed instead of letting a struct.unpack + error bubble up as this "bad data" appears in many real world zip files in + the wild and is ignored by other zip tools. -- Issue #20510: Rewrote test_exit in test_sys to match existing comments, - use modern unittest features, and use helpers from test.script_helper - instead of using subprocess directly. Patch by Gareth Rees. +- Issue #21402: tkinter.ttk now works when default root window is not set. -- Issue #20532: Tests which use _testcapi are now marked as CPython only. +- Issue #10203: sqlite3.Row now truly supports sequence protocol. In particulr + it supports reverse() and negative indices. Original patch by Claudiu Popa. -- Issue #19920: Added tests for TarFile.list(). Based on patch by Vajrasky Kok. +- Issue #18807: If copying (no symlinks) specified for a venv, then the python + interpreter aliases (python, python3) are now created by copying rather than + symlinking. -- Issue #19990: Added tests for the imghdr module. Based on patch by - Claudiu Popa. +- Issue #14710: pkgutil.get_loader() no longer raises an exception when None is + found in sys.modules. -- Issue #20474: Fix test_socket "unexpected success" failures on OS X 10.7+. +- Issue #14710: pkgutil.find_loader() no longer raises an exception when a + module doesn't exist. -- Issue #20605: Make test_socket getaddrinfo OS X segfault test more robust. +- Issue #21481: Argparse equality and inequality tests now return + NotImplemented when comparing to an unknown type. -Documentation -------------- +- Issue #8743: Fix interoperability between set objects and the + collections.Set() abstract base class. -- Issue #20488: Importlib is no longer *an* implementation of import, it's *the* - implementation. +- Issue #13355: random.triangular() no longer fails with a ZeroDivisionError + when low equals high. -Build ------ +- Issue #21538: The plistlib module now supports loading of binary plist files + when reference or offset size is not a power of two. -- Issue #20221: Removed conflicting (or circular) hypot definition when - compiled with VS 2010 or above. Initial patch by Tabrez Mohammed. +- Issue #21801: Validate that __signature__ is None or an instance of Signature. -- Issue #20609: Restored the ability to build 64-bit Windows binaries on - 32-bit Windows, which was broken by the change in issue #19788. +- Issue #21923: Prevent AttributeError in distutils.sysconfig.customize_compiler + due to possible uninitialized _config_vars. +- Issue #21323: Fix http.server to again handle scripts in CGI subdirectories, + broken by the fix for security issue #19435. Patch by Zach Byrne. -What's New in Python 3.3.4? -=========================== +Extension Modules +----------------- -*Release date: 09-Feb-2014* +- Issue #22176: Update the ctypes module's libffi to v3.1. This release + adds support for the Linux AArch64 and POWERPC ELF ABIv2 little endian + architectures. -Library -------- +Build +----- -- Issue #20374: Fix build warnings of the readline module with libedit on Mac. +- Issue #21958: Define HAVE_ROUND when building with Visual Studio 2013 and + above. Patch by Zachary Turner. +- Issue #15759: "make suspicious", "make linkcheck" and "make doctest" in Doc/ + now display special message when and only when there are failures. -What's New in Python 3.3.4 release candidate 1? -=============================================== +- Issue #17095: Fix Modules/Setup *shared* support. -*Release date: 26-Jan-2014* +- Issue #21811: Anticipated fixes to support OS X versions > 10.9. -Core and Builtins ------------------ +- Issue #21166: Prevent possible segfaults and other random failures of + python --generate-posix-vars in pybuilddir.txt build target. -- Issue #17825: Cursor "^" is correctly positioned for SyntaxError and - IndentationError. +IDLE +---- -- Issue #2382: SyntaxError cursor "^" is now written at correct position in most - cases when multibyte characters are in line (before "^"). This still not - works correctly with wide East Asian characters. +- Issue #17390: Adjust Editor window title; remove 'Python', + move version to end. -- Issue #18960: The first line of Python script could be executed twice when - the source encoding was specified on the second line. Now the source encoding - declaration on the second line isn't effective if the first line contains - anything except a comment. 'python -x' works now again with files with the - source encoding declarations, and can be used to make Python batch files - on Windows. +- Issue #14105: Idle debugger breakpoints no longer disappear + when inseting or deleting lines. -- Issue #17432: Drop UCS2 from names of Unicode functions in python3.def. +- Issue #17172: Turtledemo can now be run from Idle. + Currently, the entry is on the Help menu, but it may move to Run. + Patch by Ramchandra Apt and Lita Cho. -- Issue #19969: PyBytes_FromFormatV() now raises an OverflowError if "%c" - argument is not in range [0; 255]. +- Issue #21765: Add support for non-ascii identifiers to HyperParser. -- Issue #14432: Generator now clears the borrowed reference to the thread - state. Fix a crash when a generator is created in a C thread that is - destroyed while the generator is still used. The issue was that a generator - contains a frame, and the frame kept a reference to the Python state of the - destroyed C thread. The crash occurs when a trace function is setup. +- Issue #21940: Add unittest for WidgetRedirector. Initial patch by Saimadhav + Heblikar. -- Issue #19932: Fix typo in import.h, missing whitespaces in function prototypes. +- Issue #18592: Add unittest for SearchDialogBase. Patch by Phil Webster. -- Issue #19729: In str.format(), fix recursive expansion in format spec. +- Issue #21694: Add unittest for ParenMatch. Patch by Saimadhav Heblikar. -- Issue #19638: Fix possible crash / undefined behaviour from huge (more than 2 - billion characters) input strings in _Py_dg_strtod. +- Issue #21686: add unittest for HyperParser. Original patch by Saimadhav + Heblikar. -Library -------- +- Issue #12387: Add missing upper(lower)case versions of default Windows key + bindings for Idle so Caps Lock does not disable them. Patch by Roger Serwy. -- Issue #16042: CVE-2013-1752: smtplib: Limit amount of data read by - limiting the call to readline(). Original patch by Christian Heimes. +- Issue #21695: Closing a Find-in-files output window while the search is + still in progress no longer closes Idle. -- Issue #20317: ExitStack.__exit__ could create a self-referential loop if an - exception raised by a cleanup operation already had its context set - correctly (for example, by the @contextmanager decorator). The infinite - loop this caused is now avoided by checking if the expected context is - already set before trying to fix it. +- Issue #18910: Add unittest for textView. Patch by Phil Webster. -- Issue #20374: Fix build with GNU readline >= 6.3. +- Issue #18292: Add unittest for AutoExpand. Patch by Saihadhav Heblikar. -- Issue #20262: Warnings are raised now when duplicate names are added in the - ZIP file or too long ZIP file comment is truncated. +- Issue #18409: Add unittest for AutoComplete. Patch by Phil Webster. -- Issue #18574: Added missing newline in 100-Continue reply from - http.server.BaseHTTPRequestHandler. Patch by Nikolaus Rath. +Tests +----- -- Issue #20270: urllib.urlparse now supports empty ports. +- Issue #22166: with the assistance of a new internal _codecs._forget_codec + helping function, test_codecs now clears the encoding caches to avoid the + appearance of a reference leak -- Issue #20243: TarFile no longer raise ReadError when opened in write mode. +- Issue #22236: Tkinter tests now don't reuse default root window. New root + window is created for every test class. -- Issue #20238: TarFile opened with external fileobj and "w:gz" mode didn't - write complete output on close. +- Issue #20746: Fix test_pdb to run in refleak mode (-R). Patch by Xavier + de Gaye. -- Issue #20245: The open functions in the tarfile module now correctly handle - empty mode. +- Issue #22060: test_ctypes has been somewhat cleaned up and simplified; it + now uses unittest test discovery to find its tests. -- Issue #20242: Fixed basicConfig() format strings for the alternative - formatting styles. Thanks to kespindler for the bug report and patch. +- Issue #22104: regrtest.py no longer holds a reference to the suite of tests + loaded from test modules that don't define test_main(). -- Issue #20246: Fix buffer overflow in socket.recvfrom_into. +- Issue #22002: Added ``load_package_tests`` function to test.support and used + it to implement/augment test discovery in test_asyncio, test_email, + test_importlib, test_json, and test_tools. -- Issues #20206 and #5803: Fix edge case in email.quoprimime.encode where it - truncated lines ending in a character needing encoding but no newline by - using a more efficient algorithm that doesn't have the bug. +- Issue #21976: Fix test_ssl to accept LibreSSL version strings. Thanks + to William Orr. -- Issue #19082: Working xmlrpc.server and xmlrpc.client examples. Both in - modules and in documentation. Initial patch contributed by Vajrasky Kok. +- Issue #21918: Converted test_tools from a module to a package containing + separate test files for each tested script. -- Issue #20138: The wsgiref.application_uri() and wsgiref.request_uri() - functions now conform to PEP 3333 when handle non-ASCII URLs. +- Issue #20155: Changed HTTP method names in failing tests in test_httpservers + so that packet filtering software (specifically Windows Base Filtering Engine) + does not interfere with the transaction semantics expected by the tests. -- Issue #19097: Raise the correct Exception when cgi.FieldStorage is given an - invalid fileobj. +- Issue #19493: Refactored the ctypes test package to skip tests explicitly + rather than silently. -- Issue #20217: Fix build in SCHED_SPORADIC is defined. +- Issue #18492: All resources are now allowed when tests are not run by + regrtest.py. -- Issue #13107: argparse and optparse no longer raises an exception when output - a help on environment with too small COLUMNS. Based on patch by - Elazar Gershuni. +- Issue #21634: Fix pystone micro-benchmark: use floor division instead of true + division to benchmark integers instead of floating point numbers. Set pystone + version to 1.2. Patch written by Lennart Regebro. -- Issue #20207: Always disable SSLv2 except when PROTOCOL_SSLv2 is explicitly - asked for. +- Issue #21605: Added tests for Tkinter images. -- Issue #18960: The tokenize module now ignore the source encoding declaration - on the second line if the first line contains anything except a comment. +- Issue #21493: Added test for ntpath.expanduser(). Original patch by + Claudiu Popa. -- Issue #20078: Reading malformed zipfiles no longer hangs with 100% CPU - consumption. +- Issue #19925: Added tests for the spwd module. Original patch by Vajrasky Kok. -- Issue #20113: os.readv() and os.writev() now raise an OSError exception on - error instead of returning -1. +- Issue #21522: Added Tkinter tests for Listbox.itemconfigure(), + PanedWindow.paneconfigure(), and Menu.entryconfigure(). -- Issue #20072: Fixed multiple errors in tkinter with wantobjects is False. +Documentation +------------- -- Issue #20108: Avoid parameter name clash in inspect.getcallargs(). +- Issue #21777: The binary sequence methods on bytes and bytearray are now + documented explicitly, rather than assuming users will be able to derive + the expected behaviour from the behaviour of the corresponding str methods. -- Issue #12692: Backport the fix for ResourceWarning in test_urllib2net. This - also helps in closing the socket when Connection Close header is not sent. +Windows +------- -- Issue #19422: Explicitly disallow non-SOCK_STREAM sockets in the ssl - module, rather than silently let them emit clear text data. +- Issue #21671, #22160, CVE-2014-0224: The bundled version of OpenSSL has been + updated to 1.0.1i. -- Issue #18116: getpass was always getting an error when testing /dev/tty, - and thus was always falling back to stdin, and would then raise an exception - if stdin could not be used (such as /dev/null). It also leaked an open file. - All of these issues are now fixed. +- Issue #10747: Use versioned labels in the Windows start menu. + Patch by Olive Kilburn. -- Issue #20027: Fixed locale aliases for devanagari locales. +Tools/Demos +----------- -- Issue #20067: Tkinter variables now work when wantobjects is false. +- Issue #22201: Command-line interface of the zipfile module now correctly + extracts ZIP files with directory entries. Patch by Ryan Wilson. -- Issue #19020: Tkinter now uses splitlist() instead of split() in configure - methods. +- Issue #21906: Make Tools/scripts/md5sum.py work in Python 3. + Patch by Zachary Ware. -- Fix TypeError on "setup.py upload --show-response". +- Issue #21629: Fix Argument Clinic's "--converters" feature. -- Issue #12226: HTTPS is now used by default when connecting to PyPI. -- Issue #20045: Fix "setup.py register --list-classifiers". +What's New in Python 3.4.1? +=========================== -- Issue #18879: When a method is looked up on a temporary file, avoid closing - the file before the method is possibly called. +Release date: 2014-05-18 -- Issue #20034: Updated alias mapping to most recent locale.alias file - from X.org distribution using makelocalealias.py. +Core and Builtins +----------------- -- Issue #5815: Fixed support for locales with modifiers. Fixed support for - locale encodings with hyphens. +- Issue #21418: Fix a crash in the builtin function super() when called without + argument and without current frame (ex: embedded Python). -- Issue #20026: Fix the sqlite module to handle correctly invalid isolation - level (wrong type). +- Issue #21425: Fix flushing of standard streams in the interactive + interpreter. -- Issue #18829: csv.Dialect() now checks type for delimiter, escapechar and - quotechar fields. Original patch by Vajrasky Kok. +- Issue #21435: In rare cases, when running finalizers on objects in cyclic + trash a bad pointer dereference could occur due to a subtle flaw in + internal iteration logic. -- Issue #19855: uuid.getnode() on Unix now looks on the PATH for the - executables used to find the mac address, with /sbin and /usr/sbin as - fallbacks. +Library +------- -- Issue #20007: HTTPResponse.read(0) no more prematurely closes connection. - Original patch by Simon Sapin. +- Issue #10744: Fix PEP 3118 format strings on ctypes objects with a nontrivial + shape. -- Issue #19912: Fixed numerous bugs in ntpath.splitunc(). +- Issue #20998: Fixed re.fullmatch() of repeated single character pattern + with ignore case. Original patch by Matthew Barnett. -- Issue #19911: ntpath.splitdrive() now correctly processes the 'İ' character - (U+0130, LATIN CAPITAL LETTER I WITH DOT ABOVE). +- Issue #21075: fileinput.FileInput now reads bytes from standard stream if + binary mode is specified. Patch by Sam Kimbrel. -- Issue #19532: python -m compileall with no filename/directory arguments now - respects the -f and -q flags instead of ignoring them. +- Issue #21396: Fix TextIOWrapper(..., write_through=True) to not force a + flush() on the underlying binary stream. Patch by akira. -- Issue #19623: Fixed writing to unseekable files in the aifc module. +- Issue #21470: Do a better job seeding the random number generator by + using enough bytes to span the full state space of the Mersenne Twister. -- Issue #17919: select.poll.register() again works with poll.POLLNVAL on AIX. - Fixed integer overflow in the eventmask parameter. +- Issue #21398: Fix an unicode error in the pydoc pager when the documentation + contains characters not encodable to the stdout encoding. -- Issue #19063: if a Charset's body_encoding was set to None, the email - package would generate a message claiming the Content-Transfer-Encoding - was 7bit, and produce garbage output for the content. This now works. - A couple of other set_payload mishandlings of non-ASCII are also fixed. - -- Issue #17200: telnetlib's read_until and expect timeout was broken by the - fix to Issue #14635 in Python 3.3.0 to be interpreted as milliseconds - instead of seconds when the platform supports select.poll (ie: everywhere). - It is now treated as seconds once again. - -- Issue #17429: platform.linux_distribution() now decodes files from the UTF-8 - encoding with the surrogateescape error handler, instead of decoding from the - locale encoding in strict mode. It fixes the function on Fedora 19 which is - probably the first major distribution release with a non-ASCII name. Patch - written by Toshio Kuratomi. - -- Issue #19929: Call os.read with 32768 within subprocess.Popen.communicate - rather than 4096 for efficiency. A microbenchmark shows Linux and OS X - both using ~50% less cpu time this way. - -- Issue #19506: Use a memoryview to avoid a data copy when piping data - to stdin within subprocess.Popen.communicate. 5-10% less cpu usage. +Tests +----- -- Issue #19839: Fix regression in bz2 module's handling of non-bzip2 data at - EOF, and analogous bug in lzma module. +- Issue #17756: Fix test_code test when run from the installed location. -- Issue #19138: doctest's IGNORE_EXCEPTION_DETAIL now allows a match when - no exception detail exists (no colon following the exception's name, or - a colon does follow but no text follows the colon). +- Issue #17752: Fix distutils tests when run from the installed location. -- Issue #19834: Support unpickling of exceptions pickled by Python 2. +IDLE +---- -- Issue #15798: Fixed subprocess.Popen() to no longer fail if file - descriptor 0, 1 or 2 is closed. +- Issue #18104: Add idlelib/idle_test/htest.py with a few sample tests to begin + consolidating and improving human-validated tests of Idle. Change other files + as needed to work with htest. Running the module as __main__ runs all tests. -- Issue #19088: Fixed incorrect caching of the copyreg module in - object.__reduce__() and object.__reduce_ex__(). -- Fixed _pickle.Unpickler to not fail when loading empty strings as - persistent IDs. +What's New in Python 3.4.1rc1? +============================== -- Issue #11480: Fixed copy.copy to work with classes with custom metaclasses. - Patch by Daniel Urban. +Release date: 2014-05-05 -- Issue #6477: Added support for pickling the types of built-in singletons - (i.e., Ellipsis, NotImplemented, None). +Core and Builtins +----------------- -- Issue #11508: Fixed uuid.getnode() and uuid.uuid1() on environment with - virtual interface. Original patch by Kent Frazier. +- Issue #21274: Define PATH_MAX for GNU/Hurd in Python/pythonrun.c. -- Issue #11489: JSON decoder now accepts lone surrogates. +- Issue #21209: Fix sending tuples to custom generator objects with the yield + from syntax. -- Issue #19545: Avoid chained exceptions while passing stray % to - time.strptime(). Initial patch by Claudiu Popa. +- Issue #21134: Fix segfault when str is called on an uninitialized + UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. -- Issue #19633: Fixed writing not compressed 16- and 32-bit wave files on - big-endian platforms. +- Issue #19537: Fix PyUnicode_DATA() alignment under m68k. Patch by + Andreas Schwab. -- Issue #19449: in csv's writerow, handle non-string keys when generating the - error message that certain keys are not in the 'fieldnames' list. +- Issue #20929: Add a type cast to avoid shifting a negative number. -- Fix test.support.bind_port() to not cause an error when Python was compiled - on a system with SO_REUSEPORT defined in the headers but run on a system - with an OS kernel that does not support that reasonably new socket option. +- Issue #20731: Properly position in source code files even if they + are opened in text mode. Patch by Serhiy Storchaka. -- Fix compilation error under gcc of the ctypes module bundled libffi for arm. +- Issue #20637: Key-sharing now also works for instance dictionaries of + subclasses. Patch by Peter Ingebretson. -- Issue #19523: Closed FileHandler leak which occurred when delay was set. +- Issue #12546: Allow \x00 to be used as a fill character when using str, int, + float, and complex __format__ methods. -- Issue #13674: Prevented time.strftime from crashing on Windows when given - a year before 1900 and a format of %y. +- Issue #13598: Modify string.Formatter to support auto-numbering of + replacement fields. It now matches the behavior of str.format() in + this regard. Patches by Phil Elson and Ramchandra Apte. -- Issue #19544 and Issue #6286: Restore use of urllib over http allowing use - of http_proxy for Distutils upload command, a feature accidentally lost - in the rollback of distutils2. +Library +------- -- Issue #19544 and Issue #7457: Restore the read_pkg_file method to - distutils.dist.DistributionMetadata accidentally removed in the undo of - distutils2. +- Issue #21088: Bugfix for curses.window.addch() regression in 3.4.0. + In porting to Argument Clinic, the first two arguments were reversed. -- Issue #1575020: Fixed support of 24-bit wave files on big-endian platforms. +- Issue #21469: Reduced the risk of false positives in robotparser by + checking to make sure that robots.txt has been read or does not exist + prior to returning True in can_fetch(). -- Issue #19480: HTMLParser now accepts all valid start-tag names as defined - by the HTML5 standard. +- Issue #21321: itertools.islice() now releases the reference to the source + iterator when the slice is exhausted. Patch by Anton Afanasyev. -- Issue #6157: Fixed tkinter.Text.debug(). Original patch by Guilherme Polo. +- Issue #9815: assertRaises now tries to clear references to local variables + in the exception's traceback. -- Issue #6160: The bbox() method of tkinter.Spinbox now returns a tuple of - integers instead of a string. Based on patch by Guilherme Polo. +- Issue #13204: Calling sys.flags.__new__ would crash the interpreter, + now it raises a TypeError. -- Issue #10197: Rework subprocess.get[status]output to use subprocess - functionality and thus to work on Windows. Patch by Nick Coghlan. +- Issue #19385: Make operations on a closed dbm.dumb database always raise the + same exception. -- Issue #19286: Directories in ``package_data`` are no longer added to - the filelist, preventing failure outlined in the ticket. +- Issue #21207: Detect when the os.urandom cached fd has been closed or + replaced, and open it anew. -IDLE ----- +- Issue #21291: subprocess's Popen.wait() is now thread safe so that + multiple threads may be calling wait() or poll() on a Popen instance + at the same time without losing the Popen.returncode value. -- Issue #17390: Add Python version to Idle editor window title bar. - Original patches by Edmond Burnett and Kent Johnson. +- Issue #21127: Path objects can now be instantiated from str subclass + instances (such as ``numpy.str_``). -- Issue #18960: IDLE now ignores the source encoding declaration on the second - line if the first line contains anything except a comment. +- Issue #15002: urllib.response object to use _TemporaryFileWrapper (and + _TemporaryFileCloser) facility. Provides a better way to handle file + descriptor close. Patch contributed by Christian Theune. -- Issue #20058: sys.stdin.readline() in IDLE now always returns only one line. +- Issue #12220: mindom now raises a custom ValueError indicating it doesn't + support spaces in URIs instead of letting a 'split' ValueError bubble up. -- Issue #19481: print() of string subclass instance in IDLE no longer hangs. +- Issue #21239: patch.stopall() didn't work deterministically when the same + name was patched more than once. -- Issue #18270: Prevent possible IDLE AttributeError on OS X when no initial - shell window is present. +- Issue #21222: Passing name keyword argument to mock.create_autospec now + works. -Tests ------ +- Issue #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. -- Issue #19804: The test_find_mac test in test_uuid is now skipped if the - ifconfig executable is not available. +- Issue #17498: Some SMTP servers disconnect after certain errors, violating + strict RFC conformance. Instead of losing the error code when we issue the + subsequent RSET, smtplib now returns the error code and defers raising the + SMTPServerDisconnected error until the next command is issued. -- Issue #19886: Use better estimated memory requirements for bigmem tests. +- Issue #17826: setting an iterable side_effect on a mock function created by + create_autospec now works. Patch by Kushal Das. -- Issue #20055: Fix test_shutil under Windows with symlink privileges held. - Patch by Vajrasky Kok. +- Issue #7776: Fix ``Host:`` header and reconnection when using + http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. -- Issue #19938: Re-enabled test_bug_1333982 in test_dis, which had been - disabled since 3.0 due to the changes in listcomp handling. +- Issue #20968: unittest.mock.MagicMock now supports division. + Patch by Johannes Baiter. -- Issue #19320: test_tcl no longer fails when wantobjects is false. +- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second + parameter. Bug reported by Guido Vranken. -- Issue #19683: Removed empty tests from test_minidom. Patch by Ajitesh Gupta. +- Issue #21169: getpass now handles non-ascii characters that the + input stream encoding cannot encode by re-encoding using the + replace error handler. -- Issue #19919: Fix flaky SSL test. connect_ex() sometimes returns - EWOULDBLOCK on Windows or VMs hosted on Windows. +- Issue #21171: Fixed undocumented filter API of the rot13 codec. + Patch by Berker Peksag. -- Issue #19912: Added tests for ntpath.splitunc(). +- Issue #21172: isinstance check relaxed from dict to collections.Mapping. -- Issue #19828: Fixed test_site when the whole suite is run with -S. +- Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError + if path and sock are specified at the same time. -- Issue #19928: Implemented a test for repr() of cell objects. +- Issue #21149: Improved thread-safety in logging cleanup during interpreter + shutdown. Thanks to Devin Jeanpierre for the patch. -- Issue #19535: Fixed test_docxmlrpc when python is run with -OO. +- Issue #20145: `assertRaisesRegex` and `assertWarnsRegex` now raise a + TypeError if the second argument is not a string or compiled regex. -- Issue #19926: Removed unneeded test_main from test_abstract_numbers. - Patch by Vajrasky Kok. +- Issue #21058: Fix a leak of file descriptor in + :func:`tempfile.NamedTemporaryFile`, close the file descriptor if + :func:`io.open` fails -- Issue #19595, #19987: Re-enabled a long-disabled test in test_winsound. +- Issue #21200: Return None from pkgutil.get_loader() when __spec__ is missing. -- Issue #19588: Fixed tests in test_random that were silently skipped most - of the time. Patch by Julian Gindi. +- Issue #21013: Enhance ssl.create_default_context() when used for server side + sockets to provide better security by default. -- Issue #19596: Set untestable tests in test_importlib to None to avoid - reporting success on empty tests. +- Issue #20633: Replace relative import by absolute import. -- Issue #19440: Clean up test_capi by removing an unnecessary __future__ - import, converting from test_main to unittest.main, and running the - _testcapi module tests within a unittest TestCase. +- Issue #20980: Stop wrapping exception when using ThreadPool. -- Issue #18702, 19572: All skipped tests now reported as skipped. +- Issue #21082: In os.makedirs, do not set the process-wide umask. Note this + changes behavior of makedirs when exist_ok=True. -- Issue #19085: Added basic tests for all tkinter widget options. +- Issue #20990: Fix issues found by pyflakes for multiprocessing. -Documentation -------------- +- Issue #21015: SSL contexts will now automatically select an elliptic + curve for ECDH key exchange on OpenSSL 1.0.2 and later, and otherwise + default to "prime256v1". -- Issue #20265: Updated some parts of the Using Windows document. +- Issue #20995: Enhance default ciphers used by the ssl module to enable + better security an prioritize perfect forward secrecy. -- Issue #20266: Updated some parts of the Windows FAQ. +- Issue #20884: Don't assume that __file__ is defined on importlib.__init__. -- Issue #20255: Updated the about and bugs pages. +- Issue #21499: Ignore __builtins__ in several test_importlib.test_api tests. -- Issue #20253: Fixed a typo in the ipaddress docs that advertised an - illegal attribute name. Found by INADA Naoki. +- Issue #20879: Delay the initialization of encoding and decoding tables for + base32, ascii85 and base85 codecs in the base64 module, and delay the + initialization of the unquote_to_bytes() table of the urllib.parse module, to + not waste memory if these modules are not used. -- Issue #19963: Document that importlib.import_module() no longer requires - importing parent packages separately. +- Issue #19157: Include the broadcast address in the usuable hosts for IPv6 + in ipaddress. -- Issue #18840: Introduce the json module in the tutorial, and de-emphasize - the pickle module. +- Issue #11599: When an external command (e.g. compiler) fails, distutils now + prints out the whole command line (instead of just the command name) if the + environment variable DISTUTILS_DEBUG is set. -- Issue #19845: Updated the Compiling Python on Windows section. +- Issue #4931: distutils should not produce unhelpful "error: None" messages + anymore. distutils.util.grok_environment_error is kept but doc-deprecated. -- Issue #19795: Improved markup of True/False constants. +- Issue #20875: Prevent possible gzip "'read' is not defined" NameError. + Patch by Claudiu Popa. -- Issue #18326: Clarify that list.sort's arguments are keyword-only. Also, - attempt to reduce confusion in the glossary by not saying there are - different "types" of arguments and parameters. +- Issue #11558: ``email.message.Message.attach`` now returns a more + useful error message if ``attach`` is called on a message for which + ``is_multipart`` is False. -Build ------ +- Issue #20283: RE pattern methods now accept the string keyword parameters + as documented. The pattern and source keyword parameters are left as + deprecated aliases. -- Issue #19788: kill_python(_d).exe is now run as a PreBuildEvent on the - pythoncore sub-project. This should prevent build errors due a previous - build's python(_d).exe still running. +- Issue #20778: Fix modulefinder to work with bytecode-only modules. -- Add workaround for VS 2010 nmake clean issue. VS 2010 doesn't set up PATH - for nmake.exe correctly. +- Issue #20791: copy.copy() now doesn't make a copy when the input is + a bytes object. Initial patch by Peter Otten. -Tools/Demos ------------ +- Issue #19748: On AIX, time.mktime() now raises an OverflowError for year + outsize range [1902; 2037]. -- Issue #19936: Added executable bits or shebang lines to Python scripts which - requires them. Disable executable bits and shebang lines in test and - benchmark files in order to prevent using a random system python, and in - source files of modules which don't provide command line interface. Fixed - shebang line to use python3 executable in the unittestgui script. +- Issue #20816: Fix inspect.getcallargs() to raise correct TypeError for + missing keyword-only arguments. Patch by Jeremiah Lowin. -- Issue #18960: 2to3 and the findnocoding.py script now ignore the source - encoding declaration on the second line if the first line contains anything - except a comment. +- Issue #20817: Fix inspect.getcallargs() to fail correctly if more + than 3 arguments are missing. Patch by Jeremiah Lowin. +- Issue #6676: Ensure a meaningful exception is raised when attempting + to parse more than one XML document per pyexpat xmlparser instance. + (Original patches by Hirokazu Yamamoto and Amaury Forgeot d'Arc, with + suggested wording by David Gutteridge) -What's New in Python 3.3.3? -=========================== +- Issue #21117: Fix inspect.signature to better support functools.partial. + Due to the specifics of functools.partial implementation, + positional-or-keyword arguments passed as keyword arguments become + keyword-only. -*Release date: 17-Nov-2013* +- Issue #21209: Fix asyncio.tasks.CoroWrapper to workaround a bug + in yield-from implementation in CPythons prior to 3.4.1. -No changes from release candidate 2. +- asyncio: Add gi_{frame,running,code} properties to CoroWrapper + (upstream issue #163). +- Issue #21311: Avoid exception in _osx_support with non-standard compiler + configurations. Patch by John Szakmeister. -What's New in Python 3.3.3 release candidate 2? -=============================================== +- Issue #11571: Ensure that the turtle window becomes the topmost window + when launched on OS X. -*Release date: 11-Nov-2013* +Extension Modules +----------------- -Library -------- +- Issue #21276: posixmodule: Don't define USE_XATTRS on KFreeBSD and the Hurd. +- Issue #21226: Set up modules properly in PyImport_ExecCodeModuleObject + (and friends). -- Issue #19227: Any re-seeding of the OpenSSL RNG on fork has been removed; - this should be handled by OpenSSL itself or by the application. +IDLE +---- -- Issue #19435: Fix directory traversal attack on CGIHttpRequestHandler. +- Issue #21139: Change default paragraph width to 72, the PEP 8 recommendation. -Tests ------ +- Issue #21284: Paragraph reformat test passes after user changes reformat width. -- Issue #18964: Fix test_tcl when run with Tcl/Tk versions < 8.5. +- Issue #17654: Ensure IDLE menus are customized properly on OS X for + non-framework builds and for all variants of Tk. Build -----