The `subprocess.getstatusoutput` API was inadvertently changed
in Python 3.3.4. Document the change, it is too late to undo the
API change now as it has shipped in many stable releases.
(cherry picked from commit 738b7d9766e1a794aaaabfba0d515a467ba833ca)
bpo-30465: Fix lineno and col_offset in fstring AST nodes (GH-1800) (gh-3409)
For f-string ast nodes, fix the line and columns so that tools such as flake8 can identify them correctly.
(cherry picked from commit e7c566caf177afe43b57f0b2723e723d880368e8)
[3.6] bpo-30737: Update DevGuide links to new URL (GH-3228) (GH-3390)
Update old devguide links from https://docs.python.org/devguide to https://devguide.python.org
(cherry picked from commit 384899dfaeb83db38a6d3846d3cbc2f58a6605cd)
[3.6] bpo-31178: Avoid concatenating bytes with str in subprocess error (GH-3066) (#3388)
Avoid concatenating bytes with str in the typically rare subprocess error path (exec failed). Includes a mock based unittest to exercise the codepath.
(cherry picked from commit 3fc499bca18454b9f432b9b0106cab662bfeb549)
[3.6] Stop test_xmlrpc from writing to sys.stderr (GH-3359) (#3380)
One test case of test_xmlrpc uses HTTPServer with a subclass of
BaseHTTPRequestHandler. The BaseRequestHandler class logs to
sys.stderr by default. Override log_message() to not clobber
test output.
[3.6] bpo-31320: No traceback to sys.stderr in test_ssl (GH-3360) (GH-3369)
In case PROTOCOL_TLS_SERVER is used for both client context and server
context, the test thread dies with OSError. Catch OSError to avoid
traceback on sys.stderr
[3.6] bpo-29334: Fix ssl.getpeercert for auto-handshake (GH-1769) (#1778)
Drop handshake_done and peer_cert members from PySSLSocket struct. The
peer certificate can be acquired from *SSL directly.
SSL_get_peer_certificate() does not trigger any network activity.
Instead of manually tracking the handshake state, simply use
SSL_is_init_finished().
In combination these changes fix auto-handshake for non-blocking
MemoryBIO connections.
The ssl and hashlib modules now call OPENSSL_add_all_algorithms_noconf() on
OpenSSL < 1.1.0. The function detects CPU features and enables optimizations
on some CPU architectures such as POWER8. Patch is based on research from
Gustavo Serra Scalet.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit c941e62)
Version breakdown, support disabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will not be defined ->
False/False
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will be defined -> True/False
Version breakdown support enabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True
* Refine NPN guard:
- If NPN is disabled, but ALPN is available we need our callback
- Make clinic's ssl behave the same way
This created a working ssl module for me, with NPN disabled and ALPN
enabled for OpenSSL 1.1.0f.
Concerns to address:
The initial commit for NPN support into OpenSSL [1], had the
OPENSSL_NPN_* variables defined inside the OPENSSL_NO_NEXTPROTONEG
guard. The question is if that ever made it into a release.
This would need an ugly hack, something like:
bpo-22536 [3.6] Set filename in FileNotFoundError (#3305)
* [3.6] bpo-22536: Set the filename in FileNotFoundError. (GH-3194)
Have the subprocess module set the filename in the FileNotFoundError
exception raised on POSIX systems when the executable or cwd are missing.
(cherry picked from commit 8621bb5d93239316f97281826461b85072ff6db7)
bpo-29212: Fix the ugly ThreadPoolExecutor thread name.
Fixes the newly introduced ugly default thread name for concurrent.futures
thread.ThreadPoolExecutor threads. They'll now resemble the old <=3.5
threading default Thread-x names by being named ThreadPoolExecutor-y_n..
(cherry picked from commit a3d91b43c2851312fb942f31afa12f5961706db6)
[3.6] bpo-9146: Raise a ValueError if OpenSSL fails to init a hash func (#3274)
* [3.6] bpo-9146: Raise a ValueError if OpenSSL fails to init a hash func. (GH-1777)
This helps people in weird FIPS mode environments where common things
like MD5 are not available in the binary as a matter of policy.
(cherry picked from commit 07244a83014fad42da937c17d98474b47a570bf7)
Victor Stinner [Fri, 1 Sep 2017 13:04:47 +0000 (15:04 +0200)]
bpo-31217: Fix regrtest -R for small integer (#3260) (#3261)
Use a pool of integer objects toprevent false alarm when checking for
memory block leaks. Fill the pool with values in -1000..1000 which
are the most common (reference, memory block, file descriptor)
differences.
Pauli Virtanen [Wed, 30 Aug 2017 09:40:05 +0000 (11:40 +0200)]
[3.6] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31) (#3241)
Ctypes currently produces wrong pep3118 type codes for several types.
E.g. memoryview(ctypes.c_long()).format gives "<l" on 64-bit platforms,
but it should be "<q" instead for sizeof(c_long) == 8
The problem is that the '<>' endian specification in the struct syntax
also turns on the "standard size" mode, which makes type characters have
a platform-independent meaning, which does not match with the codes used
internally in ctypes. The struct module format syntax also does not
allow specifying native-size non-native-endian items.
This commit adds a converter function that maps the internal ctypes
codes to appropriate struct module standard-size codes in the pep3118
format strings. The tests are modified to check for this.
(cherry picked from commit 07f1658aa09f6798793c473c72b2951b7fefe220)
- Link to the regular expressions object documentation
- Clarify that it can be used with more than the two methods currently stated.
(cherry picked from commit ed94a8b2851914bcda3a77b28b25517b8baa91e6)
Łukasz Langa [Tue, 22 Aug 2017 20:20:40 +0000 (13:20 -0700)]
[3.6] bpo-30983: [gdb] Fix py-bt, etc. for non-debug shared builds (GH-3153) (#3192)
PEP 523 introduced _PyEval_EvalFrameDefault which inlines PyEval_EvalFrameEx on
non-debug shared builds. This breaks the ability to use py-bt, py-up, and
a few other Python-specific gdb integrations.
This patch fixes the problem by only looking for _PyEval_EvalFrameDefault
frames.
test_gdb passes on both a debug and a non-debug build.
Victor Stinner [Mon, 21 Aug 2017 22:17:15 +0000 (00:17 +0200)]
[3.6] bpo-30871: Add test.pythoninfo (#3174)
* bpo-30871: Add test.pythoninfo (#3075)
* Add Lib/test/pythoninfo.py: script collecting various informations
about Python to help debugging test failures.
* regrtest: remove sys.hash_info and sys.flags from header.
* Travis CI, Appveyor: run pythoninfo before tests
(cherry picked from commit b907abc88589f7bea52c5afe172ececc6edcda70)
* bpo-30871: pythoninfo: add expat and _decimal (#3121)
bpo-30121: Test the Popen failure when Popen was created with pipes.
Create also NONEXISTING_CMD variable in test_subprocess.py.
(cherry picked from commit 9a83f651f31b47b3f6c8b210f7807b26e8c373a5)
xmlrpc.server now explicitly breaks reference cycles when using
sys.exc_info() in code handling exceptions.
(cherry picked from commit 84524454d0ba77d299741c47bd0f5841ac3f66ce)
Cheryl Sabella [Sun, 20 Aug 2017 23:21:51 +0000 (19:21 -0400)]
[3.6] bpo-31206: IDLE: Factor HighPage class from ConfigDialog (GH-3160) (#3162)
Part 3 of 3. Remove old highlight functions and load_config as this functionality is now contained within classes. Patch by Cheryl Sabella.
(cherry picked from commit 4bfebc63012f0f4e00f6a98c3d96e1c0ebe93408)
Cheryl Sabella [Sun, 20 Aug 2017 12:03:05 +0000 (08:03 -0400)]
[3.6] bpo-31206: IDLE: Factor HighPage class from ConfigDialog (GH-3156) (#3159)
Patch 2 of 3, to avoid horrendous diff. Create highlights page from new HighPage class instead of old ConfigDialog methods and change tests to match. Patch by Cheryl Sabella.
(cherry picked from commit 8f7a798edbdbca9a400105e3225463e59b334666)
Cheryl Sabella [Sat, 19 Aug 2017 13:22:54 +0000 (09:22 -0400)]
[3.6] bpo-31206: IDLE: Factor HighPage class from ConfigDialog (GH-3141) (#3154)
This is the first half of a patch similar to the one for for bpo-31205. It is being split into 2 PRs to avoid what happened with PR-3096 -- an incomprehensible diff that could not be cleanly backported to 3.6. This half copies several methods of ConfigDialog and turns them into a new class. Patch by Cheryl Sabella.
(cherry picked from commit a32e40561a24de373d1c5a437a8aa329758ba8e4)
bpo-30721 added a "Did you mean ...?" suggestion to rshift
TypeError messages that triggers when the LHS is a Python
C function called "print".
Since this custom error message is expected to be triggered
primarily by attempts to use Python 2 print redirection syntax
in Python 3, and is incredibly unlikely to be encountered
otherwise, it is also being backported to the next 3.6
maintenance release.
test_bare_raise_in_brand_new_thread() now explicitly breaks a
reference cycle to not leak a dangling thread.
(cherry picked from commit 3d284c081fc3042036adfe1bf2ce92c34d743b0b)
Mariatta [Fri, 18 Aug 2017 13:21:33 +0000 (06:21 -0700)]
Fix broken `Show Source` links on documentation pages (GH-3113) (GH-3125)
The `Show Source` was broken because of a change made in sphinx 1.5.1
In Sphinx 1.4.9, the sourcename was "index.txt".
In Sphinx 1.5.1+, it is now "index.rst.txt".
(cherry picked from commit b9ff498793611d1c6a9b99df464812931a1e2d69)
Julien Palard [Thu, 17 Aug 2017 14:37:00 +0000 (16:37 +0200)]
Update the language selection in the docs language switch. (GH-3114) (#3115)
Change the option for `Français` into `French` to be consistent with the other language selections that are already in English.
(cherry picked from commit b616b972999cdd5fe54ef8a43c131a27ca538dee)
Victor Stinner [Wed, 16 Aug 2017 11:14:38 +0000 (13:14 +0200)]
bpo-31069, test_multiprocessing: Fix dangling process (#3103) (#3104)
Fix a warning about dangling processes in test_rapid_restart() of
_test_multiprocessing: join the process.
(cherry picked from commit 17657bb9458ff8f8804b7637d61686a68f4b9471)
Christian Heimes [Tue, 15 Aug 2017 08:45:40 +0000 (10:45 +0200)]
[3.6] bpo-30714: ALPN changes for OpenSSL 1.1.0f (#3093)
OpenSSL 1.1.0 to 1.1.0e aborted the handshake when server and client
could not agree on a protocol using ALPN. OpenSSL 1.1.0f changed that.
The most recent version now behaves like OpenSSL 1.0.2 again. The ALPN
callback can pretend to not been set.
See https://github.com/openssl/openssl/pull/3158 for more details
Łukasz Langa [Mon, 14 Aug 2017 23:06:28 +0000 (16:06 -0700)]
[3.6] bpo-30983: eval frame rename in pep 0523 broke gdb's python extension (GH-2803) (#3090)
pep 0523 renames PyEval_EvalFrameEx to _PyEval_EvalFrameDefault while the gdb python extension only looks for PyEval_EvalFrameEx to understand if it is dealing with a frame.
Final effect is that attaching gdb to a python3.6 process doesnt resolve python objects. Eg. py-list and py-bt dont work properly.
This patch fixes that. Tested locally on python3.6
(cherry picked from commit 2e0f4db114)
Victor Stinner [Fri, 11 Aug 2017 00:36:30 +0000 (02:36 +0200)]
bpo-31173: Rewrite WSTOPSIG test of test_subprocess (#3055) (#3070)
The current test_child_terminated_in_stopped_state() function test
creates a child process which calls ptrace(PTRACE_TRACEME, 0, 0) and
then crash (SIGSEGV). The problem is that calling os.waitpid() in the
parent process is not enough to close the process: the child process
remains alive and so the unit test leaks a child process in a
strange state. Closing the child process requires non-trivial code,
maybe platform specific.
Remove the functional test and replaces it with an unit test which
mocks os.waitpid() using a new _testcapi.W_STOPCODE() function to
test the WIFSTOPPED() path.
(cherry picked from commit 7b7c6dcfff6a35333988a3c74c895ed19dff2e09)
Julien Palard [Thu, 10 Aug 2017 14:54:26 +0000 (16:54 +0200)]
bpo-31159: fix language switch regex on unknown yet built languages. … (#3051)
* bpo-31159: fix language switch regex on unknown yet built languages. (#3039)
This fix a regex issue (a missing non-matching group around an 'or'
list) and the specific possible case where a translation is built but
not yet in known by the picker, but not explicitly listing possible
languages in the regex.
(cherry picked from commit 122081deef86174beee965be1207fa46ea23533d)
* bpo-31149: Doc: Add Japanese to the language switcher. (#3028)
Victor Stinner [Thu, 10 Aug 2017 14:02:00 +0000 (16:02 +0200)]
[3.6] bpo-31160: Backport reap_children() fixes from master to 3.6 (#3060)
* bpo-31160: Fix test_builtin for zombie process (#3043)
PtyTests.run_child() now calls os.waitpid() to read the exit status
of the child process to avoid creating zombie process and leaking
processes in the background.
(cherry picked from commit 4baca1b0f7325032598cd38e7ceffc79b616d255)
* bpo-31160: regrtest now reaps child processes (#3044)
TestRandomNameSequence.test_process_awareness() now calls
os.waitpid() to avoid leaking a zombie process.
(cherry picked from commit 6c8c2943d996b59a48d331f61f22cbe72933910e)
Terry Jan Reedy [Thu, 10 Aug 2017 04:46:29 +0000 (00:46 -0400)]
[3.6] bpo-19903: IDLE: Calltips changed to use inspect.signature (GH-2822) (#3053)
Idlelib.calltips.get_argspec now uses inspect.signature instead of inspect.getfullargspec, like help() does. This improves the signature in the call tip in a few different cases, including builtins converted to provide a signature. A message is added if the object is not callable, has an invalid signature, or if it has positional-only parameters.
Patch by Louie Lu..
(cherry picked from commit 3b0f620c1a2a21272a9e2aeca6ca1d1ac10f8162)
Victor Stinner [Wed, 9 Aug 2017 08:48:11 +0000 (10:48 +0200)]
ttk: fix LabeledScale and OptionMenu destroy() method (#3025) (#3030)
bpo-31135: Call the parent destroy() method even if the used
attribute doesn't exist.
The LabeledScale.destroy() method now also explicitly clears label
and scale attributes to help the garbage collector to destroy all
widgets.
(cherry picked from commit cd7e9c1b67d3d07ee03e0a79af2791c19031cecb)