[3.6] bpo-31323: Fix reference leak in test_ssl (GH-3263) (#3538)
Store exceptions as string rather than object to prevent reference
cycles which cause leaking dangling threads.
(cherry picked from commit 868710158910fa38e285ce0e6d50026e1d0b2a8c)
ProcessPoolShutdownTest.test_del_shutdown() now closes the call queue
and joins its thread, to prevent leaking a dangling thread.
(cherry picked from commit 3bcf157c115ba3e48bce62ac8cb13c703475a113)
* test_thread.test_forkinthread() now waits until the thread completes.
* Check the status in the test method, not in the thread function
* Don't ignore RuntimeError anymore: since the commit 346cbd351ee0dd3ab9cb9f0e4cb625556707877e (bpo-16500,
os.register_at_fork(), os.fork() cannot fail anymore with
RuntimeError.
* Replace 0.01 literal with a new POLL_SLEEP constant
* test_forkinthread(): test if os.fork() exists rather than testing
the platform.
(cherry picked from commit a15d155aadfad232158f530278505cdc6f326f93)
* Explicitly call shutdown(wait=True) on executors to wait until all
threads complete to prevent side effects between tests.
* Fix test_loop_self_reading_exception(): don't mock loop.close().
Previously, the original close() method was called rather than the
mock, because how set_event_loop() registered loop.close().
[3.6] bpo-31421: Document how IDLE runs tkinter programs. (GH-3513) (#3514)
IDLE calls tcl/tk update in the background in order to make live
interaction and experimentatin with tkinter applications much easier.
(cherry picked from commit 98758bc67fb39b74bab368bef8ff3b34554c77c8)
Adding to an int entry is not the same as deleting and inserting
because int('') will fail.
(cherry picked from commit 667522efa8fedfb57fd89d7335d7dfd270f274d5)
[3.6] bpo-30781: IDLE: Fix help button on configdialog (GH-3238) (#3489)
This fixes an omission in the ttk conversion patch for this issue, hence no new news.
Patch by Cheryl Sabella.
(cherry picked from commit 3866d9bbcf808cea98b3d00007f9f246b83858ce)
Terry Jan Reedy [Mon, 11 Sep 2017 00:30:46 +0000 (20:30 -0400)]
[3.6] bpo-27099: IDLE - Convert built-in extensions to regular features (GH-2494) (#3487)
About 10 IDLE features were implemented as supposedly optional
extensions. Their different behavior could be confusing or worse for
users and not good for maintenance. Hence the conversion.
The main difference for users is that user configurable key bindings
for builtin features are now handled uniformly. Now, editing a binding
in a keyset only affects its value in the keyset. All bindings are
defined together in the system-specific default keysets in config-
extensions.def. All custom keysets are saved as a whole in config-
extension.cfg. All take effect as soon as one clicks Apply or Ok.
The affected events are '<<force-open-completions>>', '<<expand-word>>',
'<<force-open-calltip>>', '<<flash-paren>>', '<<format-paragraph>>',
'<<run-module>>', '<<check-module>>', and '<<zoom-height>>'. Any
(global) customizations made before 3.6.3 will not affect their keyset-
specific customization after 3.6.3. and vice versa.
Inital patch by Charles Wohlganger, revised by Terry Jan Reedy.
(cherry picked from commit 58fc71c)
R. David Murray [Sun, 10 Sep 2017 05:54:05 +0000 (01:54 -0400)]
[3.6] Clarify nature of parse_args 'args' argument. (GH-3292) (GH-3325)
Patch by Paul.j3. Includes an unrelated but useful addition to the
optparse porting section.
(cherry picked from commit 0c7983e4adf9604d0ac93757a45d14be06c27696)
[3.6] bpo-26669: Fix nan arg value error in pytime.c (GH-3085) (GH-3467)
* Modify NaN check function and error message
* Fix pytime.c when arg is nan
* fix whitespace
(cherry picked from commit 829dacce4fca60fc3c3367980e75e21dfcdbe6be)
TLS 1.3 introduces a new, distinct set of cipher suites. The TLS 1.3
cipher suites don't overlap with cipher suites from TLS 1.2 and earlier.
Since Python sets its own set of permitted ciphers, TLS 1.3 handshake
will fail as soon as OpenSSL 1.1.1 is released. Let's enable the common
AES-GCM and ChaCha20 suites.
Additionally the flag OP_NO_TLSv1_3 is added. It defaults to 0 (no op) with
OpenSSL prior to 1.1.1. This allows applications to opt-out from TLS 1.3
now.
[3.6] bpo-31036: Allow sphinx and blurb to be found automatically (GH-3440) (#3441)
Rather than requiring the path to blurb and/or sphinx-build to be specified to the make rule, enhance the Doc/Makefile to look for each first in a virtual environment created by make venv and, if not found, look on the normal process PATH. This allows the Doc/Makefile to take advantage of an installed spinx-build or blurb and, thus, do the right thing most of the time. Also, make the directory for the venv be configurable and document the `make venv` target.
(cherry picked from commit 590665c399fc4aa3c4a9f8e7104d43a02e9f3a0c)
[3.6] bpo-27340: Use memoryview in SSLSocket.sendall() (GH-3384) (#3434)
* bpo-27340: Use memoryview in SSLSocket.sendall()
SSLSocket.sendall() now uses memoryview to create slices of data. This fix
support for all bytes-like object. It is also more efficient and avoids
costly copies.
Signed-off-by: Christian Heimes <christian@python.org>
* Cast view to bytes, fix typo
[3.6] bpo-22635: Update the getstatusoutput docstring. (GH-3435) (#3439)
To match the documentation updates already made.
Also renames the local variable used within to match
what it actually holds.
(cherry picked from commit 2eb0cb4787d02d995a9bb6dc075983792c12835c)
The `subprocess.getstatusoutput` API was inadvertently changed
in Python 3.3.4. Document the change, it is too late to undo the
API change now as it has shipped in many stable releases.
(cherry picked from commit 738b7d9766e1a794aaaabfba0d515a467ba833ca)
bpo-30465: Fix lineno and col_offset in fstring AST nodes (GH-1800) (gh-3409)
For f-string ast nodes, fix the line and columns so that tools such as flake8 can identify them correctly.
(cherry picked from commit e7c566caf177afe43b57f0b2723e723d880368e8)
[3.6] bpo-30737: Update DevGuide links to new URL (GH-3228) (GH-3390)
Update old devguide links from https://docs.python.org/devguide to https://devguide.python.org
(cherry picked from commit 384899dfaeb83db38a6d3846d3cbc2f58a6605cd)
[3.6] bpo-31178: Avoid concatenating bytes with str in subprocess error (GH-3066) (#3388)
Avoid concatenating bytes with str in the typically rare subprocess error path (exec failed). Includes a mock based unittest to exercise the codepath.
(cherry picked from commit 3fc499bca18454b9f432b9b0106cab662bfeb549)
[3.6] Stop test_xmlrpc from writing to sys.stderr (GH-3359) (#3380)
One test case of test_xmlrpc uses HTTPServer with a subclass of
BaseHTTPRequestHandler. The BaseRequestHandler class logs to
sys.stderr by default. Override log_message() to not clobber
test output.
[3.6] bpo-31320: No traceback to sys.stderr in test_ssl (GH-3360) (GH-3369)
In case PROTOCOL_TLS_SERVER is used for both client context and server
context, the test thread dies with OSError. Catch OSError to avoid
traceback on sys.stderr
[3.6] bpo-29334: Fix ssl.getpeercert for auto-handshake (GH-1769) (#1778)
Drop handshake_done and peer_cert members from PySSLSocket struct. The
peer certificate can be acquired from *SSL directly.
SSL_get_peer_certificate() does not trigger any network activity.
Instead of manually tracking the handshake state, simply use
SSL_is_init_finished().
In combination these changes fix auto-handshake for non-blocking
MemoryBIO connections.
The ssl and hashlib modules now call OPENSSL_add_all_algorithms_noconf() on
OpenSSL < 1.1.0. The function detects CPU features and enables optimizations
on some CPU architectures such as POWER8. Patch is based on research from
Gustavo Serra Scalet.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit c941e62)
Version breakdown, support disabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will not be defined ->
False/False
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will be defined -> True/False
Version breakdown support enabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True
* Refine NPN guard:
- If NPN is disabled, but ALPN is available we need our callback
- Make clinic's ssl behave the same way
This created a working ssl module for me, with NPN disabled and ALPN
enabled for OpenSSL 1.1.0f.
Concerns to address:
The initial commit for NPN support into OpenSSL [1], had the
OPENSSL_NPN_* variables defined inside the OPENSSL_NO_NEXTPROTONEG
guard. The question is if that ever made it into a release.
This would need an ugly hack, something like:
bpo-22536 [3.6] Set filename in FileNotFoundError (#3305)
* [3.6] bpo-22536: Set the filename in FileNotFoundError. (GH-3194)
Have the subprocess module set the filename in the FileNotFoundError
exception raised on POSIX systems when the executable or cwd are missing.
(cherry picked from commit 8621bb5d93239316f97281826461b85072ff6db7)
bpo-29212: Fix the ugly ThreadPoolExecutor thread name.
Fixes the newly introduced ugly default thread name for concurrent.futures
thread.ThreadPoolExecutor threads. They'll now resemble the old <=3.5
threading default Thread-x names by being named ThreadPoolExecutor-y_n..
(cherry picked from commit a3d91b43c2851312fb942f31afa12f5961706db6)
[3.6] bpo-9146: Raise a ValueError if OpenSSL fails to init a hash func (#3274)
* [3.6] bpo-9146: Raise a ValueError if OpenSSL fails to init a hash func. (GH-1777)
This helps people in weird FIPS mode environments where common things
like MD5 are not available in the binary as a matter of policy.
(cherry picked from commit 07244a83014fad42da937c17d98474b47a570bf7)
Victor Stinner [Fri, 1 Sep 2017 13:04:47 +0000 (15:04 +0200)]
bpo-31217: Fix regrtest -R for small integer (#3260) (#3261)
Use a pool of integer objects toprevent false alarm when checking for
memory block leaks. Fill the pool with values in -1000..1000 which
are the most common (reference, memory block, file descriptor)
differences.
Pauli Virtanen [Wed, 30 Aug 2017 09:40:05 +0000 (11:40 +0200)]
[3.6] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31) (#3241)
Ctypes currently produces wrong pep3118 type codes for several types.
E.g. memoryview(ctypes.c_long()).format gives "<l" on 64-bit platforms,
but it should be "<q" instead for sizeof(c_long) == 8
The problem is that the '<>' endian specification in the struct syntax
also turns on the "standard size" mode, which makes type characters have
a platform-independent meaning, which does not match with the codes used
internally in ctypes. The struct module format syntax also does not
allow specifying native-size non-native-endian items.
This commit adds a converter function that maps the internal ctypes
codes to appropriate struct module standard-size codes in the pep3118
format strings. The tests are modified to check for this.
(cherry picked from commit 07f1658aa09f6798793c473c72b2951b7fefe220)
projects / python / log