Antoine Pitrou [Thu, 25 May 2017 14:57:46 +0000 (16:57 +0200)]
[3.6] bpo-30414: multiprocessing.Queue._feed do not break from main loop on exc (GH-1683) (#1815)
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc
Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc
Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
(cherry picked from commit bc50f03db4f58c869b78e98468e374d7e61f1227)
Zachary Ware [Wed, 24 May 2017 21:11:01 +0000 (16:11 -0500)]
[3.6] bpo-30160: Clarify intended usage of wfile (gh-1300) (GH-1793)
The library does not enforce compliance with the HTTP protocol,
so violations are not technically disallowed. Extend the stream's
description to avoid suggesting that intentional protocol violations are
not supported.
(cherry picked from commit a083c8e)
Łukasz Langa [Tue, 23 May 2017 05:23:29 +0000 (22:23 -0700)]
[3.6] bpo-23894: make lib2to3 recognize f-strings (GH-1733) (#1737)
Note: this doesn't unpack f-strings into the underlying JoinedStr AST.
Ideally we'd fully implement JoinedStr here but given its additional
complexity, I think this is worth bandaiding as is. This unblocks tools like
https://github.com/google/yapf to format 3.6 syntax using f-strings.
(cherry picked from commit 1b9530c536664276ce866ae602ce04adce0810e1)
head_lock could be held by another thread when fork happened. We should
reset it to avoid deadlock.
(cherry picked from commit f82c951d1c5416f3550d544e50ff5662d3836e73)
Victor Stinner [Wed, 17 May 2017 00:06:14 +0000 (17:06 -0700)]
bpo-30357: test_thread now uses threading_cleanup() (#1592) (#1622)
test_thread: setUp() now uses support.threading_setup() and
support.threading_cleanup() to wait until threads complete to avoid
random side effects on following tests.
* Use explicit numbering for footnotes referred by explicit number.
* Restore missed footnote reference in stdtypes.rst.
* Fix literal strings formatting in howto/urllib2.rst.
* Update susp-ignored.csv for zipapp.rst.
* Fix suspicious mark up in Misc/NEWS..
(cherry picked from commit d97b7dc94b19063f0589d401bdc4aaadc7030762)
Serhiy Storchaka [Tue, 16 May 2017 15:16:15 +0000 (18:16 +0300)]
[3.6] bpo-30375: Correct the stacklevel of regex compiling warnings. (GH-1595) (#1604)
Warnings emitted when compile a regular expression now always point
to the line in the user code. Previously they could point into inners
of the re module if emitted from inside of groups or conditionals..
(cherry picked from commit c7ac7280c321b3c1679fe5f657a6be0f86adf173)
Victor Stinner [Wed, 10 May 2017 06:47:22 +0000 (08:47 +0200)]
bpo-30320: test_eintr now uses pthread_sigmask() (#1523) (#1524)
Rewrite sigwaitinfo() and sigtimedwait() unit tests for EINTR using
pthread_sigmask() to fix a race condition between the child and the
parent process.
Remove the pipe which was used as a weak workaround against the race
condition.
sigtimedwait() is now tested with a child process sending a signal
instead of testing the timeout feature which is more unstable
(especially regarding to clock resolution depending on the platform).
(cherry picked from commit 211a392cc15f9a7b1b8ce65d8f6c9f8237d1b77f)
Serhiy Storchaka [Wed, 10 May 2017 03:44:02 +0000 (06:44 +0300)]
[3.6] bpo-30298: Weaken the condition of deprecation warnings for inline modifiers. (GH-1490) (#1525)
Now allowed several subsequential inline modifiers at the start of the
pattern (e.g. '(?i)(?s)...'). In verbose mode whitespaces and comments
now are allowed before and between inline modifiers (e.g.
'(?x) (?i) (?s)...')..
(cherry picked from commit 305ccbe27ea5ba82fd2d8c32ec739f980e524330)
torsava [Tue, 9 May 2017 14:05:10 +0000 (16:05 +0200)]
[3.6] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478) (#1518)
* bpo-29243: Fix Makefile with respect to --enable-optimizations (#1478)
* bpo-29243: Fix Makefile with respect to --enable-optimizations
When using the Profile Guided Optimization (./configure --enable-optimizations)
Python is built not only during `make` but rebuilt again during `make test`,
`make install` and others. This patch fixes the issue.
Note that this fix produces no change at all in the Makefile if configure is
run witout --enable-optimizations.
* [3.6] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478)
* bpo-29243: Fix Makefile with respect to --enable-optimizations
When using the Profile Guided Optimization (./configure --enable-optimizations)
Python is built not only during `make` but rebuilt again during `make test`,
`make install` and others. This patch fixes the issue.
Note that this fix produces no change at all in the Makefile if configure is
run witout --enable-optimizations.
Victor Stinner [Fri, 5 May 2017 08:04:57 +0000 (10:04 +0200)]
bpo-30264: ExpatParser closes the source on error (#1451) (#1474)
ExpatParser.parse() of xml.sax.xmlreader now always closes the
source: close the file object or the urllib object if source is a
string (not an open file-like object). The change fixes a
ResourceWarning on parsing error.
[3.6] bpo-30243: Fixed the possibility of a crash in _json. (GH-1420) (#1469)
It was possible to get a core dump by using uninitialized
_json objects. Now __new__ methods create initialized objects.
__init__ methods are removed..
(cherry picked from commit 76a3e51a403bc84ed536921866c86dd7d07aaa7e)
Victor Stinner [Thu, 4 May 2017 22:46:56 +0000 (00:46 +0200)]
[3.6] bpo-23404: make touch becomes make regen-all (#1405) (#1461)
* bpo-23404: make touch becomes make regen-all (#1405)
Don't rebuild generated files based on file modification time
anymore, the action is now explicit. Replace "make touch"
with "make regen-all".
Changes:
* Remove "make touch", Tools/hg/hgtouch.py and .hgtouch
* Add a new "make regen-all" command to rebuild all generated files
* Add subcommands to only generate specific files:
- regen-ast: Include/Python-ast.h and Python/Python-ast.c
- regen-grammar: Include/graminit.h and Python/graminit.c
- regen-importlib: Python/importlib_external.h and Python/importlib.h
- regen-opcode: Include/opcode.h
- regen-opcode-targets: Python/opcode_targets.h
- regen-typeslots: Objects/typeslots.inc
* Rename PYTHON_FOR_GEN to PYTHON_FOR_REGEN
* pgen is now only built by by "make regen-grammar"
* Add $(srcdir)/ prefix to paths to source files to handle correctly
compilation outside the source directory
Note: $(PYTHON_FOR_REGEN) is no more used nor needed by "make"
default target building Python.
(cherry picked from commit a5c62a8e9f0de6c4133825a5710984a3cd5e102b)
Victor Stinner [Thu, 4 May 2017 11:21:10 +0000 (13:21 +0200)]
bpo-30225: Fix is_valid_fd() on macOS Tiger (#1443) (#1449)
is_valid_fd() now uses fstat() instead of dup() on macOS to return 0
on a pipe when the other side of the pipe is closed. fstat() fails
with EBADF in that case, whereas dup() succeed.
(cherry picked from commit 1c4670ea0cc3d208121af11b9b973e6bb268e570)
Mariatta [Wed, 3 May 2017 16:38:01 +0000 (09:38 -0700)]
[3.6] bpo-28556: Routine updates to typing (GH-1366) (#1416)
- Add NoReturn type
- Use WrapperDescriptorType (original PR by Jim Fasarakis-Hilliard)
- Minor bug-fixes
(cherry picked from commit f06e0218ef6007667f5d61184b85a81a0466d3ae)
Victor Stinner [Wed, 3 May 2017 00:12:22 +0000 (02:12 +0200)]
regrtest: always show before/after of modified env (#1192) (#1406)
Buildbots don't run tests with -vv and so only log "xxx was modified
by test_xxx" which is not enough to debug such random issue. In many
cases, I'm unable to reproduce the warning and so unable to fix it.
Always logging the value before and value after should help to debug
such warning on buildbots.
(cherry picked from commit ec4b17239d899550be4ee6104b61751bb3c70382)
Victor Stinner [Tue, 2 May 2017 14:49:10 +0000 (16:49 +0200)]
bpo-30199: test_ssl closes all asyncore channels (#1381) (#1390)
AsyncoreEchoServer of test_ssl now calls
asyncore.close_all(ignore_all=True) to ensure that
asyncore.socket_map is cleared once the test completes, even if
ConnectionHandler was not correctly unregistered.
Fix the following warning:
Warning -- asyncore.socket_map was modified by test_ssl
Before: {}
After: {6: <test.test_ssl.AsyncoreEchoServer.EchoServer.ConnectionHandler>}
(cherry picked from commit 1dae7450c68bad498e57800387b24cb103c461fa)
Victor Stinner [Tue, 2 May 2017 13:50:42 +0000 (15:50 +0200)]
bpo-30132: distutils BuildExtTestCase use temp_cwd (#1387)
BuildExtTestCase of test_distutils now uses support.temp_cwd() in
setUp() to remove files created in the current working in all
BuildExtTestCase unit tests, not only test_build_ext().
Fix the following warning:
Warning -- files was modified by test_distutils
Before: []
After: ['vc140.pdb']
Victor Stinner [Tue, 2 May 2017 09:45:42 +0000 (11:45 +0200)]
bpo-30108: Restore sys.path in test_site (#1197) (#1378)
Add setUpModule() and tearDownModule() functions to test_site to
save/restore sys.path at the module level to prevent warning if the
user site directory is created, since site.addsitedir() modifies
sys.path.
(cherry picked from commit b85c136903c6d2368162f7c4a58f258c9c69ead0)
Victor Stinner [Tue, 2 May 2017 08:47:37 +0000 (10:47 +0200)]
bpo-30104: Use -fno-strict-aliasing on clang (#1376)
Python/dtoa.c is not compiled correctly with clang 4.0 and
optimization level -O2 or higher, because of an aliasing issue on the
double/ULong[2] union. Only compile dtoa.c with -fno-strict-aliasing.
[3.6] bpo-30197: Enhance functions swap_attr() and swap_item() in test.support. (GH-1341) (#1345)
They now work when delete replaced attribute or item inside the with
statement. The old value of the attribute or item (or None if it doesn't
exist) now will be assigned to the target of the "as" clause, if there is
one.
Victor Stinner [Fri, 28 Apr 2017 02:13:53 +0000 (04:13 +0200)]
bpo-30131: test_logging now joins queue threads (#1298) (#1317)
QueueListenerTest of test_logging now closes the multiprocessing
Queue and joins its thread to prevent leaking dangling threads to
following tests.
Add also @support.reap_threads to detect earlier if a test leaks
threads (and try to "cleanup" these threads).
(cherry picked from commit 8ca2f2faefa8dba323a2e4c4b86efb633d7a53cf)
Don't use addCleanup() in test_quick_connect() because it keeps the
Thread object alive and so @reap_threads fails on its timeout of 1
second. "./python -m test -v test_asyncore -m test_quick_connect"
now takes 185 ms, instead of 11 seconds.
Other minor changes:
* Use "with sock:" to close the socket instead of
try/finally: sock.close()
* Use self.skipTest() in test_quick_connect() to remove one
indentation level and notice user that the test is specific to
AF_INET and AF_INET6
* bpo-30106: Fix tearDown() of test_asyncore (#1194)
Call asyncore.close_all() with ignore_all=True in the tearDown()
method of the test_asyncore base test case. It should prevent keeping
alive sockets in asyncore.socket_map if close() fails with an
unexpected error.
Revert also an unwanted change of my previous commit: remove name
parameter of Thread in test_quick_connect().
test_quick_connect() runs a thread up to 50 seconds, whereas the
socket is connected in 0.2 second and then the thread is expected to
end in less than 3 second. On Linux, the thread ends quickly because
select() seems to always return quickly. On FreeBSD, sometimes
select() fails with timeout and so the thread runs much longer than
expected.
Fix the thread timeout to fix a race condition in the test.
Victor Stinner [Fri, 28 Apr 2017 01:27:51 +0000 (03:27 +0200)]
bpo-30107: Make SuppressCrashReport quiet on macOS (#1279) (#1335)
On macOS, SuppressCrashReport now redirects /usr/bin/defaults command
stderr into a pipe to not pollute stderr. It fixes a
test_io.test_daemon_threads_shutdown_stderr_deadlock() failure when
the CrashReporter domain doesn't exists. Message logged into stderr:
2017-04-24 16:57:21.432 defaults[41046:2462851]
The domain/default pair of (com.apple.CrashReporter, DialogType) does not exist
(cherry picked from commit d819ad9832292d854e9710493ecdf959b69802e3)
bpo-29974: Improve typing.TYPE_CHECKING example (GH-982)
* Fix PEP 8 (SomeType instead of some_type)
* Add a function parameter annotation
* Explain, using wording from PEP 484 and PEP 526,
why one annotation is in quotes and another is not.
These tests call Py_FatalError() if the expected bug is triggered
which calls abort(). Use test.support.SuppressCrashReport to prevent
the creation on a core dump, to fix the warning: