Victor Stinner [Tue, 5 Sep 2017 23:35:39 +0000 (01:35 +0200)]
bpo-31339: Rewrite time.asctime() and time.ctime() (#3293)
* bpo-31339: Rewrite time.asctime() and time.ctime()
Backport and adapt the _asctime() function from the master branch to
not depend on the implementation of asctime() and ctime() from the
external C library. This change fixes a bug when Python is run using
the musl C library.
bpo-27448: Work around a gc.disable race condition in subprocess. (#1932)
* bpo-27448: Work around a gc.disable race condition in subprocess.
This works around a gc.isenabled/gc.disable race condition in the 2.7
subprocess module by using a lock for the critical section. It'll
prevent multiple simultaneous subprocess launches from winding up with
gc remaining disabled but it can't fix the ultimate problem: gc enable
and disable is a global setting and a hack.
Users are *strongly encouraged* to use subprocess32 from PyPI instead
of the 2.7 standard library subprocess module. Mixing threads with
subprocess is a recipie for disaster otherwise even with "fixes" to
ameliorate common issues like this.
The ssl and hashlib modules now call OPENSSL_add_all_algorithms_noconf() on
OpenSSL < 1.1.0. The function detects CPU features and enables optimizations
on some CPU architectures such as POWER8. Patch is based on research from
Gustavo Serra Scalet.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit c941e62)
Version breakdown, support disabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will not be defined ->
False/False
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will be defined -> True/False
Version breakdown support enabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True
* Refine NPN guard:
- If NPN is disabled, but ALPN is available we need our callback
- Make clinic's ssl behave the same way
This created a working ssl module for me, with NPN disabled and ALPN
enabled for OpenSSL 1.1.0f.
Concerns to address:
The initial commit for NPN support into OpenSSL [1], had the
OPENSSL_NPN_* variables defined inside the OPENSSL_NO_NEXTPROTONEG
guard. The question is if that ever made it into a release.
This would need an ugly hack, something like:
Zachary Ware [Mon, 4 Sep 2017 23:05:33 +0000 (16:05 -0700)]
[2.7] bpo-30450: Pull Windows dependencies from GitHub rather than SVN (GH-1783) (GH-3306)
The Windows build now depends on Python 3.6 to fetch externals, but it
will be downloaded via NuGet (which is downloaded via PowerShell) if it
is not available via `py -3.6`. This means the only thing that must be
installed on a modern Windows box to do a full build of CPython with all
extensions is Visual Studio.
* Update sqlite3 to 3.14.2.0 since it's the version we use on 3.x, and it's simpler to just use it than to also upload the old version to cpython-source-deps
Pauli Virtanen [Sat, 2 Sep 2017 16:24:32 +0000 (18:24 +0200)]
[2.7] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31) (#3242)
[2.7] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31)
Ctypes currently produces wrong pep3118 type codes for several types.
E.g. memoryview(ctypes.c_long()).format gives "<l" on 64-bit platforms,
but it should be "<q" instead for sizeof(c_long) == 8
The problem is that the '<>' endian specification in the struct syntax
also turns on the "standard size" mode, which makes type characters have
a platform-independent meaning, which does not match with the codes used
internally in ctypes. The struct module format syntax also does not
allow specifying native-size non-native-endian items.
This commit adds a converter function that maps the internal ctypes
codes to appropriate struct module standard-size codes in the pep3118
format strings. The tests are modified to check for this..
(cherry picked from commit 07f1658aa09f6798793c473c72b2951b7fefe220)
Victor Stinner [Tue, 22 Aug 2017 01:40:26 +0000 (03:40 +0200)]
bpo-30871: Add test.pythoninfo (#3174) (#3175)
* bpo-30871: Add test.pythoninfo (#3075)
* Add Lib/test/pythoninfo.py: script collecting various informations
about Python to help debugging test failures.
* regrtest: remove sys.hash_info and sys.flags from header.
* Travis CI, Appveyor: run pythoninfo before tests
(cherry picked from commit b907abc88589f7bea52c5afe172ececc6edcda70)
* bpo-30871: pythoninfo: add expat and _decimal (#3121)
Mariatta [Fri, 18 Aug 2017 13:22:32 +0000 (06:22 -0700)]
Fix broken `Show Source` links on documentation pages (GH-3113) (GH-3128)
The `Show Source` was broken because of a change made in sphinx 1.5.1
In Sphinx 1.4.9, the sourcename was "index.txt".
In Sphinx 1.5.1+, it is now "index.rst.txt".
Julien Palard [Thu, 17 Aug 2017 14:37:56 +0000 (16:37 +0200)]
Update the language selection in the docs language switch. (GH-3114) (#3117)
Change the option for `Français` into `French` to be consistent with the other language selections that are already in English.
(cherry picked from commit b616b972999cdd5fe54ef8a43c131a27ca538dee)
Victor Stinner [Wed, 16 Aug 2017 11:14:40 +0000 (13:14 +0200)]
bpo-31069, test_multiprocessing: Fix dangling process (#3103) (#3105)
Fix a warning about dangling processes in test_rapid_restart() of
_test_multiprocessing: join the process.
(cherry picked from commit 17657bb9458ff8f8804b7637d61686a68f4b9471)
Christian Heimes [Tue, 15 Aug 2017 08:55:03 +0000 (10:55 +0200)]
[2.7] bpo-30714: ALPN changes for OpenSSL 1.1.0f (#3094)
OpenSSL 1.1.0 to 1.1.0e aborted the handshake when server and client
could not agree on a protocol using ALPN. OpenSSL 1.1.0f changed that.
The most recent version now behaves like OpenSSL 1.0.2 again. The ALPN
callback can pretend to not been set.
See https://github.com/openssl/openssl/pull/3158 for more details
Julien Palard [Sun, 13 Aug 2017 21:44:51 +0000 (23:44 +0200)]
bpo-31159: fix language switch regex on unknown yet built languages. … (#3051) (#3081)
* bpo-31159: fix language switch regex on unknown yet built languages. (#3039)
This fix a regex issue (a missing non-matching group around an 'or'
list) and the specific possible case where a translation is built but
not yet in known by the picker, but not explicitly listing possible
languages in the regex.
(cherry picked from commit 122081deef86174beee965be1207fa46ea23533d)
* bpo-31149: Doc: Add Japanese to the language switcher. (#3028)
Victor Stinner [Fri, 11 Aug 2017 12:38:37 +0000 (14:38 +0200)]
bpo-31173: Rewrite WSTOPSIG test of test_subprocess (#3055) (#3071)
The current test_child_terminated_in_stopped_state() function test
creates a child process which calls ptrace(PTRACE_TRACEME, 0, 0) and
then crash (SIGSEGV). The problem is that calling os.waitpid() in the
parent process is not enough to close the process: the child process
remains alive and so the unit test leaks a child process in a
strange state. Closing the child process requires non-trivial code,
maybe platform specific.
Remove the functional test and replaces it with an unit test which
mocks os.waitpid() using a new _testcapi.W_STOPCODE() function to
test the WIFSTOPPED() path.
(cherry picked from commit 7b7c6dcfff6a35333988a3c74c895ed19dff2e09)
TestRandomNameSequence.test_process_awareness() now calls
os.waitpid() to avoid leaking a zombie process.
(cherry picked from commit 6c8c2943d996b59a48d331f61f22cbe72933910e)
Victor Stinner [Thu, 27 Jul 2017 16:44:43 +0000 (18:44 +0200)]
[2.7] bpo-31044: Skip test_posix.test_makedev() on FreeBSD (#2915) (#2918)
* bpo-31044: Skip test_posix.test_makedev() on FreeBSD (#2915)
There is a bug in FreeBSD CURRENT with 64-bit dev_t. Skip the test if
dev_t is larger than 32-bit, until the bug is fixed in FreeBSD
CURRENT.
(cherry picked from commit 12953ffe12ac781332b384c36b25d12216b1db62)
bpo-30980: Fix double close in asyncore.file_wrapper (#2789) (#2900)
* bpo-30980: Fix close test to fail
test_close_twice was not considering the fact that file_wrapper is
duping the file descriptor. Closing the original descriptor left the
duped one open, hiding the fact that close protection is not effective.
* bpo-30980: Fix double close protection
Invalidated self.fd before closing, handling correctly the case when
os.close raises.
* bpo-30980: Fix fd leak introduced in the fixed test
multiprocessing.Queue.get() with a timeout now polls its reader in
non-blocking mode if it succeeded to aquire the lock but the acquire
took longer than the timeout.
_test_multiprocessing.test_queue_feeder_donot_stop_onexc() now uses a
timeout of 1 second on Queue.get(), instead of 0.1 second, for slow
buildbots.
(cherry picked from commit 8f6eeaf21cdf4aea25fdefeec814a1ce07453fe9)
Victor Stinner [Mon, 24 Jul 2017 11:01:59 +0000 (13:01 +0200)]
test_bsddb3 tolerates smaller timeout on Windows (#2840)
bpo-30850: On Windows, test04_lock_timeout2() now tolerates 50 ms
whereas 100 ms is expected. The lock sometimes times out after only
58 ms. Windows clocks have a bad resolution and bad accuracy.
Nicholas [Fri, 21 Jul 2017 06:29:44 +0000 (23:29 -0700)]
[2.7] bpo-30964: Mention ensurepip in package installation docs (GH-2795)
Adds a new 'Pip not installed' section that covers
running `ensurepip` manually, and also references
the relevant section of the Python Packaging User
Guide.
[2.7] bpo-29854: Fix segfault in call_readline() (GH-728)
If history-length is set in .inputrc, and the history file is double the
history size (or more), history_get(N) returns NULL, and python
segfaults. Fix that by checking for NULL return value.
It seems that the root cause is incorrect handling of bigger history in
readline, but Python should not segfault even if readline returns
unexpected value.
This issue affects only GNU readline. When using libedit emulation
system history size option does not work.
This is a backport of the actual fix from master without the test, since
the test depends on new run_pty() helper which is not available in 2.7.
Victor Stinner [Wed, 5 Jul 2017 08:52:06 +0000 (10:52 +0200)]
bpo-30759: regrtest: list_cases() now unload modules (#2582)
list_cases() now unload modules, as the test runner does, to prevent
a failure in test_xpickle about test.pickletester loaded after
loading test_cpickle:
./python -m test --list-cases test_cpickle test_xpickle
Victor Stinner [Wed, 5 Jul 2017 08:00:33 +0000 (10:00 +0200)]
bpo-30759: Copy test_robotparser from master (#2546)
Copy Lib/test/test_robotparser.py from master to 2.7 and adapt it for
Python 2.7:
* Replace urllib.robotparser with robotparser
* Adjust HTTPServer import
* Replace io.StringIO with StringIO.StringIO
* Remove tests on crawl_delay() and request_rate() since these
methods were added to Python 3
* Remove subTest()
* Add test_main() which explicitly lists all test cases
Victor Stinner [Tue, 4 Jul 2017 09:36:16 +0000 (11:36 +0200)]
bpo-29796: test_weakref: Fix collect_in_thread() on Windows (#2553)
Sleep 1 ms instead of 0.1 ms to workaround a rounding issue on
Windows. On Windows, time.sleep(0.0001) sleeps 0 ms, so
collect_in_thread() calls gc.collect() in a loop and tests using this
thread takes too long. Sleep 1 ms so time.sleep() sleeps 15.6 ms on
Windows.
[2.7] Rename test_ file that is really a support file to remove test_ prefix. (#2548)
I thought I had run the full test suite before the last checkin, but
obviously I didn't. test_multibytecodec_support.py isn't really a test file,
it is a support file that contains a base test class. Rename it to
multibytecodec_support so that regrtest test discovery doesn't think it is a
test file that should be run..
(cherry picked from commit 75d9aca97ad3c3d823d2f2211f579454b3216f51)
* regrtest --list-cases now supports --match and --match-file options.
Example: ./python -m test --list-cases -m FileTests test_os
* --list-cases now also sets support.verbose to False to prevent
messages to stdout when loading test modules.
* Add support._match_test() private function.
(cherry picked from commit ace56d583664f855d89d1219ece7c21c2fddcf30)
* Change the regrtest --huntrleaks checker to decide if a test file
leaks or not. Require that each run leaks at least 1 reference.
* Warmup runs are now completely ignored: ignored in the checker test
and not used anymore to compute the sum.
* Add an unit test for a reference leak.
Example of reference differences previously considered a failure
(leak) and now considered as success (success, no leak):
* Add test.support._crash_python() which triggers a crash but uses
test.support.SuppressCrashReport() to prevent a crash report from
popping up.
* Modify test_child_terminated_in_stopped_state() of test_subprocess
and test_crashed() of test_regrtest to use _crash_python().
Victor Stinner [Tue, 27 Jun 2017 12:31:40 +0000 (14:31 +0200)]
bpo-30764: Windows support.SuppressCrashReport (#2423)
* Add Windows support to test.support.SuppressCrashReport: call
SetErrorMode() and CrtSetReportMode().
* _testcapi: add CrtSetReportMode() and CrtSetReportFile() functions
and CRT_xxx and CRTDBG_xxx constants needed by SuppressCrashReport.
Backport test.support.SuppressCrashReport context-manager from
master. Drop the Windows implementation since it depends on
msvcrt.CrtSetReportMode() which isn't available on Python 2.7.
bpo-30764, bpo-29335: test_child_terminated_in_stopped_state() of
test_subprocess now uses support.SuppressCrashReport() to prevent the
creation of a core dump on FreeBSD.
(cherry picked from commit cdee3f14f7f4c995e7eedb0bf6a67e260c739f7d)