]> granicus.if.org Git - python/log
python
4 years agobpo-38449: Revert "bpo-22347: Update mimetypes.guess_type to allow oper parsing of...
Abhilash Raj [Sat, 12 Oct 2019 05:41:35 +0000 (22:41 -0700)]
bpo-38449: Revert "bpo-22347: Update mimetypes.guess_type to allow oper parsing of URLs (GH-15522)" (GH-16724)

This reverts commit 87bd2071c756188b6cd577889fb1682831142ceb.

https://bugs.python.org/issue38449

4 years agobpo-38347: find pathfix for Python scripts whose name contain a '-' (GH-16536)
Ruediger Pluem [Fri, 11 Oct 2019 13:36:50 +0000 (15:36 +0200)]
bpo-38347: find pathfix for Python scripts whose name contain a '-' (GH-16536)

 pathfix.py: Assume all files that end on '.py' are Python scripts when working recursively.

4 years agobpo-38442: Remove an execution bit from Doc/whatsnew/3.8.rst. (GH-16715)
Serhiy Storchaka [Fri, 11 Oct 2019 08:44:42 +0000 (11:44 +0300)]
bpo-38442: Remove an execution bit from Doc/whatsnew/3.8.rst. (GH-16715)

4 years agoFix strict-aliasing rules errors on gcc 4.8.5. (GH-16714)
Dong-hee Na [Fri, 11 Oct 2019 08:43:11 +0000 (17:43 +0900)]
Fix strict-aliasing rules errors on gcc 4.8.5. (GH-16714)

4 years agoTypo fix: "throuhgh" should be "through". (GH-16704)
Hansraj Das [Fri, 11 Oct 2019 03:43:13 +0000 (09:13 +0530)]
Typo fix: "throuhgh" should be "through". (GH-16704)

4 years agodocs: Add asyncio source code links (GH-16640)
Kyle Stanley [Thu, 10 Oct 2019 23:18:46 +0000 (19:18 -0400)]
docs: Add asyncio source code links (GH-16640)

4 years agobpo-38437: Activate GC_DEBUG when PY_DEBUG is set (GH-16707)
Pablo Galindo [Thu, 10 Oct 2019 21:45:17 +0000 (22:45 +0100)]
bpo-38437: Activate GC_DEBUG when PY_DEBUG is set (GH-16707)

4 years agoRemove AppVeyor badge now that we don't use it in the CI anymore (GH-16708)
Pablo Galindo [Thu, 10 Oct 2019 21:39:15 +0000 (22:39 +0100)]
Remove AppVeyor badge now that we don't use it in the CI anymore (GH-16708)

4 years agobpo-38282: Rewrite getsockaddrarg() helper function (GH-16698)
Victor Stinner [Thu, 10 Oct 2019 19:30:20 +0000 (21:30 +0200)]
bpo-38282: Rewrite getsockaddrarg() helper function (GH-16698)

Rewrite getsockaddrarg() helper function of socketmodule.c (_socket
module) to prevent a false alarm when compiling codde using GCC with
_FORTIFY_SOURCE=2. Pass a pointer of the sock_addr_t union, rather
than passing a pointer to a sockaddr structure.

Add "struct sockaddr_tipc tipc;" to the sock_addr_t union.

4 years agoRe-enable the OverflowError test for test_truediv on test_complex (GH-16591)
Dong-hee Na [Thu, 10 Oct 2019 16:23:36 +0000 (01:23 +0900)]
Re-enable the OverflowError test for test_truediv on test_complex (GH-16591)

4 years agobpo-38353: Fix typos in calculate_argv0_path_framework() (GH-16695)
Victor Stinner [Thu, 10 Oct 2019 13:42:30 +0000 (15:42 +0200)]
bpo-38353: Fix typos in calculate_argv0_path_framework() (GH-16695)

bpo-38353, bpo-38429: Fix typos introduced by commit
c02b41b1fb115c87693530ea6a480b2e15460424 in
calculate_argv0_path_framework() of getpath.c.

4 years agobpo-32996: Documentation fix-up. (GH-16646)
M. Eric Irrgang [Thu, 10 Oct 2019 11:11:33 +0000 (14:11 +0300)]
bpo-32996: Documentation fix-up. (GH-16646)

PR #4906 changed the typing.Generic class hierarchy, leaving an
outdated comment in the library reference. User-defined Generic ABCs now
must get a abc.ABCMeta metaclass from something other than typing.Generic
inheritance.

4 years agobpo-38425: Fix ‘res’ may be used uninitialized warning (GH-16688)
Dong-hee Na [Thu, 10 Oct 2019 07:41:26 +0000 (16:41 +0900)]
bpo-38425: Fix ‘res’ may be used uninitialized warning (GH-16688)

4 years agobpo-38109: Add missing constants to Lib/stat.py (GH-16665)
Ronan Lamy [Thu, 10 Oct 2019 07:34:46 +0000 (09:34 +0200)]
bpo-38109: Add missing constants to Lib/stat.py (GH-16665)

Add missing stat.S_IFDOOR, stat.S_IFPORT, stat.S_IFWHT,
stat.S_ISDOOR, stat.S_ISPORT, and stat.S_ISWHT values to
the Python implementation of the stat module.

4 years agobpo-38392: Only declare visit_validate() if Py_DEBUG is defined (GH-16689)
Victor Stinner [Thu, 10 Oct 2019 07:32:13 +0000 (09:32 +0200)]
bpo-38392: Only declare visit_validate() if Py_DEBUG is defined (GH-16689)

bpo-38392, bpo-38426: Fix a compiler warning in gcmodule.c.

Fix also a typo in PYMEM_DEADBYTE macro comment.

4 years agobpo-38409: Fix grammar in str.strip() docstring (GH-16682)
Zachary Ware [Wed, 9 Oct 2019 21:09:00 +0000 (16:09 -0500)]
bpo-38409: Fix grammar in str.strip() docstring (GH-16682)

4 years agobpo-38379: don't claim objects are collected when they aren't (#16658)
Tim Peters [Wed, 9 Oct 2019 17:37:30 +0000 (12:37 -0500)]
bpo-38379:  don't claim objects are collected when they aren't (#16658)

* bpo-38379:  when a finalizer resurrects an object,
nothing is actually collected in this run of gc.
Change the stats to relect that truth.

4 years agoTypo fix: "empy" should be "empty". (GH-16666)
Hansraj Das [Wed, 9 Oct 2019 02:24:02 +0000 (07:54 +0530)]
Typo fix: "empy" should be "empty". (GH-16666)

4 years agocloses bpo-36161: Use thread-safe ttyname_r instead of ttyname. (GH-14868)
Antonio Gutierrez [Wed, 9 Oct 2019 02:19:48 +0000 (04:19 +0200)]
closes bpo-36161: Use thread-safe ttyname_r instead of ttyname. (GH-14868)

Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
4 years agobpo-38368: Added fix for ctypes crash when handling arrays in structs… (GH-16589)
Vinay Sajip [Tue, 8 Oct 2019 20:59:06 +0000 (21:59 +0100)]
bpo-38368: Added fix for ctypes crash when handling arrays in structs… (GH-16589)

4 years agobpo-37531: regrtest ignores output on timeout (GH-16659)
Victor Stinner [Tue, 8 Oct 2019 16:45:43 +0000 (18:45 +0200)]
bpo-37531: regrtest ignores output on timeout (GH-16659)

bpo-37531, bpo-38207: On timeout, regrtest no longer attempts to call
`popen.communicate() again: it can hang until all child processes
using stdout and stderr pipes completes. Kill the worker process and
ignores its output.

Reenable test_regrtest.test_multiprocessing_timeout().

bpo-37531: Change also the faulthandler timeout of the main process
from 1 minute to 5 minutes, for Python slowest buildbots.

4 years agotest_dictviews: Add testcase for dictviews_sub (GH-16660)
Dong-hee Na [Tue, 8 Oct 2019 15:59:10 +0000 (00:59 +0900)]
test_dictviews: Add testcase for dictviews_sub (GH-16660)

4 years agobpo-38395: Fix ownership in weakref.proxy methods (GH-16632)
Pablo Galindo [Tue, 8 Oct 2019 15:30:50 +0000 (16:30 +0100)]
bpo-38395: Fix ownership in weakref.proxy methods (GH-16632)

The implementation of weakref.proxy's methods call back into the Python
API using a borrowed references of the weakly referenced object
(acquired via PyWeakref_GET_OBJECT). This API call may delete the last
reference to the object (either directly or via GC), leaving a dangling
pointer, which can be subsequently dereferenced.

To fix this, claim a temporary ownership of the referenced object when
calling the appropriate method. Some functions because at the moment they
do not need to access the borrowed referent, but to protect against
future changes to these functions, ownership need to be fixed in
all potentially affected methods.

4 years agobpo-38118: Ignore Valgrind false alarm in PyUnicode_Decode() (GH-16651)
Victor Stinner [Tue, 8 Oct 2019 13:42:17 +0000 (15:42 +0200)]
bpo-38118: Ignore Valgrind false alarm in PyUnicode_Decode() (GH-16651)

Valgrind emits "Conditional jump or move depends on uninitialised
value(s)" false alarms on GCC builtin strcmp() function. The GCC code
is correct.

Valgrind bug: https://bugs.kde.org/show_bug.cgi?id=264936

4 years agobpo-38407: Add docstrings for typing.SupportsXXX classes. (GH-16644)
Serhiy Storchaka [Tue, 8 Oct 2019 13:30:17 +0000 (16:30 +0300)]
bpo-38407: Add docstrings for typing.SupportsXXX classes. (GH-16644)

4 years agobpo-38405: Make nested subclasses of typing.NamedTuple pickleable. (GH-16641)
Serhiy Storchaka [Tue, 8 Oct 2019 13:29:52 +0000 (16:29 +0300)]
bpo-38405: Make nested subclasses of typing.NamedTuple pickleable. (GH-16641)

4 years agobpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH...
Serhiy Storchaka [Tue, 8 Oct 2019 11:32:25 +0000 (14:32 +0300)]
bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)

It now escapes them with a backslash, as the regular Python interpreter.
Added the "errors" field to the standard streams.

4 years agobpo-38371: Tkinter: deprecate the split() method. (GH-16584)
Serhiy Storchaka [Tue, 8 Oct 2019 11:31:35 +0000 (14:31 +0300)]
bpo-38371: Tkinter: deprecate the split() method. (GH-16584)

4 years agobpo-33714: Output an exception raised in module's m_clear(). (GH-16592)
Serhiy Storchaka [Tue, 8 Oct 2019 10:46:17 +0000 (13:46 +0300)]
bpo-33714: Output an exception raised in module's m_clear(). (GH-16592)

It is similar to the more general code in the gc module, but
here we know the name of the module.

https://bugs.python.org/issue33714

Automerge-Triggered-By: @encukou
4 years agoFix typo in _warnings.warn_explicit() docstring (GH-16625)
Hansraj Das [Tue, 8 Oct 2019 08:56:07 +0000 (14:26 +0530)]
Fix typo in _warnings.warn_explicit() docstring (GH-16625)

4 years agocloses bpo-38402: Check error of primitive crypt/crypt_r. (GH-16599)
Antonio Gutierrez [Tue, 8 Oct 2019 04:22:17 +0000 (06:22 +0200)]
closes bpo-38402: Check error of primitive crypt/crypt_r. (GH-16599)

Checks also for encryption algorithms methods not supported in different
OSs.

Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
4 years agobpo-38070: Enhance visit_decref() debug trace (GH-16631)
Victor Stinner [Tue, 8 Oct 2019 00:37:38 +0000 (02:37 +0200)]
bpo-38070: Enhance visit_decref() debug trace (GH-16631)

subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.

4 years agobpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed (GH-16630)
Pablo Galindo [Mon, 7 Oct 2019 23:43:14 +0000 (00:43 +0100)]
bpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed (GH-16630)

Some objects like Py_None are not initialized with conventional means
that prepare the circular linked list pointers, leaving them unlinked
from the rest of the objects. For those objects, NULL pointers does
not mean that they are freed, so we need to skip the check in those
cases.

4 years agobpo-38392: PyObject_GC_Track() validates object in debug mode (GH-16615)
Victor Stinner [Mon, 7 Oct 2019 22:09:31 +0000 (00:09 +0200)]
bpo-38392: PyObject_GC_Track() validates object in debug mode (GH-16615)

In debug mode, PyObject_GC_Track() now calls tp_traverse() of the
object type to ensure that the object is valid: test that objects
visited by tp_traverse() are valid.

Fix pyexpat.c: only track the parser in the GC once the parser is
fully initialized.

4 years agobpo-36389: Add newline to _PyObject_AssertFailed() (GH-16629)
Victor Stinner [Mon, 7 Oct 2019 21:44:05 +0000 (23:44 +0200)]
bpo-36389: Add newline to _PyObject_AssertFailed() (GH-16629)

Add a newline between the verbose object dump and the Py_FatalError()
logs for readability.

4 years agobpo-38344: Fix syntax in activate.bat (GH-16533)
James Abel [Mon, 7 Oct 2019 21:07:19 +0000 (14:07 -0700)]
bpo-38344: Fix syntax in activate.bat (GH-16533)

4 years agobpo-38294: Add list of no-longer-escaped chars to re.escape documentation. (GH-16442)
Ricardo Bánffy [Mon, 7 Oct 2019 20:54:35 +0000 (21:54 +0100)]
bpo-38294: Add list of no-longer-escaped chars to re.escape documentation. (GH-16442)

Prior to 3.7, re.escape escaped many characters that don't have
special meaning in Python, but that use to require escaping in other
tools and languages. This commit aims to make it clear which characters
were, but are no longer escaped.

4 years agobpo-36389: Fix _PyBytesWriter in release mode (GH-16624)
Victor Stinner [Mon, 7 Oct 2019 20:31:42 +0000 (22:31 +0200)]
bpo-36389: Fix _PyBytesWriter in release mode (GH-16624)

Fix _PyBytesWriter API when Python is built in release mode with
assertions.

4 years agobpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
Victor Stinner [Mon, 7 Oct 2019 16:42:01 +0000 (18:42 +0200)]
bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)

bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.

4 years agobpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
Ben Harper [Mon, 7 Oct 2019 16:19:58 +0000 (12:19 -0400)]
bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)

4 years agoFix a compile warning in selectmodule.c. (GH-16617)
Xiang Zhang [Mon, 7 Oct 2019 12:01:54 +0000 (20:01 +0800)]
Fix a compile warning in selectmodule.c. (GH-16617)

4 years agobpo-38391: Fixing a typo for Py_DECREF (GH-16616)
Krishna Oza [Mon, 7 Oct 2019 11:57:45 +0000 (17:27 +0530)]
bpo-38391: Fixing a typo for Py_DECREF (GH-16616)

4 years agobpo-38353: Simplify calculate_pybuilddir() (GH-16614)
Victor Stinner [Mon, 7 Oct 2019 10:56:59 +0000 (12:56 +0200)]
bpo-38353: Simplify calculate_pybuilddir() (GH-16614)

Calling _Py_wfopen() is enough to check if filename is an existing
file or not. There is no need to check first isfile().

4 years agobpo-38210: Fix compiler warning in dictobject.c (GH-16611)
Victor Stinner [Mon, 7 Oct 2019 10:28:16 +0000 (12:28 +0200)]
bpo-38210: Fix compiler warning in dictobject.c (GH-16611)

The "tmp" variable is no longer used.

4 years agoFix a compile warning in dictobject.c (GH-16610)
Hai Shi [Mon, 7 Oct 2019 10:20:05 +0000 (18:20 +0800)]
Fix a compile warning in dictobject.c (GH-16610)

4 years agobpo-25988: Do not expose abstract collection classes in the collections module. ...
Serhiy Storchaka [Mon, 7 Oct 2019 09:10:15 +0000 (12:10 +0300)]
bpo-25988: Do not expose abstract collection classes in the collections module. (GH-10596)

4 years ago bpo-26510: Add versionchanged for required arg of add_subparsers (GH-16588)
Adam J. Stewart [Mon, 7 Oct 2019 02:08:48 +0000 (21:08 -0500)]
 bpo-26510: Add versionchanged for required arg of add_subparsers (GH-16588)

The `required` argument to `argparse.add_subparsers` was added in #3027. This PR specifies the earliest version of Python where it is available.

https://bugs.python.org/issue26510

Automerge-Triggered-By: @merwok
4 years agobpo-38383: Fix possible integer overflow in startswith() of bytes and bytearray....
Hai Shi [Sun, 6 Oct 2019 12:17:18 +0000 (20:17 +0800)]
bpo-38383: Fix possible integer overflow in startswith() of bytes and bytearray. (GH-16603)

4 years agobpo-38210: Fix intersection operation with dict view and iterator. (GH-16602)
Dong-hee Na [Sun, 6 Oct 2019 11:28:33 +0000 (20:28 +0900)]
bpo-38210: Fix intersection operation with dict view and iterator. (GH-16602)

4 years agobpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)
Andrei Troie [Sat, 5 Oct 2019 16:19:15 +0000 (17:19 +0100)]
bpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)

KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead.

4 years agobpo-38341: Add SMTPNotSupportedError in the exports of smtplib (#16525)
nde [Sat, 5 Oct 2019 00:30:58 +0000 (02:30 +0200)]
bpo-38341: Add SMTPNotSupportedError in the exports of smtplib (#16525)

Add SMTPNotSupportedError in the exports of smtplib

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
4 years agobpo-38353: getpath.c: allocates strings on the heap (GH-16585)
Victor Stinner [Fri, 4 Oct 2019 17:53:43 +0000 (19:53 +0200)]
bpo-38353: getpath.c: allocates strings on the heap (GH-16585)

* _Py_FindEnvConfigValue() now returns a string allocated
  by PyMem_RawMalloc().
* calculate_init() now decodes VPATH macro.
* Add calculate_open_pyenv() function.
* Add substring() and joinpath2() functions.

* Fix add_exe_suffix()

And a few cleanup changes.

4 years agobpo-38353: getpath.c uses dynamically allocated strings (GH-16582)
Victor Stinner [Fri, 4 Oct 2019 13:25:50 +0000 (15:25 +0200)]
bpo-38353: getpath.c uses dynamically allocated strings (GH-16582)

* PyCalculatePath: add "_macro" to the 3 fields storing macro values
* Add dynamically allocated strings to PyCalculatePath

4 years agobpo-38266: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal (GH-16558)
Joannah Nanjekye [Fri, 4 Oct 2019 11:35:42 +0000 (08:35 -0300)]
bpo-38266: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal (GH-16558)

Revert the removal of PyThreadState_DeleteCurrent() with documentation.

4 years agobpo-13153: Use OS native encoding for converting between Python and Tcl. (GH-16545)
Serhiy Storchaka [Fri, 4 Oct 2019 10:09:52 +0000 (13:09 +0300)]
bpo-13153: Use OS native encoding for converting between Python and Tcl. (GH-16545)

On Windows use UTF-16 (or UTF-32 for 32-bit Tcl_UniChar) with the
"surrogatepass" error handler for converting to/from Tcl Unicode objects.

On Linux use UTF-8 with the "surrogateescape" error handler for converting
to/from Tcl String objects.

Converting strings from Tcl to Python and back now never fails
(except MemoryError).

4 years agoUpdated CODEOWNERS to indicate ownership of some modules. (GH-16578)
Vinay Sajip [Fri, 4 Oct 2019 08:48:24 +0000 (09:48 +0100)]
Updated CODEOWNERS to indicate ownership of some modules. (GH-16578)

4 years agobpo-38235: Correct some arguments names in logging documentation (GH-16571)
Ashley Whetter [Fri, 4 Oct 2019 07:55:14 +0000 (00:55 -0700)]
bpo-38235: Correct some arguments names in logging documentation (GH-16571)

4 years agobpo-38353: Rework ismodule() in getpath.c (GH-16574)
Victor Stinner [Fri, 4 Oct 2019 01:53:26 +0000 (03:53 +0200)]
bpo-38353: Rework ismodule() in getpath.c (GH-16574)

* ismodule() no longer modify the input string.
* copy_absolute(), joinpath(): rename parameters and local variables.

4 years agobpo-38353: Add subfunctions to getpath.c (GH-16572)
Victor Stinner [Fri, 4 Oct 2019 00:22:39 +0000 (02:22 +0200)]
bpo-38353: Add subfunctions to getpath.c (GH-16572)

Following symbolic links is now limited to 40 attempts, just to
prevent loops.

Add subfunctions:

* Add resolve_symlinks()
* Add calculate_argv0_path_framework()
* Add calculate_which()
* Add calculate_program_macos()

Fix also _Py_wreadlink(): readlink() result type is Py_ssize_t, not
int.

4 years agobpo-38353: Fix compiler warning in internal headers (GH-16573)
Victor Stinner [Fri, 4 Oct 2019 00:21:05 +0000 (02:21 +0200)]
bpo-38353: Fix compiler warning in internal headers (GH-16573)

Replace "_PyRuntimeState" with "struct pyruntimestate" to avoid
a warning on typedef re-definition.

4 years agobpo-38353: Fix compiler warning in pycore_initconfig.h (GH-16570)
Victor Stinner [Thu, 3 Oct 2019 22:43:10 +0000 (00:43 +0200)]
bpo-38353: Fix compiler warning in pycore_initconfig.h (GH-16570)

Replace "_PyRuntimeState" with "struct pyruntimestate" to avoid
a warning on typedef re-definition.

4 years agobpo-38353: Fix calculate_argv0_path() for symlinks (GH-16549)
Victor Stinner [Thu, 3 Oct 2019 21:17:00 +0000 (23:17 +0200)]
bpo-38353: Fix calculate_argv0_path() for symlinks (GH-16549)

calculate_argv0_path() must read argv0_path link, not read
program_full_path link.

4 years agobpo-34344 Fix AbstractEventLoopPolicy.get_event_loop docstring (GH-16463)
idomic [Thu, 3 Oct 2019 21:08:29 +0000 (00:08 +0300)]
bpo-34344 Fix AbstractEventLoopPolicy.get_event_loop docstring (GH-16463)

4 years agobpo-38359: Ensures pyw.exe launcher reads correct registry key (GH-16561)
Steve Dower [Thu, 3 Oct 2019 15:31:21 +0000 (08:31 -0700)]
bpo-38359: Ensures pyw.exe launcher reads correct registry key (GH-16561)

4 years agobpo-38355: Fix ntpath.realpath failing on sys.executable (GH-16551)
Steve Dower [Thu, 3 Oct 2019 15:31:03 +0000 (08:31 -0700)]
bpo-38355: Fix ntpath.realpath failing on sys.executable (GH-16551)

4 years agobpo-36670: Enhance regrtest (GH-16556)
Victor Stinner [Thu, 3 Oct 2019 14:15:16 +0000 (16:15 +0200)]
bpo-36670: Enhance regrtest (GH-16556)

* Add log() method: add timestamp and load average prefixes
  to main messages.
* WindowsLoadTracker:

  * LOAD_FACTOR_1 is now computed using SAMPLING_INTERVAL
  * Initialize the load to the arithmetic mean of the first 5 values
    of the Processor Queue Length value (so over 5 seconds), rather
    than 0.0.
  * Handle BrokenPipeError and when typeperf exit.

* format_duration(1.5) now returns '1.5 sec', rather than
  '1 sec 500 ms'

4 years agobpo-36670: Enhance regrtest WindowsLoadTracker (GH-16553)
Victor Stinner [Thu, 3 Oct 2019 08:53:17 +0000 (10:53 +0200)]
bpo-36670: Enhance regrtest WindowsLoadTracker (GH-16553)

The last line is now passed to the parser even if it does not end
with a newline, but only if it's a valid value.

4 years agobpo-36670, regrtest: Fix WindowsLoadTracker() for partial line (GH-16550)
Victor Stinner [Wed, 2 Oct 2019 23:04:09 +0000 (01:04 +0200)]
bpo-36670, regrtest: Fix WindowsLoadTracker() for partial line (GH-16550)

WindowsLoadTracker.read_output() now uses a short buffer for
incomplete line.

4 years agobpo-38353: Cleanup includes in the internal C API (GH-16548)
Victor Stinner [Wed, 2 Oct 2019 21:51:20 +0000 (23:51 +0200)]
bpo-38353: Cleanup includes in the internal C API (GH-16548)

Use forward declaration of types to avoid includes in the internal C
API. Add also comment to justify other includes.

4 years agobpo-38338, test.pythoninfo: add more ssl infos (GH-16539)
Victor Stinner [Wed, 2 Oct 2019 15:52:35 +0000 (17:52 +0200)]
bpo-38338, test.pythoninfo: add more ssl infos (GH-16539)

test.pythoninfo now logs environment variables used by OpenSSL and
Python ssl modules, and logs attributes of 3 SSL contexts
(SSLContext, default HTTPS context, stdlib context).

4 years agobpo-36670: regrtest bug fixes (GH-16537)
Victor Stinner [Wed, 2 Oct 2019 11:35:11 +0000 (13:35 +0200)]
bpo-36670: regrtest bug fixes (GH-16537)

* Fix TestWorkerProcess.__repr__(): start_time is only valid
  if _popen is not None.
* Fix _kill(): don't set _killed to True if _popen is None.
* _run_process(): only set _killed to False after calling
  run_test_in_subprocess().

4 years agobpo-38343: Fixes version handling for nuget packages (GH-16527)
Steve Dower [Tue, 1 Oct 2019 23:42:24 +0000 (16:42 -0700)]
bpo-38343: Fixes version handling for nuget packages (GH-16527)

4 years agobpo-37802: Fix a compiler warning in longobject.c (GH-16517)
Victor Stinner [Tue, 1 Oct 2019 11:29:53 +0000 (13:29 +0200)]
bpo-37802: Fix a compiler warning in longobject.c (GH-16517)

bpo-37802, bpo-38321: Fix the following warnings:

    longobject.c(420): warning C4244: 'function': conversion from
    'unsigned __int64' to 'sdigit', possible loss of data

    longobject.c(428): warning C4267: 'function': conversion from
    'size_t' to 'sdigit', possible loss of data

4 years agobpo-37474: Don't call fedisableexcept() on FreeBSD (GH-16515)
Victor Stinner [Tue, 1 Oct 2019 11:12:29 +0000 (13:12 +0200)]
bpo-37474: Don't call fedisableexcept() on FreeBSD (GH-16515)

On FreeBSD, Python no longer calls fedisableexcept() at startup to
control the floating point control mode. The call became useless
since FreeBSD 6: it became the default mode.

4 years agobpo-38321: Fix compiler warning in _randommodule.c (GH-16512)
Victor Stinner [Tue, 1 Oct 2019 10:45:52 +0000 (12:45 +0200)]
bpo-38321: Fix compiler warning in _randommodule.c (GH-16512)

Fix the GCC warning: "initialization discards ‘const’ qualifier from
pointer target type".

4 years agobpo-36670: Multiple regrtest bugfixes (GH-16511)
Victor Stinner [Tue, 1 Oct 2019 10:29:36 +0000 (12:29 +0200)]
bpo-36670: Multiple regrtest bugfixes (GH-16511)

* Windows: Fix counter name in WindowsLoadTracker. Counter names are
  localized: use the registry to get the counter name. Original
  change written by Lorenz Mende.
* Regrtest.main() now ensures that the Windows load tracker is also
  killed if an exception is raised
* TestWorkerProcess now ensures that worker processes are no longer
  running before exiting: kill also worker processes when an
  exception is raised.
* Enhance regrtest messages and warnings: include test name,
  duration, add a worker identifier, etc.
* Rename MultiprocessRunner to TestWorkerProcess
* Use print_warning() to display warnings.

Co-Authored-By: Lorenz Mende <Lorenz.mende@gmail.com>
4 years agobpo-38304: PyConfig_InitPythonConfig() cannot fail anymore (GH-16509)
Victor Stinner [Tue, 1 Oct 2019 10:06:16 +0000 (12:06 +0200)]
bpo-38304: PyConfig_InitPythonConfig() cannot fail anymore (GH-16509)

PyConfig_InitPythonConfig() and PyConfig_InitIsolatedConfig() no
longer return PyStatus: they cannot fail anymore.

4 years agobpo-38304: Remove PyConfig.struct_size (GH-16500) (GH-16508)
Victor Stinner [Tue, 1 Oct 2019 08:56:37 +0000 (10:56 +0200)]
bpo-38304: Remove PyConfig.struct_size (GH-16500) (GH-16508)

For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.

We may revisit this in the future if we further explore the idea of offering a stable embedding API.

(cherry picked from commit bdace21b769998396d0ccc8da99a8ca9b507bfdf)

4 years agobpo-38319: Fix shutil._fastcopy_sendfile(): set sendfile() max block size (GH-16491)
Giampaolo Rodola [Tue, 1 Oct 2019 03:40:54 +0000 (11:40 +0800)]
bpo-38319: Fix shutil._fastcopy_sendfile(): set sendfile() max block size (GH-16491)

4 years agobpo-32689: Updates shutil.move to allow for Path objects to be used as source arg...
Maxwell A McKinnon [Tue, 1 Oct 2019 02:41:16 +0000 (19:41 -0700)]
bpo-32689: Updates shutil.move to allow for Path objects to be used as source arg (GH-15326)

Important work originally done by @emilyemorehouse two years ago and nearly ready to go in.

This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwise wonderful pathlib and PEP519. https://stackoverflow.com/questions/33625931/copy-file-with-pathlib-in-python.

This adds the outstanding test request from that PR @vstinner (https://github.com/python/cpython/pull/5393).

Test fails without the change, passes with it, along with every other test in test_shutil.

Some variants were experimented with to make the one line change and the most performant one was picked.

# Added Test for PathLike directory destination, the current fail case

```
Lib/test/test_shutil.py::TestMove::test_move_file_pathlike FAILED                                                               [100%]

============================================================== FAILURES ===============================================================
__________________________________________________ TestMove.test_move_file_pathlike ___________________________________________________

self = <test.test_shutil.TestMove testMethod=test_move_file_pathlike>

    def test_move_file_pathlike(self):
        # Move a file to another location on the same filesystem.
        src = pathlib.Path(self.src_file)
>       self._check_move_file(src, self.dst_dir, self.dst_file)

Lib/test/test_shutil.py:1563:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Lib/test/test_shutil.py:1545: in _check_move_file
    shutil.move(src, dst)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:562: in move
    real_dst = os.path.join(dst, _basename(src))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = PosixPath('/var/folders/r2/psq74t5x3nbfzlph8bh2pvdw0000gn/T/tmp9ie0wh9_/foo')

    def _basename(path):
        # A basename() variant which first strips the trailing slash, if present.
        # Thus we always get the last component of the path, even for directories.
        sep = os.path.sep + (os.path.altsep or '')
>       return os.path.basename(path.rstrip(sep))
E       AttributeError: 'PosixPath' object has no attribute 'rstrip'

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:526: AttributeError
============================================== 1 failed, 102 deselected in 0.30 seconds ===============================================
```

After change:

```
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/maxwellmckinnon/dev/cpython
plugins: cov-2.7.1, mock-1.10.4
collected 103 items / 102 deselected / 1 selected

Lib/test/test_shutil.py::TestMove::test_move_file_pathlike PASSED                                                               [100%]

============================================== 1 passed, 102 deselected in 0.06 seconds ===============================================
```

Running all the tests in test_shutil.py
```
╰─ pytest Lib/test/test_shutil.py -v
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/maxwellmckinnon/dev/cpython
plugins: cov-2.7.1, mock-1.10.4
collected 103 items

Lib/test/test_shutil.py::TestShutil::test_chown PASSED                                                                          [  0%]
Lib/test/test_shutil.py::TestShutil::test_copy PASSED                                                                           [  1%]
...
Lib/test/test_shutil.py::TermsizeTests::test_stty_match SKIPPED                                                                 [ 99%]
Lib/test/test_shutil.py::PublicAPITests::test_module_all_attribute PASSED                                                       [100%]

================================================ 96 passed, 7 skipped in 1.25 seconds =================================================
```

# Performance Considerations
Is it considered poor form to get rid of _basename altogether and make use of pathlib in the move function? I'm not sure if the idea is for all these modules to strictly avoid circular dependencies. They are already using os.path which is just as much a citizen in 3.8 as pathlib right?

e.g.

`real_dst = os.path.join(dst, _basename(src))`
becomes
`real_dst = Path(dst) / Path(src).name`

I've looked around and familiarized myself, and I now think importing pathlib here is fine. My only remaining concern is that of performance.

Here's the performance difference for this step.

```
In [46]: %timeit real_dst = os.path.join("a/b/c", _basename('b/'))
2.71 µs ± 62.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [47]: %timeit real_dst = Path("a/b/c") / Path('b/').name
12.4 µs ± 65.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```

Is 10us significant or insignificant compared to the least expensive operation this function will do? I don't know. Let's find out.

```
In [55]: %timeit os.rename('/tmp/a/a.txt', '/tmp/a/b.txt'); os.rename('/tmp/a/b.txt', '/tmp/a/a.txt')
124 µs ± 2.18 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```
62us to rename. 10us seems significant enough that we wouldn't want to favor the Path sugar suggestion. 16% speed decrease from adding the 10us.

What do people think? I was hoping to get to use pathlib.Path here, but I suspect for this low level move, it should be as fast as possible, and 16% is not worth one line of sugary code to me.

https://bugs.python.org/issue32689

Automerge-Triggered-By: @gvanrossum
4 years agoFix and improve `asyncio.run()` docs (GH-16403)
Kyle Stanley [Tue, 1 Oct 2019 00:12:21 +0000 (20:12 -0400)]
Fix and improve `asyncio.run()` docs (GH-16403)

4 years agobpo-38321: Fix _testcapimodule.c warning (GH-16494)
Victor Stinner [Mon, 30 Sep 2019 19:14:26 +0000 (21:14 +0200)]
bpo-38321: Fix _testcapimodule.c warning (GH-16494)

Fix the following warning:

    modules\_testcapimodule.c(6409):
    warning C4146: unary minus operator applied to unsigned type,
    result still unsigned

4 years agoClear weakrefs in garbage found by the GC (#16495)
Neil Schemenauer [Mon, 30 Sep 2019 17:06:45 +0000 (10:06 -0700)]
Clear weakrefs in garbage found by the GC (#16495)

Fix a bug due to the interaction of weakrefs and the cyclic garbage
collector. We must clear any weakrefs in garbage in order to prevent
their callbacks from executing and causing a crash.

4 years agobpo-38321: Fix PyCStructUnionType_update_stgdict() warning (GH-16492)
Victor Stinner [Mon, 30 Sep 2019 15:49:48 +0000 (17:49 +0200)]
bpo-38321: Fix PyCStructUnionType_update_stgdict() warning (GH-16492)

bpo-22273, bpo-38321: Fix following warning:

    modules\_ctypes\stgdict.c(704):
    warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data

4 years agobpo-38321: Fix _asynciomodule.c compiler warning (GH-16493)
Victor Stinner [Mon, 30 Sep 2019 14:41:34 +0000 (16:41 +0200)]
bpo-38321: Fix _asynciomodule.c compiler warning (GH-16493)

bpo-38248, bpo-38321: Fix warning:

    modules\_asynciomodule.c(2667):
    warning C4102: 'set_exception': unreferenced label

The related goto has been removed by
commit edad4d89e357c92f70c0324b937845d652b20afd.

4 years agobpo-38322: Fix gotlandmark() of PC/getpathp.c (GH-16489)
Victor Stinner [Mon, 30 Sep 2019 12:49:34 +0000 (14:49 +0200)]
bpo-38322: Fix gotlandmark() of PC/getpathp.c (GH-16489)

Write the filename into a temporary buffer instead of reusing prefix.
The problem is that join() modifies prefix inplace. If prefix is not
normalized, join() can make prefix shorter and so gotlandmark()
does modify prefix instead of returning it unmodified.

4 years agobpo-38304: Fix PyConfig usage in python_uwp.cpp (GH-16487)
Victor Stinner [Mon, 30 Sep 2019 10:52:29 +0000 (12:52 +0200)]
bpo-38304: Fix PyConfig usage in python_uwp.cpp (GH-16487)

* Set PyPreConfig.struct_size and PyConfig.struct_size as required by
  the API.
* PyPreConfig_InitPythonConfig() can now fail: check PyStatus result.

4 years agobpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel (#7468)
Yury Selivanov [Mon, 30 Sep 2019 05:59:11 +0000 (22:59 -0700)]
bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel (#7468)

4 years agobpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
Yury Selivanov [Mon, 30 Sep 2019 04:59:55 +0000 (21:59 -0700)]
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)

See https://bugs.python.org/issue38242 for more details

4 years agobpo-38163: Child mocks detect their type as sync or async (GH-16471)
Lisa Roach [Mon, 30 Sep 2019 04:56:47 +0000 (21:56 -0700)]
bpo-38163: Child mocks detect their type as sync or async (GH-16471)

4 years agobpo-37096: Add large-file tests for modules using sendfile(2) (GH-13676)
Giampaolo Rodola [Mon, 30 Sep 2019 04:51:55 +0000 (12:51 +0800)]
bpo-37096: Add large-file tests for modules using sendfile(2) (GH-13676)

4 years agobpo-38161: Removes _AwaitEvent from AsyncMock. (GH-16443)
Lisa Roach [Mon, 30 Sep 2019 04:01:28 +0000 (21:01 -0700)]
bpo-38161: Removes _AwaitEvent from AsyncMock. (GH-16443)

4 years agobpo-38317: Fix PyConfig.warnoptions priority (GH-16478)
Victor Stinner [Sun, 29 Sep 2019 23:40:17 +0000 (01:40 +0200)]
bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)

Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.

* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
  with "_Py".
* test_embed: add test_init_warnoptions().

4 years agobpo-38019: correctly handle pause/resume reading of closed asyncio unix pipe (GH...
Andrew Svetlov [Sun, 29 Sep 2019 12:00:35 +0000 (15:00 +0300)]
bpo-38019: correctly handle pause/resume reading of closed asyncio unix pipe (GH-16472)

4 years agobpo-38108: Makes mock objects inherit from Base (GH-16060)
Lisa Roach [Sun, 29 Sep 2019 01:42:44 +0000 (18:42 -0700)]
bpo-38108: Makes mock objects inherit from Base (GH-16060)

4 years agobpo-38310: Predict BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX. (GH-16467)
Brandt Bucher [Sun, 29 Sep 2019 00:12:49 +0000 (17:12 -0700)]
bpo-38310: Predict BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX. (GH-16467)

4 years agobpo-37408: Precise that Tarfile "format" argument only concerns writing. (GH-14389)
Pascal Chambon [Sat, 28 Sep 2019 15:04:44 +0000 (17:04 +0200)]
bpo-37408: Precise that Tarfile "format" argument only concerns writing. (GH-14389)

4 years agobpo-38115: Deal with invalid bytecode offsets in lnotab (GH-16079)
T. Wouters [Sat, 28 Sep 2019 14:49:15 +0000 (16:49 +0200)]
bpo-38115: Deal with invalid bytecode offsets in lnotab (GH-16079)

Document that lnotab can contain invalid bytecode offsets (because of
terrible reasons that are difficult to fix). Make dis.findlinestarts()
ignore invalid offsets in lnotab. All other uses of lnotab in CPython
(various reimplementations of addr2line or line2addr in Python, C and gdb)
already ignore this, because they take an address to look for, instead.

Add tests for the result of dis.findlinestarts() on wacky constructs in
test_peepholer.py, because it's the easiest place to add them.

4 years agobpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding...
Jason R. Coombs [Sat, 28 Sep 2019 12:32:01 +0000 (08:32 -0400)]
bpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding behavior (GH-16448)

* bpo-38216: Allow bypassing input validation

* bpo-36274: Also allow the URL encoding to be overridden.

* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.

* Call with skip_host to avoid tripping on the host checking in the URL.

* Remove obsolete comment.

* Make _prepare_path_encoding its own attr.

This makes overriding just that simpler.

Also, don't use the := operator to make backporting easier.

* Add a news entry.

* _prepare_path_encoding -> _encode_prepared_path()

* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.

4 years agobpo-38304: Add PyConfig.struct_size (GH-16451)
Victor Stinner [Sat, 28 Sep 2019 02:28:35 +0000 (04:28 +0200)]
bpo-38304: Add PyConfig.struct_size (GH-16451)

Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.

* Replace private _config_version field with public struct_size field
  in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
  PyPreConfig_InitPythonConfig()
  return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
  _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
  _PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
  instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.