Victor Stinner [Thu, 25 Apr 2019 18:13:10 +0000 (20:13 +0200)]
bpo-21536: C extensions are no longer linked to libpython (GH-12946)
On Unix, C extensions are no longer linked to libpython.
It is now possible to load a C extension built using a shared library
Python with a statically linked Python.
When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.
distutils, python-config and python-config.py have been modified.
Victor Stinner [Thu, 25 Apr 2019 12:30:16 +0000 (14:30 +0200)]
bpo-35537: Document posix_spawn() change in subprocess (GH-11668)
Document that subprocess.Popen no longer raise an exception on error
like missing program on very specific platforms when using
os.posix_spawn() is used.
Victor Stinner [Wed, 24 Apr 2019 22:56:28 +0000 (00:56 +0200)]
bpo-36465: Make release and debug ABI compatible (GH-12615)
Release build and debug build are now ABI compatible: the Py_DEBUG
define no longer implies Py_TRACE_REFS define which introduces the
only ABI incompatibility.
A new "./configure --with-trace-refs" build option is now required to
get Py_TRACE_REFS define which adds sys.getobjects() function and
PYTHONDUMPREFS environment variable.
Changes:
* Add ./configure --with-trace-refs
* Py_DEBUG no longer implies Py_TRACE_REFS
Victor Stinner [Wed, 24 Apr 2019 16:23:53 +0000 (18:23 +0200)]
bpo-36710: Add runtime variable to Py_InitializeEx() (GH-12939)
Py_InitializeEx() now uses a runtime variable passed to subfunctions,
rather than working directly on the global variable _PyRuntime.
Add 'runtime' parameter to _PyCoreConfig_Write(), _PySys_Create(),
_PySys_InitMain(), _PyGILState_Init(),
emit_stderr_warning_for_legacy_locale() and other subfunctions.
Victor Stinner [Wed, 24 Apr 2019 15:24:01 +0000 (17:24 +0200)]
bpo-36710: Add runtime variable to Py_FinalizeEx() (GH-12937)
* Add a 'runtime' variable to Py_FinalizeEx() rather than working
directly on the global variable _PyRuntime
* Add a 'runtime' parameter to _PyGC_Fini(), _PyGILState_Fini()
and call_ll_exitfuncs()
The PyOS_AfterFork_Child() function now pass a 'runtime' parameter to
subfunctions.
* Fix _PyRuntimeState_ReInitThreads(): use the correct memory allocator
* Add runtime parameter to _PyRuntimeState_ReInitThreads(),
_PyGILState_Reinit() and _PyInterpreterState_DeleteExceptMain()
* Move _PyGILState_Reinit() to the internal C API.
Victor Stinner [Wed, 24 Apr 2019 14:47:40 +0000 (16:47 +0200)]
bpo-36710: Add runtime parameter to _PyThreadState_Init() (GH-12935)
* Add 'runtime' parameter to _PyThreadState_Init()
* Add 'gilstate' parameter to _PyGILState_NoteThreadState()
* Move _PyThreadState_Init() and _PyThreadState_DeleteExcept()
to the internal C API.
Change test_time.test_monotonic() to test only the lower bound of elapsed time
after a sleep command rather than the upper bound. This prevents unnecessary
test failures on slow buildbots. Patch by Victor Stinner.
Stefan Behnel [Thu, 18 Apr 2019 17:05:03 +0000 (19:05 +0200)]
bpo-30485: Change the prefix for defining the default namespace in ElementPath from None to '' since there is existing code that uses that and it's more convenient to have an all-string-keys dict (e.g. when sorting items etc.). (#12860)
In the documentation for the call_later and the call_at methods there is a note which says that the delay cannot be longer than a day, but both methods have a note saying that this limitation was removed in Python 3.8
Here I fixed this issue by removing the pre-exising note and added a versionchanged.
To test my changes I have rebuilt the documentation with ```make html```. I did not have any errors and the effected page displayed correctly on a browser.
Victor Stinner [Wed, 17 Apr 2019 21:02:26 +0000 (23:02 +0200)]
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.
The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.
Changes:
* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
defined, just to ensure that Python is build with the correct
defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
shutil.which() and distutils.spawn.find_executable() now use
os.confstr("CS_PATH") if available instead of os.defpath, if the PATH
environment variable is not set.
Don't use os.confstr("CS_PATH") nor os.defpath if the PATH
environment variable is set to an empty string to mimick Unix 'which'
command behavior.
Changes:
* find_executable() now starts by checking for the executable in the
current working directly case. Add an explicit
"if not path: return None".
* Add tests for PATH='' (empty string), PATH=':' and for PATHEXT.
The imap.IMAP4.logout() method no longer ignores silently arbitrary
exceptions.
Changes:
* The IMAP4.logout() method now expects a "BYE" untagged response,
rather than relying on _check_bye() which raises a self.abort()
exception.
* IMAP4.__exit__() now does nothing if the client already logged out.
* Add more debug info if test_logout() tests fail.
Stefan Behnel [Sun, 14 Apr 2019 19:12:34 +0000 (21:12 +0200)]
bpo-30485: Re-allow empty strings in ElementPath namespace mappings since they might actually be harmless and unused (and thus went undetected previously). (#12830)
bpo-36227: ElementTree.tostring() default_namespace and xml_declaration arguments (GH-12225)
Add new keyword arguments "default_namespace" and "xml_declaration" to functions ET.tostring() and ET.tostringlist(), as known from ElementTree.write().
bpo-36593: Fix isinstance check for Mock objects with spec executed under tracing (GH-12790)
In Python having a trace function in effect while mock is imported causes isinstance to be wrong for MagicMocks. This is due to the usage of super() in some class methods, as this sets the __class__ attribute. To avoid this, as a workaround, alias the usage of super .
Pablo Galindo [Sat, 13 Apr 2019 16:23:24 +0000 (17:23 +0100)]
bpo-36427: Document that PyEval_RestoreThread and PyGILState_Ensure can terminate the calling thread (GH-12541)
Calling these function from a thread when the runtime is finalizing will terminate
the thread, even if the thread was not created by Python. Users can use
_Py_IsFinalizing or sys.is_finalizing to check if the interpreter is in the process of
being finalized before calling this function to avoid unwanted termination.
Victor Stinner [Fri, 12 Apr 2019 19:54:06 +0000 (21:54 +0200)]
bpo-36611: Disable serialno field of debug memory allocators (#12796)
Omit serialno field from debug hooks on Python memory allocators to
reduce the memory footprint by 5%.
Enable tracemalloc to get the traceback where a memory block has been
allocated when a fatal memory error is logged to decide where to put
a breakpoint.
Compile Python with PYMEM_DEBUG_SERIALNO defined to get back the
field.
Victor Stinner [Fri, 12 Apr 2019 19:27:37 +0000 (21:27 +0200)]
bpo-36618: Add -fmax-type-align=8 flag for clang (GH-12809)
Add -fmax-type-align=8 to CFLAGS when clang compiler is detected.
The pymalloc memory allocator aligns memory on 8 bytes. On x86-64,
clang expects alignment on 16 bytes by default and so uses MOVAPS
instruction which can lead to segmentation fault. Instruct clang that
Python is limited to alignemnt on 8 bytes to use MOVUPS instruction
instead: slower but don't trigger a SIGSEGV if the memory is not
aligned on 16 bytes.
Sadly, the flag must be expected to CFLAGS and not just
CFLAGS_NODIST, since third party C extensions can have the same
issue.