bpo-21536: C extensions are no longer linked to libpython (GH-12946)

On Unix, C extensions are no longer linked to libpython.

It is now possible to load a C extension built using a shared library
Python with a statically linked Python.

When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.

distutils, python-config and python-config.py have been modified.

diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst
index a825efc1a67243cecdbb4ceeca2d62fad5c8c64a..c3cdfc8a0a8ec67430a34a53a052832371e167d0 100644 (file)
--- a/Doc/distutils/apiref.rst
+++ b/Doc/distutils/apiref.rst
@@ -277,6 +277,10 @@ the full reference.
    |                        | simply skip the extension.     |                           |
    +------------------------+--------------------------------+---------------------------+
 
+   .. versionchanged:: 3.8
+
+      On Unix, C extensions are no longer linked to libpython.
+
 
 .. class:: Distribution
 

diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst
index 344656b9288f6526f0d475102eaf997d3e44e8fe..2270334a281b0874725988fbb3f8db4b18240ba2 100644 (file)
--- a/Doc/whatsnew/3.8.rst
+++ b/Doc/whatsnew/3.8.rst
@@ -851,16 +851,19 @@ Changes in the Python API
 Changes in the C API
 --------------------
 
+* On Unix, C extensions are no longer linked to libpython. When Python is
+  embedded, ``libpython`` must not be loaded with ``RTLD_LOCAL``, but
+  ``RTLD_GLOBAL`` instead. Previously, using ``RTLD_LOCAL``, it was already not
+  possible to load C extensions which were not linked to ``libpython``, like C
+  extensions of the standard library built by the ``*shared*`` section of
+  ``Modules/Setup``.
+
 * Use of ``#`` variants of formats in parsing or building value (e.g.
   :c:func:`PyArg_ParseTuple`, :c:func:`Py_BuildValue`, :c:func:`PyObject_CallFunction`,
   etc.) without ``PY_SSIZE_T_CLEAN`` defined raises ``DeprecationWarning`` now.
   It will be removed in 3.10 or 4.0.  Read :ref:`arg-parsing` for detail.
   (Contributed by Inada Naoki in :issue:`36381`.)
 
-
-Changes in the C API
---------------------------
-
 * Instances of heap-allocated types (such as those created with
   :c:func:`PyType_FromSpec`) hold a reference to their type object.
   Increasing the reference count of these type objects has been moved from

diff --git a/Lib/distutils/command/build_ext.py b/Lib/distutils/command/build_ext.py
index 0428466b00c902f8a0f2e5d655f1cb0352b932b4..1672d02acf1f7a17496d45ab4f80c09844d6956c 100644 (file)
--- a/Lib/distutils/command/build_ext.py
+++ b/Lib/distutils/command/build_ext.py
@@ -714,20 +714,5 @@ class build_ext(Command):
                 # don't extend ext.libraries, it may be shared with other
                 # extensions, it is a reference to the original list
                 return ext.libraries + [pythonlib]
-            else:
-                return ext.libraries
-        elif sys.platform == 'darwin':
-            # Don't use the default code below
-            return ext.libraries
-        elif sys.platform[:3] == 'aix':
-            # Don't use the default code below
-            return ext.libraries
-        else:
-            from distutils import sysconfig
-            if sysconfig.get_config_var('Py_ENABLE_SHARED'):
-                pythonlib = 'python{}.{}{}'.format(
-                    sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff,
-                    sysconfig.get_config_var('ABIFLAGS'))
-                return ext.libraries + [pythonlib]
-            else:
-                return ext.libraries
+
+        return ext.libraries

diff --git a/Makefile.pre.in b/Makefile.pre.in
index cd7098cac72b1d10963d46e699ab1bd6d64aac9c..68ac7723556e4e980f743ec2af2fba57c5a78e95 100644 (file)
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1460,7 +1460,7 @@ libinstall:       build_all $(srcdir)/Modules/xxmodule.c
        -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
                $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt
 
-python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
+python-config: $(srcdir)/Misc/python-config.in $(srcdir)/Misc/python-config.sh
        @ # Substitution happens here, as the completely-expanded BINDIR
        @ # is not available in configure
        sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py

diff --git a/Misc/NEWS.d/next/Build/2019-04-25-01-51-52.bpo-21536.ACQkiC.rst b/Misc/NEWS.d/next/Build/2019-04-25-01-51-52.bpo-21536.ACQkiC.rst
new file mode 100644 (file)
index 0000000..5e1e717
--- /dev/null
+++ b/Misc/NEWS.d/next/Build/2019-04-25-01-51-52.bpo-21536.ACQkiC.rst
@@ -0,0 +1,12 @@
+On Unix, C extensions are no longer linked to libpython.
+
+It is now possible to load a C extension built using a shared library Python
+with a statically linked Python.
+
+When Python is embedded, ``libpython`` must not be loaded with ``RTLD_LOCAL``,
+but ``RTLD_GLOBAL`` instead. Previously, using ``RTLD_LOCAL``, it was already
+not possible to load C extensions which were not linked to ``libpython``, like
+C extensions of the standard library built by the ``*shared*`` section of
+``Modules/Setup``.
+
+distutils, python-config and python-config.py have been modified.

diff --git a/Misc/python-config.in b/Misc/python-config.in
index 714415222798d65e925134c628d6013344557cb5..31ad55822e55a26e7381486f169c44e88fd86e93 100644 (file)
--- a/Misc/python-config.in
+++ b/Misc/python-config.in
@@ -47,9 +47,7 @@ for opt in opt_flags:
         print(' '.join(flags))
 
     elif opt in ('--libs', '--ldflags'):
-        libs = ['-lpython' + pyver + sys.abiflags]
-        libs += getvar('LIBS').split()
-        libs += getvar('SYSLIBS').split()
+        libs = getvar('LIBS').split() + getvar('SYSLIBS').split()
         # add the prefix/lib/pythonX.Y/config dir, but only if there is no
         # shared library in prefix/lib/.
         if opt == '--ldflags':

diff --git a/Misc/python-config.sh.in b/Misc/python-config.sh.in
index a3c479ce571f87b1307b58b34bf1b99b465304ef..ac1a467678e44ba4a7b76cd3fdc4a538869a2bc3 100644 (file)
--- a/Misc/python-config.sh.in
+++ b/Misc/python-config.sh.in
@@ -41,7 +41,7 @@ LIBM="@LIBM@"
 LIBC="@LIBC@"
 SYSLIBS="$LIBM $LIBC"
 ABIFLAGS="@ABIFLAGS@"
-LIBS="-lpython${VERSION}${ABIFLAGS} @LIBS@ $SYSLIBS"
+LIBS="@LIBS@ $SYSLIBS"
 BASECFLAGS="@BASECFLAGS@"
 LDLIBRARY="@LDLIBRARY@"
 OPT="@OPT@"