latex_documents = [
('c-api/index', 'c-api.tex',
'The Python/C API', _stdauthor, 'manual'),
- ('distutils/index', 'distutils.tex',
+ ('distributing/index', 'distributing.tex',
'Distributing Python Modules', _stdauthor, 'manual'),
('extending/index', 'extending.tex',
'Extending and Embedding Python', _stdauthor, 'manual'),
- ('install/index', 'install.tex',
+ ('installing/index', 'installing.tex',
'Installing Python Modules', _stdauthor, 'manual'),
('library/index', 'library.tex',
'The Python Library Reference', _stdauthor, 'manual'),
library/index.rst
extending/index.rst
c-api/index.rst
- distutils/index.rst
- install/index.rst
+ distributing/index.rst
+ installing/index.rst
howto/index.rst
faq/index.rst
glossary.rst
bugs.rst
copyright.rst
license.rst
+
+.. include legacy packaging docs in build
+
+.. toctree::
+ :hidden:
+
+ distutils/index.rst
+ install/index.rst
--- /dev/null
+.. _distributing-index:
+
+###############################
+ Distributing Python Modules
+###############################
+
+:Email: distutils-sig@python.org
+
+
+As a popular open source development project, Python has an active
+supporting community of contributors and users that also make their software
+available for other Python developers to use under open source license terms.
+
+This allows Python users to share and collaborate effectively, benefiting
+from the solutions others have already created to common (and sometimes
+even rare!) problems, as well as potentially contributing their own
+solutions to the common pool.
+
+This guide covers the distribution part of the process. For a guide to
+installing other Python projects, refer to the
+:ref:`installation guide <installing-index>`.
+
+.. note::
+
+ For corporate and other institutional users, be aware that many
+ organisations have their own policies around using and contributing to
+ open source software. Please take such policies into account when making
+ use of the distribution and installation tools provided with Python.
+
+
+Key terms
+=========
+
+* the `Python Package Index <https://pypi.python.org/pypi>`__ is a public
+ repository of open source licensed packages made available for use by
+ other Python users
+* the `Python Packaging Authority
+ <http://packaging.python.org/en/latest/future.html>`__ are the group of
+ developers and documentation authors responsible for the maintenance and
+ evolution of the standard packaging tools and the associated metadata and
+ file format standards. They maintain a variety of tools, documentation
+ and issue trackers on both `GitHub <https://github.com/pypa>` and
+ `BitBucket <https://bitbucket.org/pypa/>`__.
+* ``distutils`` is the original build and distribution system first added to
+ the Python standard library in 1998. While direct use of ``distutils`` is
+ being phased out, it still laid the foundation for the current packaging
+ and distribution infrastructure, and it not only remains part of the
+ standard library, but its name lives on in other ways (such as the name
+ of the mailing list used to coordinate Python packaging standards
+ development).
+
+
+Open source licensing and collaboration
+=======================================
+
+In most parts of the world, software is automatically covered by copyright.
+This means that other developers require explicit permission to copy, use,
+modify and redistribute the software.
+
+Open source licensing is a way of explicitly granting such permission in a
+relatively consistent way, allowing developers to share and collaborate
+efficiently by making common solutions to various problems freely available.
+This leaves many developers free to spend more time focusing on the problems
+that are relatively unique to their specific situation.
+
+The distribution tools provided with Python are designed to make it
+reasonably straightforward for developers to make their own contributions
+back to that common pool of software if they choose to do so.
+
+The same distribution tools can also be used to distribute software within
+an organisation, regardless of whether that software is published as open
+source software or not.
+
+
+Installing the tools
+====================
+
+The standard library does not include build tools that support modern
+Python packaging standards, as the core development team has found that it
+is important to have standard tools that work consistently, even on older
+versions of Python.
+
+The currently recommended build and distribution tools can be installed
+using ``pip``::
+
+ pip install setuptools wheel twine
+
+
+Reading the guide
+=================
+
+The Python Packaging User Guide covers the various key steps and elements
+involved in creating a project
+
+* `Project structure`_
+* `Building and packaging the project`_
+* `Uploading the project to the Python Package Index`_
+
+.. _Project structure: \
+ http://packaging.python.org/en/latest/tutorial.html#creating-your-own-project
+.. _Building and packaging the project: \
+ http://packaging.python.org/en/latest/tutorial.html#building-packaging-your-project
+.. _Uploading the project to the Python Package Index: \
+ http://packaging.python.org/en/latest/tutorial.html#building-uploading-your-project-to-pypi
+
+
+How do I...?
+============
+
+These are quick answers or links for some common tasks.
+
+... choose a name for my project?
+---------------------------------
+
+This isn't an easy topic, but here are a few tips:
+
+* check the Python Package Index to see if the name is already in use
+* check popular hosting sites like GitHub, BitBucket, etc to see if there
+ is already a project with that name
+* check what comes up in a web search for the name you're considering
+* avoid particularly common words, especially ones with multiple meanings,
+ as they can make it difficult for users to find your software when
+ searching for it
+
+
+... create and distribute binary extensions?
+--------------------------------------------
+
+This is actually quite a complex topic, with a variety of alternatives
+available depending on exactly what you're aiming to achieve. See the
+Python Packaging User Guide for more information and recommendations.
+
+.. seealso::
+
+ `Python Packaging User Guide: Binary Extensions
+ <http://packaging.python.org/en/latest/extensions.html>`__
+
+.. other topics:
+
+ Once the Development & Deployment part of PPUG is fleshed out, some of
+ those sections should be linked from new questions here (most notably,
+ we should have a question about avoiding depending on PyPI that links to
+ http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)
.. _distutils-index:
-###############################
- Distributing Python Modules
-###############################
+##############################################
+ Distributing Python Modules (Legacy version)
+##############################################
:Authors: Greg Ward, Anthony Baxter
:Email: distutils-sig@python.org
For a detailed description of the whole Python/C API, see the separate
:ref:`c-api-index`.
-.. note::
- This guide only covers the basic tools for creating extensions provided
- as part of this version of CPython. Third party tools may offer simpler
- alternatives. Refer to the `binary extensions section
- <https://python-packaging-user-guide.readthedocs.org/en/latest/extensions.html>`__
- in the Python Packaging User Guide for more information.
+Recommended third party tools
+=============================
+This guide only covers the basic tools for creating extensions provided
+as part of this version of CPython. Third party tools like Cython,
+``cffi``, SWIG and Numba offer both simpler and more sophisticated
+approaches to creating C and C++ extensions for Python.
+
+.. seealso::
+
+ `Python Packaging User Guide: Binary Extensions <https://packaging.python.org/en/latest/extensions.html>`_
+ The Python Packaging User Guide not only covers several available
+ tools that simplify the creation of binary extensions, but also
+ discusses the various reasons why creating an extension module may be
+ desirable in the first place.
+
+
+Creating extensions without third party tools
+=============================================
+
+This section of the guide covers creating C and C++ extensions without
+assistance from third party tools. It is intended primarily for creators
+of those tools, rather than being a recommended way to create your own
+C extensions.
.. toctree::
:maxdepth: 2
newtypes.rst
building.rst
windows.rst
+
+Embedding the CPython runtime in a larger application
+=====================================================
+
+Sometimes, rather than creating an extension that runs inside the Python
+interpreter as the main application, it is desirable to instead embed
+the CPython runtime inside a larger application. This section covers
+some of the details involved in doing that successfully.
+
+.. toctree::
+ :maxdepth: 2
+ :numbered:
+
embedding.rst
.. _install-index:
-*****************************
- Installing Python Modules
-*****************************
+********************************************
+ Installing Python Modules (Legacy version)
+********************************************
:Author: Greg Ward
--- /dev/null
+.. highlightlang:: none
+
+.. _installing-index:
+
+*****************************
+ Installing Python Modules
+*****************************
+
+:Email: distutils-sig@python.org
+
+As a popular open source development project, Python has an active
+supporting community of contributors and users that also make their software
+available for other Python developers to use under open source license terms.
+
+This allows Python users to share and collaborate effectively, benefiting
+from the solutions others have already created to common (and sometimes
+even rare!) problems, as well as potentially contributing their own
+solutions to the common pool.
+
+This guide covers the installation part of the process. For a guide to
+creating and sharing your own Python projects, refer to the
+:ref:`distribution guide <distributing-index>`.
+
+.. note::
+
+ For corporate and other institutional users, be aware that many
+ organisations have their own policies around using and contributing to
+ open source software. Please take such policies into account when making
+ use of the distribution and installation tools provided with Python.
+
+
+Key terms
+=========
+
+* ``pip`` is the preferred installer program. Starting with Python 3.4, it
+ is included by default with the Python binary installers.
+ installed into virtual environments
+* a virtual environment is a semi-isolated Python environment that allows
+ packages to be installed for use by a particular application, rather than
+ being installed system wide
+* ``pyvenv`` is the standard tool for creating virtual environments, and has
+ been part of Python since Python 3.3. Starting with Python 3.4, it
+ defaults to installing ``pip`` into all created virtual environments
+* the `Python Package Index <https://pypi.python.org/pypi>`__ is a public
+ repository of open source licensed packages made available for use by
+ other Python users
+* the `Python Packaging Authority
+ <http://packaging.python.org/en/latest/future.html>`__ are the group of
+ developers and documentation authors responsible for the maintenance and
+ evolution of the standard packaging tools and the associated metadata and
+ file format standards. They maintain a variety of tools, documentation
+ and issue trackers on both `GitHub <https://github.com/pypa>` and
+ `BitBucket <https://bitbucket.org/pypa/>`__.
+* ``distutils`` is the original build and distribution system first added to
+ the Python standard library in 1998. While direct use of ``distutils`` is
+ being phased out, it still laid the foundation for the current packaging
+ and distribution infrastructure, and it not only remains part of the
+ standard library, but its name lives on in other ways (such as the name
+ of the mailing list used to coordinate Python packaging standards
+ development).
+
+
+Basic usage
+===========
+
+The standard packaging tools are all designed to be used from the command
+line. For Windows users, the examples below assume that the option to
+adjust the system PATH environment variable was selected when installing
+Python. For Linux users, the command to install into the system version of
+Python 3 is likely to be ``pip3`` rather than ``pip``.
+
+The following command will install the latest version of a module and its
+dependencies from the Python Package Index::
+
+ pip install SomePackage
+
+It's also possible to specify an exact or minimum version directly on the
+command line::
+
+ pip install SomePackage==1.0.4 # specific version
+ pip install 'SomePackage>=1.0.4' # minimum version
+
+Normally, if a suitable module is already installed, attempting to install
+it again will have no effect. Upgrading existing modules must be requested
+explicitly::
+
+ pip install --upgrade SomePackage
+
+More information and resources regarding ``pip`` and its capabilities can be
+found in the `Python Packaging User Guide <http://packaging.python.org>`__.
+
+``pyvenv`` has its own documentation at :ref:`scripts-pyvenv`. Installing
+into an active virtual environment uses the commands shown above.
+
+.. seealso::
+
+ `Python Packaging User Guide: Installing Python packages
+ <http://packaging.python.org/en/latest/tutorial.html#installing-python-packages>`__
+
+
+How do I ...?
+=============
+
+These are quick answers or links for some common tasks.
+
+... install ``pip`` in versions of Python prior to Python 3.4?
+--------------------------------------------------------------
+
+Python only started bundling ``pip`` with Python 3.4. For earlier versions,
+``pip`` needs to be "bootstrapped" as described in the Python Packaging
+User Guide.
+
+.. seealso::
+
+ `Python Packaging User Guide: Installing the Tools
+ <http://packaging.python.org/en/latest/tutorial.html#installing-the-tools>`__
+
+
+.. installing-per-user-installation:
+
+... install packages just for the current user?
+-----------------------------------------------
+
+Passing the ``--user`` option to ``pip install`` will install a package
+just for the current user, rather than for all users of the system.
+
+
+... install scientific Python packages?
+---------------------------------------
+
+A number of scientific Python packages have complex binary dependencies, and
+aren't currently easy to install using ``pip`` directly. At this point in
+time, it will often be easier for users to install these packages by
+`other means
+<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
+rather than attempting to install them with ``pip``.
+
+.. seealso::
+
+ `Python Packaging User Guide: Installing Scientific Packages
+ <http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
+
+
+... work with multiple versions of Python installed in parallel?
+----------------------------------------------------------------
+
+On Linux, Mac OS X and other POSIX systems, use the versioned Python commands
+in combination with the ``-m`` switch to run the appropriate copy of
+``pip``::
+
+ python2 -m pip install SomePackage # default Python 2
+ python2.7 -m pip install SomePackage # specifically Python 2.7
+ python3 -m pip install SomePackage # default Python 3
+ python3.4 -m pip install SomePackage # specifically Python 3.4
+
+(appropriately versioned ``pip`` commands may also be available)
+
+On Windows, use the ``py`` Python launcher in combination with the ``-m``
+switch::
+
+ py -2 -m pip install SomePackage # default Python 2
+ py -2.7 -m pip install SomePackage # specifically Python 2.7
+ py -3 -m pip install SomePackage # default Python 3
+ py -3.4 -m pip install SomePackage # specifically Python 3.4
+
+.. other questions:
+
+ Once the Development & Deployment part of PPUG is fleshed out, some of
+ those sections should be linked from new questions here (most notably,
+ we should have a question about avoiding depending on PyPI that links to
+ http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)
+
+
+Common installation issues
+==========================
+
+Installing into the system Python on Linux
+------------------------------------------
+
+On Linux systems, a Python installation will typically be included as part
+of the distribution. Installing into this Python installation requires
+root access to the system, and may interfere with the operation of the
+system package manager and other components of the system if a component
+is unexpectedly upgraded using ``pip``.
+
+On such systems, it is often better to use a virtual environment or a
+per-user installation when installing packages with ``pip``.
+
+
+Installing binary extensions
+----------------------------
+
+Python has typically relied heavily on source based distribution, with end
+users being expected to compile extension modules from source as part of
+the installation process.
+
+With the introduction of support for the binary ``wheel`` format, and the
+ability to publish wheels for at least Windows and Mac OS X through the
+Python Package Index, this problem is expected to diminish over time,
+as users are more regularly able to install pre-built extensions rather
+than needing to build them themselves.
+
+Some of the solutions for installing `scientific software
+<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
+that is not yet available as pre-built ``wheel`` files may also help with
+obtaining other binary extensions without needing to build them locally.
+
+.. seealso::
+
+ `Python Packaging User Guide: Binary Extensions
+ <http://packaging.python.org/en/latest/extensions.html>`__
100%-pure Python, or may be extension modules written in C, or may be
collections of Python packages which include modules coded in both Python and C.
+Most Python users will *not* want to use this module directly, but instead
+use the cross-version tools maintained by the Python Packaging Authority.
+Refer to the `Python Packaging User Guide <http://packaging.python.org>`_
+for more information.
-User documentation and API reference are provided in another document:
+For the benefits of packaging tool authors and users seeking a deeper
+understanding of the details of the current packaging and distribution
+system, the legacy :mod:`distutils` based user documentation and API
+reference remain available:
-.. seealso::
-
- :ref:`distutils-index`
- The manual for developers and packagers of Python modules. This describes
- how to prepare :mod:`distutils`\ -based packages so that they may be
- easily installed into an existing Python installation. It also contains
- instructions for end-users wanting to install a distutils-based package,
- :ref:`install-index`.
+* :ref:`install-index`
+* :ref:`distutils-index`
.. seealso::
- :ref:`install-index`
+ :ref:`installing-index`
The end user guide for installing Python packages
:pep:`453`: Explicit bootstrapping of pip in Python installations
<p class="biglink"><a class="biglink" href="{{ pathto("howto/index") }}">Python HOWTOs</a><br/>
<span class="linkdescr">in-depth documents on specific topics</span></p>
</td><td width="50%">
+ <p class="biglink"><a class="biglink" href="{{ pathto("installing/index") }}">Installing Python Modules</a><br/>
+ <span class="linkdescr">installing from the Python Package Index & other sources</span></p>
+ <p class="biglink"><a class="biglink" href="{{ pathto("distributing/index") }}">Distributing Python Modules</a><br/>
+ <span class="linkdescr">publishing modules for installation by others</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("extending/index") }}">Extending and Embedding</a><br/>
<span class="linkdescr">tutorial for C/C++ programmers</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("c-api/index") }}">Python/C API</a><br/>
<span class="linkdescr">reference for C/C++ programmers</span></p>
- <p class="biglink"><a class="biglink" href="{{ pathto("install/index") }}">Installing Python Modules</a><br/>
- <span class="linkdescr">information for installers & sys-admins</span></p>
- <p class="biglink"><a class="biglink" href="{{ pathto("distutils/index") }}">Distributing Python Modules</a><br/>
- <span class="linkdescr">sharing modules with others</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("faq/index") }}">FAQs</a><br/>
<span class="linkdescr">frequently asked questions (with answers!)</span></p>
</td></tr>
and many other tasks. Skimming through the Library Reference will give you an
idea of what's available.
-* :ref:`install-index` explains how to install external modules written by other
- Python users.
+* :ref:`installing-index` explains how to install additional modules written
+ by other Python users.
* :ref:`reference-index`: A detailed explanation of Python's syntax and
semantics. It's heavy reading, but is useful as a complete guide to the
Windows). It also creates an (initially empty) ``lib/pythonX.Y/site-packages``
subdirectory (on Windows, this is ``Lib\site-packages``).
+.. seealso::
+
+ `Python Packaging User Guide: Creating and using virtual environments
+ <http://packaging.python.org/en/latest/tutorial.html#creating-and-using-virtual-environments>`__
+
.. highlight:: none
On Windows, you may have to invoke the ``pyvenv`` script as follows, if you
This document aims to give an overview of Windows-specific behaviour you should
know about when using Python on Microsoft Windows.
+.. XXX (ncoghlan)
+
+ This looks rather stale to me...
+
Installing Python
=================
PEP 453: Explicit Bootstrapping of PIP in Python Installations
--------------------------------------------------------------
+Bootstrapping pip by default
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard
cross-platform mechanism to bootstrap the pip installer into Python
installations and virtual environments.
__ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors
+
+Documentation changes
+~~~~~~~~~~~~~~~~~~~~~
+
+As part of this change, the :ref:`installing-index` and
+:ref:`distributing-index` sections of the documentation have been
+completely redesigned as short getting started and FAQ documents. Most
+packaging documentation has now been moved out to the Python Packaging
+Authority maintained `Python Packaging User Guide
+<http://packaging.python.org>`__ and the documentation of the individual
+projects.
+
+However, as this migration is currently still incomplete, the legacy
+versions of those guides remaining available as :ref:`install-index`
+and :ref:`distutils-index`.
+
.. note::
To avoid conflicts between parallel Python 2 and Python 3 installations,
- Issue #20765: Add missing documentation for PurePath.with_name() and
PurePath.with_suffix().
+- Issue #19407: New package installation and distribution guides based on
+ the Python Packaging Authority tools. Existing guides have been retained
+ as legacy links from the distutils docs, as they still contain some
+ required reference material for tool developers that isn't recorded
+ anywhere else.
+
Tests
-----
projects / python / commitdiff