Issue 17538: Document XML vulnerabilties

diff --git a/Doc/library/markup.rst b/Doc/library/markup.rst
index 1b4cca51a741eb2a0f1bff096bf5ebfd529dab9f..ed24ba2ce343062fa24774df468c1ba0c2f27f61 100644 (file)
--- a/Doc/library/markup.rst
+++ b/Doc/library/markup.rst
@@ -23,6 +23,7 @@ definition of the Python bindings for the DOM and SAX interfaces.
    html.rst
    html.parser.rst
    html.entities.rst
+   xml.rst
    xml.etree.elementtree.rst
    xml.dom.rst
    xml.dom.minidom.rst

diff --git a/Doc/library/pyexpat.rst b/Doc/library/pyexpat.rst
index 861546c2e8752f06b7aea789d0394d214cb1b469..420e40766304f817671e634b73084598410000b6 100644 (file)
--- a/Doc/library/pyexpat.rst
+++ b/Doc/library/pyexpat.rst
@@ -14,6 +14,13 @@
    references to these attributes should be marked using the :member: role.
 
 
+.. warning::
+
+   The :mod:`pyexpat` module is not secure against maliciously
+   constructed data.  If you need to parse untrusted or unauthenticated data see
+   :ref:`xml-vulnerabilities`.
+
+
 .. index:: single: Expat
 
 The :mod:`xml.parsers.expat` module is a Python interface to the Expat

diff --git a/Doc/library/xml.dom.minidom.rst b/Doc/library/xml.dom.minidom.rst
index b512d7ee72f9635fc72e6c953bc459b12d13b626..30182e4e501728ad2fa8e6a26e55d42c77bff7bf 100644 (file)
--- a/Doc/library/xml.dom.minidom.rst
+++ b/Doc/library/xml.dom.minidom.rst
@@ -17,6 +17,14 @@ to be simpler than the full DOM and also significantly smaller.  Users who are
 not already proficient with the DOM should consider using the
 :mod:`xml.etree.ElementTree` module for their XML processing instead
 
+
+.. warning::
+
+   The :mod:`xml.dom.minidom` module is not secure against
+   maliciously constructed data.  If you need to parse untrusted or
+   unauthenticated data see :ref:`xml-vulnerabilities`.
+
+
 DOM applications typically start by parsing some XML into a DOM.  With
 :mod:`xml.dom.minidom`, this is done through the parse functions::
 

diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst
index eb16a0933da761e73ea687249304721a732e78f3..8aa9cfb7d518fd332ed78edc1f7caba2460c2df5 100644 (file)
--- a/Doc/library/xml.dom.pulldom.rst
+++ b/Doc/library/xml.dom.pulldom.rst
@@ -17,6 +17,14 @@ processing model together with callbacks, the user of a pull parser is
 responsible for explicitly pulling events from the stream, looping over those
 events until either processing is finished or an error condition occurs.
 
+
+.. warning::
+
+   The :mod:`xml.dom.pulldom` module is not secure against
+   maliciously constructed data.  If you need to parse untrusted or
+   unauthenticated data see :ref:`xml-vulnerabilities`.
+
+
 Example::
 
    from xml.dom import pulldom

diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst
index cf0c33f55aabcfaa222fc095c9d7c15cb60faf75..dc9ebb999781a156ac4373dbe69908f966f58650 100644 (file)
--- a/Doc/library/xml.etree.elementtree.rst
+++ b/Doc/library/xml.etree.elementtree.rst
@@ -13,6 +13,14 @@ The :class:`Element` type is a flexible container object, designed to store
 hierarchical data structures in memory.  The type can be described as a cross
 between a list and a dictionary.
 
+
+.. warning::
+
+   The :mod:`xml.etree.ElementTree` module is not secure against
+   maliciously constructed data.  If you need to parse untrusted or
+   unauthenticated data see :ref:`xml-vulnerabilities`.
+
+
 Each element has a number of properties associated with it:
 
 * a tag which is a string identifying what kind of data this element represents

diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst
new file mode 100644 (file)
index 0000000..f84af58
--- /dev/null
+++ b/Doc/library/xml.rst
@@ -0,0 +1,131 @@
+.. _xml:
+
+XML Processing Modules
+======================
+
+.. module:: xml
+   :synopsis: Package containing XML processing modules
+.. sectionauthor:: Christian Heimes <christian@python.org>
+.. sectionauthor:: Georg Brandl <georg@python.org>
+
+
+Python's interfaces for processing XML are grouped in the ``xml`` package.
+
+.. warning::
+
+   The XML modules are not secure against erroneous or maliciously
+   constructed data.  If you need to parse untrusted or unauthenticated data see
+   :ref:`xml-vulnerabilities`.
+
+It is important to note that modules in the :mod:`xml` package require that
+there be at least one SAX-compliant XML parser available. The Expat parser is
+included with Python, so the :mod:`xml.parsers.expat` module will always be
+available.
+
+The documentation for the :mod:`xml.dom` and :mod:`xml.sax` packages are the
+definition of the Python bindings for the DOM and SAX interfaces.
+
+The XML handling submodules are:
+
+* :mod:`xml.etree.ElementTree`: the ElementTree API, a simple and lightweight
+
+..
+
+* :mod:`xml.dom`: the DOM API definition
+* :mod:`xml.dom.minidom`: a lightweight DOM implementation
+* :mod:`xml.dom.pulldom`: support for building partial DOM trees
+
+..
+
+* :mod:`xml.sax`: SAX2 base classes and convenience functions
+* :mod:`xml.parsers.expat`: the Expat parser binding
+
+
+.. _xml-vulnerabilities:
+
+XML vulnerabilities
+===================
+
+The XML processing modules are not secure against maliciously constructed data.
+An attacker can abuse vulnerabilities for e.g. denial of service attacks, to
+access local files, to generate network connections to other machines, or
+to or circumvent firewalls. The attacks on XML abuse unfamiliar features
+like inline `DTD`_ (document type definition) with entities.
+
+
+=========================  ========  =========  =========  ========  =========
+kind                       sax       etree      minidom    pulldom   xmlrpc
+=========================  ========  =========  =========  ========  =========
+billion laughs             **True**  **True**   **True**   **True**  **True**
+quadratic blowup           **True**  **True**   **True**   **True**  **True**
+external entity expansion  **True**  False (1)  False (2)  **True**  False (3)
+DTD retrieval              **True**  False      False      **True**  False
+decompression bomb         False     False      False      False     **True**
+=========================  ========  =========  =========  ========  =========
+
+1. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a
+   ParserError when an entity occurs.
+2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns
+   the unexpanded entity verbatim.
+3. :mod:`xmlrpclib` doesn't expand external entities and omits them.
+
+
+billion laughs / exponential entity expansion
+  The `Billion Laughs`_ attack -- also known as exponential entity expansion --
+  uses multiple levels of nested entities. Each entity refers to another entity
+  several times, the final entity definition contains a small string. Eventually
+  the small string is expanded to several gigabytes. The exponential expansion
+  consumes lots of CPU time, too.
+
+quadratic blowup entity expansion
+  A quadratic blowup attack is similar to a `Billion Laughs`_ attack; it abuses
+  entity expansion, too. Instead of nested entities it repeats one large entity
+  with a couple of thousand chars over and over again. The attack isn't as
+  efficient as the exponential case but it avoids triggering countermeasures of
+  parsers against heavily nested entities.
+
+external entity expansion
+  Entity declarations can contain more than just text for replacement. They can
+  also point to external resources by public identifiers or system identifiers.
+  System identifiers are standard URIs or can refer to local files. The XML
+  parser retrieves the resource with e.g. HTTP or FTP requests and embeds the
+  content into the XML document.
+
+DTD retrieval
+  Some XML libraries like Python's mod:'xml.dom.pulldom' retrieve document type
+  definitions from remote or local locations. The feature has similar
+  implications as the external entity expansion issue.
+
+decompression bomb
+  The issue of decompression bombs (aka `ZIP bomb`_) apply to all XML libraries
+  that can parse compressed XML stream like gzipped HTTP streams or LZMA-ed
+  files. For an attacker it can reduce the amount of transmitted data by three
+  magnitudes or more.
+
+The documentation of `defusedxml`_ on PyPI has further information about
+all known attack vectors with examples and references.
+
+defused packages
+----------------
+
+`defusedxml`_ is a pure Python package with modified subclasses of all stdlib
+XML parsers that prevent any potentially malicious operation. The courses of
+action are recommended for any server code that parses untrusted XML data. The
+package also ships with example exploits and an extended documentation on more
+XML exploits like xpath injection.
+
+`defusedexpat`_ provides a modified libexpat and patched replacment
+:mod:`pyexpat` extension module with countermeasures against entity expansion
+DoS attacks. Defusedexpat still allows a sane and configurable amount of entity
+expansions. The modifications will be merged into future releases of Python.
+
+The workarounds and modifications are not included in patch releases as they
+break backward compatibility. After all inline DTD and entity expansion are
+well-definied XML features.
+
+
+.. _defusedxml: <https://pypi.python.org/pypi/defusedxml/>
+.. _defusedexpat: <https://pypi.python.org/pypi/defusedexpat/>
+.. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs
+.. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb
+.. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition

diff --git a/Doc/library/xml.sax.rst b/Doc/library/xml.sax.rst
index 1bf55b4ee034c6040354d4028b295628eeba6958..d5c56b6966ba5ec2abcc77f1c5aa988f32bb8ed3 100644 (file)
--- a/Doc/library/xml.sax.rst
+++ b/Doc/library/xml.sax.rst
@@ -13,6 +13,14 @@ Simple API for XML (SAX) interface for Python.  The package itself provides the
 SAX exceptions and the convenience functions which will be most used by users of
 the SAX API.
 
+
+.. warning::
+
+   The :mod:`xml.sax` module is not secure against maliciously
+   constructed data.  If you need to parse untrusted or unauthenticated data see
+   :ref:`xml-vulnerabilities`.
+
+
 The convenience functions are:
 
 

diff --git a/Doc/library/xmlrpc.client.rst b/Doc/library/xmlrpc.client.rst
index e72770ae9357a893bfb119bc471fa34b2fe28326..4592fb40a8154e95464db886c4392f2c64bba1d9 100644 (file)
--- a/Doc/library/xmlrpc.client.rst
+++ b/Doc/library/xmlrpc.client.rst
@@ -21,6 +21,13 @@ supports writing XML-RPC client code; it handles all the details of translating
 between conformable Python objects and XML on the wire.
 
 
+.. warning::
+
+   The :mod:`xmlrpc.client` module is not secure against maliciously
+   constructed data.  If you need to parse untrusted or unauthenticated data see
+   :ref:`xml-vulnerabilities`.
+
+
 .. class:: ServerProxy(uri, transport=None, encoding=None, verbose=False, allow_none=False, use_datetime=False)
 
    A :class:`ServerProxy` instance is an object that manages communication with a

diff --git a/Doc/library/xmlrpc.server.rst b/Doc/library/xmlrpc.server.rst
index 67feba6d1fd355632334ad33ac82e04835310b11..6b4c202ca0bbc4a4a89aaed5d0cf483e08103530 100644 (file)
--- a/Doc/library/xmlrpc.server.rst
+++ b/Doc/library/xmlrpc.server.rst
@@ -16,6 +16,13 @@ servers written in Python.  Servers can either be free standing, using
 :class:`CGIXMLRPCRequestHandler`.
 
 
+.. warning::
+
+   The :mod:`xmlrpc.client` module is not secure against maliciously
+   constructed data.  If you need to parse untrusted or unauthenticated data see
+   :ref:`xml-vulnerabilities`.
+
+
 .. class:: SimpleXMLRPCServer(addr, requestHandler=SimpleXMLRPCRequestHandler, logRequests=True, allow_none=False, encoding=None, bind_and_activate=True)
 
    Create a new server instance.  This class provides methods for registration of

diff --git a/Misc/NEWS b/Misc/NEWS
index dbc12e7a3a64681a1caddf95016c2861c1610a07..5bdca7ff546cd463371c4278655332e282bc29fd 100644 (file)
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1156,6 +1156,8 @@ Tools/Demos
 Documentation
 -------------
 
+- Issue 17538: Document XML vulnerabilties
+
 - Issue #17047: remove doubled words in docs and docstrings
   reported by Serhiy Storchaka and Matthew Barnett.