The regex to test a mail server's timestamp is susceptible to
catastrophic backtracking on long evil responses from the server.
Happily, the maximum length of malicious inputs is 2K thanks
to a limit introduced in the fix for CVE-2013-1752.
A 2KB evil response from the mail server would result in small slowdowns
(milliseconds vs. microseconds) accumulated over many apop calls.
This is a potential DOS vector via accumulated slowdowns.
Replace it with a similar non-vulnerable regex.
The new regex is RFC compliant.
The old regex was non-compliant in edge cases.
* Prevent difflib REDOS (CVE-2018-1061)
The default regex for IS_LINE_JUNK is susceptible to
catastrophic backtracking.
This is a potential DOS vector.
Replace it with an equivalent non-vulnerable regex.
Also introduce unit and REDOS tests for difflib.
Co-authored-by: Tim Peters <tim.peters@gmail.com> Co-authored-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 0e6c8ee2358a2e23117501826c008842acb835ac)
Serhiy Storchaka [Tue, 23 Jan 2018 11:21:21 +0000 (13:21 +0200)]
[3.5] bpo-32072: Fix issues with binary plists. (GH-4455) (#4656)
* [3.5] bpo-32072: Fix issues with binary plists. (GH-4455)
* Fixed saving bytearrays.
* Identical objects will be saved only once.
* Equal references will be load as identical objects.
* Added support for saving and loading recursive data structures.
(cherry picked from commit a897aee)
* Fix implementation dependent assertion in test_plistlib. (#4813)
Directory and zipfile execution previously added
the parent directory of the directory or zipfile
as sys.path[0] and then subsequently overwrote
it with the directory or zipfile itself.
This caused problems in isolated mode, as it
overwrote the "stdlib as a zip archive" entry
in sys.path, as the parent directory was
never added.
The attempted fix to that issue in bpo-29319
created the opposite problem in *non*-isolated
mode, by potentially leaving the parent
directory on sys.path instead of overwriting it.
This change fixes the root cause of the problem
by removing the whole "add-and-overwrite" dance
for sys.path[0], and instead simply never adds
the parent directory to sys.path in the first
place.
Victor Stinner [Mon, 25 Sep 2017 00:58:32 +0000 (17:58 -0700)]
[3.5][Security] bpo-30947, bpo-31170: Update expat from 2.2.1 to 2.2.4 (#3354)
* bpo-30947, bpo-31170: Update expat from 2.2.1 to 2.2.4
* Upgrade libexpat embedded copy from version 2.2.1 to 2.2.3 to get security
fixes.
* Update libexpat from 2.2.3 to 2.2.4. Fix copying of partial
characters for UTF-8 input (libexpat bug 115):
https://github.com/libexpat/libexpat/issues/115
The `Show Source` was broken because of a change made in sphinx 1.5.1
In Sphinx 1.4.9, the sourcename was "index.txt".
In Sphinx 1.5.1+, it is now "index.rst.txt".
(cherry picked from commit b9ff498793611d1c6a9b99df464812931a1e2d69)
Ned Deily [Fri, 8 Sep 2017 17:42:19 +0000 (10:42 -0700)]
[3.5] bpo-31036: Allow sphinx and blurb to be found automatically (GH-3440)
Rather than requiring the path to blurb and/or sphinx-build to be specified to
the make rule, enhance the Doc/Makefile to look for each first in a virtual
environment created by make venv and, if not found, look on the normal process
PATH. This allows the Doc/Makefile to take advantage of an installed
spinx-build or blurb and, thus, do the right thing most of the time. Also, make
the directory for the venv be configurable and document the `make venv` target.
Victor Stinner [Mon, 24 Jul 2017 15:40:47 +0000 (17:40 +0200)]
bpo-30908: Fix dangling thread in test_os.TestSendfile (#2680) (#2845)
tearDown() now clears explicitly the self.server variable to make
sure that the thread is completely cleared when tearDownClass()
checks if all threads have been cleaned up.
Fix the following warning:
$ ./python -m test --fail-env-changed -m test.test_os.TestSendfile.test_keywords -R 3:1 test_os
(...)
Warning -- threading_cleanup() failed to cleanup 0 threads after 3 sec (count: 0, dangling: 2)
(...)
Tests result: ENV CHANGED
(cherry picked from commit d1cc037d1442cc35d1b194ec8e50901514360949)
* bpo-18018: Import raises ImportError instead of SystemError if a relative
import is attempted without a known parent package.
* bpo-26367: importlib.__init__() raises ImportError like
builtins.__import__() when ``level`` is specified but without an accompanying
package specified.
Cython will, in the right circumstances, offer a MethodType instance
where im_func is a builtin function. Any instance of MethodType is
automatically assumed to be a Python-defined function (more
specifically, a function that has an inspectable signature), but
_set_signature was still conservative in its assumptions. As a result
_set_signature would return early with None instead of a mock since
the im_func had no inspectable signature. This causes problems
deeper inside mock, as _set_signature is assumed to _always_
return a mock, and nothing checked its return value.
In similar corner cases, autospec will simply not check the spec of the
function, so _set_signature is amended to now return early with the
original, not-wrapped mock object.
Victor Stinner [Thu, 20 Jul 2017 16:08:54 +0000 (18:08 +0200)]
bpo-30822: regrtest: remove tzdata (#2782)
* Oops, tzdata was introduced in Python 3.6: remove it from regrtest
* Remove also Lib/test/libregrtest/__init__.py file: add by mistake
on a backport.
Victor Stinner [Thu, 20 Jul 2017 15:08:51 +0000 (17:08 +0200)]
bpo-30822: Exclude tzdata from regrtest --all (#2775) (#2781)
When running the test suite using --use=all / -u all, exclude tzdata
since it makes test_datetime too slow (15-20 min on some buildbots)
which then times out on some buildbots.
-u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata,
to run all test_datetime tests.
Fix also regrtest command line parser to allow passing -u
extralargefile to run test_zipfile64.
Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all.
(cherry picked from commit 5b392bbaeb9d9b1db961ecfc7315d8c8662c27f6)
[3.5] bpo-29854: Fix segfault in call_readline() (GH-728)
If history-length is set in .inputrc, and the history file is double the
history size (or more), history_get(N) returns NULL, and python
segfaults. Fix that by checking for NULL return value.
It seems that the root cause is incorrect handling of bigger history in
readline, but Python should not segfault even if readline returns
unexpected value.
This issue affects only GNU readline. When using libedit emulation
system history size option does not work.
Joel Hillacre [Thu, 6 Jul 2017 21:29:09 +0000 (15:29 -0600)]
bpo-30532: Fix whitespace folding in certain cases (#2592)
Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm. This makes the whitespace handling code consistent.
Improve the description of the high and low parameters for set_write_buffer_limits.
Also fixed a small grammatical issue.
(cherry picked from commit 5200a7c7f9ea65a96330c5f276f4acc6ec70854e)
Victor Stinner [Fri, 30 Jun 2017 15:20:31 +0000 (17:20 +0200)]
bpo-30280: Cleanup threads in ayncio tests (#2501) (#2512)
* bpo-30280: asyncio now cleans up threads
asyncio base TestCase now uses threading_setup() and
threading_cleanup() of test.support to cleanup threads.
* asyncio: Fix TestBaseSelectorEventLoop cleanup
bpo-30280: TestBaseSelectorEventLoop of
test.test_asyncio.test_selector_events now correctly closes the event
loop: cleanup its executor to not leak threads.
Don't override the close() method of the event loop, only override
the_close_self_pipe() method.
Victor Stinner [Fri, 30 Jun 2017 11:12:15 +0000 (13:12 +0200)]
bpo-30813: Fix unittest when hunting refleaks (#2502) (#2506)
bpo-11798, bpo-16662, bpo-16935, bpo-30813: Skip
test_discover_with_module_that_raises_SkipTest_on_import() and
test_discover_with_init_module_that_raises_SkipTest_on_import() of
test_unittest when hunting reference leaks using regrtest.
(cherry picked from commit e4f9a2d2be42d5a2cdd624f8ed7cdf5028c5fbc3)
Add private C API function _PyUnicode_AsUnicode() which is similar to
PyUnicode_AsUnicode(), but checks for null characters..
(cherry picked from commit f7eae0adfcd4c50034281b2c69f461b43b68db84)
* regrtest --list-cases now supports --match and --match-file options.
Example: ./python -m test --list-cases -m FileTests test_os
* --list-cases now also sets support.verbose to False to prevent
messages to stdout when loading test modules.
* Add support._match_test() private function.
(cherry picked from commit ace56d583664f855d89d1219ece7c21c2fddcf30)
* Change the regrtest --huntrleaks checker to decide if a test file
leaks or not. Require that each run leaks at least 1 reference.
* Warmup runs are now completely ignored: ignored in the checker test
and not used anymore to compute the sum.
* Add an unit test for a reference leak.
Example of reference differences previously considered a failure
(leak) and now considered as success (success, no leak):
bpo-30764, bpo-29335: test_child_terminated_in_stopped_state() of
test_subprocess now uses support.SuppressCrashReport() to prevent the
creation of a core dump on FreeBSD.
(cherry picked from commit cdee3f14f7f4c995e7eedb0bf6a67e260c739f7d)
Brett Cannon [Fri, 23 Jun 2017 18:39:53 +0000 (11:39 -0700)]
[3.5] bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268) (GH-2365)
Bug didn't manifest itself when importing a module with source as .py files are always the first on the search path. The issue only showed up in bytecode-only packages where the calculated file path would be ``__init__.py/__init__.pyc``.
larryhastings [Thu, 22 Jun 2017 23:01:00 +0000 (16:01 -0700)]
Add "Misc/NEWS.d" directory tree for "blurb". GH-2330
CPython workflow is changing! We're going to start using "blurb"
to manage Misc/NEWS entries:
https://github.com/python/core-workflow
(This will be a big win for release managers, honest.)
This checkin simply populates the "Misc/NEWS.d" subdirectory tree
so that people can start putting their news entries in there.
No other changes (yet).
projects / python / log