Christian Heimes [Thu, 26 Sep 2019 15:02:59 +0000 (17:02 +0200)]
bpo-38275: Skip ssl tests for disabled versions (GH-16386)
test_ssl now handles disabled TLS/SSL versions better. OpenSSL's crypto
policy and run-time settings are recognized and tests for disabled versions
are skipped.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38275
Victor Stinner [Thu, 26 Sep 2019 13:51:50 +0000 (15:51 +0200)]
bpo-38234: Py_Initialize() sets global path configuration (GH-16421)
* Py_InitializeFromConfig() now writes PyConfig path configuration to
the global path configuration (_Py_path_config).
* Add test_embed.test_get_pathconfig().
* Fix typo in _PyWideStringList_Join().
* Group tests for specific functions and groups of related functions
into separate classes.
* Clean up creating and cleaning up temporary directories.
* Simplify and make more robust monkey patching of shutil.open.
Petr Viktorin [Thu, 26 Sep 2019 09:53:51 +0000 (11:53 +0200)]
bpo-38112: compileall: Skip long path path on Windows if the path can't be created (GH-16414)
This avoids the buildbot failure on Windows:
```
FileNotFoundError: [WinError 206] The filename or extension is too long: 'd:\\temp\\tmp5r3z438t\\long\\1\\2\\3\\4\\5\\6\\7\\8\\9\\10\\11\\12\\13\\14\\15\\16\\17\\18\\19\\20\\21\\22\\23\\24\\25\\26\\27\\28\\29\\30\\31\\32\\33\\34\\35\\36\\37\\38\\39\\40\\41\\42\\43\\44\\45\\46\\47\\48\\49\\50\\51\\52\\53\\54\\55\\56\\57\\58\\59\\60\\61\\62\\63\\64\\65\\66\\67\\68\\69\\70\\71\\72\\73\\74\\75\\76\\77\\78'
```
Creates a path that's long but avoids OS restrictions.
* Raise the limit of maximum path depth to actual recursion limit
* Add posibilities to adjust a path compiled in .pyc file.
Now, you can:
- Strip a part of path from a beggining of path into compiled file
example "-s /test /test/build/real/test.py" → "build/real/test.py"
- Append some new path to a beggining of path into compiled file
example "-p /boo real/test.py" → "/boo/real/test.py"
You can also use both options in the same time. In that case,
striping is done before appending.
* Add a possibility to specify multiple optimization levels
Each optimization level then leads to separated compiled file.
Use `action='append'` instead of `nargs='+'` for the -o option.
Instead of `-o 0 1 2`, specify `-o 0 -o 1 -o 2`. It's more to type,
but much more explicit.
* Add a symlinks limitation feature
This feature allows us to limit byte-compilation of symbolic
links if they are pointing outside specified dir (build root
for example).
Victor Stinner [Thu, 26 Sep 2019 02:01:49 +0000 (04:01 +0200)]
bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406)
Explicitly preinitializes with a Python preconfiguration to avoid
Py_SetPath() implicit preinitialization with a compat
preconfiguration.
Fix also test_init_setpath() and test_init_setpythonhome() on macOS:
use self.test_exe as the executable (and base_executable), rather
than shutil.which('python3').
Victor Stinner [Thu, 26 Sep 2019 00:22:35 +0000 (02:22 +0200)]
bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)
* Add test_embed.test_init_setpath_config(): test Py_SetPath()
with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import and base_executable
fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
calculated base_executable: _PyConfig_InitPathConfig() copies
executable to base_executable.
* Complete path config documentation.
Christian Heimes [Wed, 25 Sep 2019 21:03:30 +0000 (23:03 +0200)]
bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)
* Updated _hashopenssl.c to be PEP 384 compliant
* Remove refleak test from test_hashlib. The updated type no longer accepts random arguments to __init__.
Christian Heimes [Wed, 25 Sep 2019 15:55:02 +0000 (17:55 +0200)]
bpo-38271: encrypt private key test files with AES256 (GH-16385)
The private keys for test_ssl were encrypted with 3DES in traditional
PKCS#5 format. 3DES and the digest algorithm of PKCS#5 are blocked by
some strict crypto policies. Use PKCS#8 format with AES256 encryption
instead.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38271
bpo-38234: On macOS and FreeBSD, the temporary directory can be
symbolic link. For example, /tmp can be a symbolic link to /var/tmp.
Call realpath() to resolve all symbolic links.
Christian Heimes [Wed, 25 Sep 2019 14:30:20 +0000 (16:30 +0200)]
bpo-38270: Check for hash digest algorithms and avoid MD5 (GH-16382)
Make it easier to run and test Python on systems with restrict crypto policies:
* add requires_hashdigest to test.support to check if a hash digest algorithm is available and working
* avoid MD5 in test_hmac
* replace MD5 with SHA256 in test_tarfile
* mark network tests that require MD5 for MD5-based digest auth or CRAM-MD5
Victor Stinner [Wed, 25 Sep 2019 13:52:49 +0000 (15:52 +0200)]
bpo-36046: posix_spawn() doesn't support uid/gid (GH-16384)
* subprocess.Popen now longer uses posix_spawn() if uid, gid or gids are set.
* test_subprocess: add "nobody" and "nfsnobody" group names for test_group().
* test_subprocess: test_user() and test_group() are now also tested with close_fds=False.
Victor Stinner [Wed, 25 Sep 2019 00:54:25 +0000 (02:54 +0200)]
bpo-38234: Cleanup getpath.c (GH-16367)
* search_for_prefix() directly calls reduce() if found is greater
than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path
Samuel Freilich [Tue, 24 Sep 2019 19:08:31 +0000 (15:08 -0400)]
bpo-36871: Handle spec errors in assert_has_calls (GH-16005)
The fix in PR 13261 handled the underlying issue about the spec for specific methods not being applied correctly, but it didn't fix the issue that was causing the misleading error message.
The code currently grabs a list of responses from _call_matcher (which may include exceptions). But it doesn't reach inside the list when checking if the result is an exception. This results in a misleading error message when one of the provided calls does not match the spec.
Multiprocessing test test_mymanager() now also expects -SIGTERM, not
only exitcode 0.
bpo-30356: BaseManager._finalize_manager() sends SIGTERM to the
manager process if it takes longer than 1 second to stop, which
happens on slow buildbots.
Marc Hartmayer [Tue, 24 Sep 2019 03:34:12 +0000 (05:34 +0200)]
closes bpo-16637: libpython: construct integer object directly from gdbvalue (GH-15232)
This fixes the exception '`ValueError: invalid literal for int() with base 10`
if `str(gdbval)` returns a hexadecimal value (e.g. '0xa0'). This is the case if
the output-radix is set to 16 in gdb. See
https://sourceware.org/gdb/onlinedocs/gdb/Numbers.html for more information.
Victor Stinner [Mon, 23 Sep 2019 22:55:48 +0000 (00:55 +0200)]
bpo-38234: read_pth_file() now returns PyStatus (GH-16338)
Refactor path configuration code:
* read_pth_file() now returns PyStatus to report errors, rather than
calling Py_FatalError().
* Move argv0_path and zip_path buffers out of PyCalculatePath
structures.
* On Windows, _PyPathConfig.home is now preferred over PyConfig.home.
* _PyConfig_InitPathConfig() now starts by copying the global path
configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
_PyPathConfig fields which are already computed. For example,
if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
"_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
Victor Stinner [Fri, 20 Sep 2019 23:50:16 +0000 (01:50 +0200)]
bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298)
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() now
uses its value.
* Py_Initialize() now longer copies path configuration from PyConfig
to the global path configuration (_Py_path_config).
Kyle Stanley [Fri, 20 Sep 2019 23:46:21 +0000 (19:46 -0400)]
Doc: Remove provisional note for asyncio.run() (GH-16310)
Based on a comment from @asvetlov https://github.com/python/cpython/pull/15735#discussion_r323619076, this removes the provisional note for ``asyncio.run()`` in the documentation.
bpo-37937: Mention frame.f_trace in sys.settrace docs (GH-15439)
Mention frame.f_trace in sys.settrace docs, as well as the fact you still
need to call `sys.settrace` to enable the tracing machinery before setting
`frame.f_trace` will have any effect.
Michael Selik [Fri, 20 Sep 2019 03:25:55 +0000 (20:25 -0700)]
bpo-34002: Minor efficiency and clarity improvements in email package. (GH-7999)
* Check intersection of two sets explicitly
Comparing ``len(a) > ``len(a - b)`` is essentially looking for an
intersection between the two sets. If set ``b`` does not intersect ``a``
then ``len(a - b)`` will be equal to ``len(a)``. This logic is more
clearly expressed as ``a & b``.
* Change while/pop to a for-loop
Copying the list, then repeatedly popping the first element was
unnecessarily slow. I also cleaned up a couple other inefficiencies.
There's no need to unpack a tuple, then re-pack and append it. The list
can be created with the first element instead of empty. Secondly, the
``endswith`` method returns a bool, so there's no need for an if-
statement to set ``encoding`` to True or False.
* Use set.intersection to check for intersections
``a.intersection(b)`` method is more clear of purpose than ``not
a.isdisjoint(b)`` and avoids an unnecessary set construction that ``a &
set(b)`` performs.
* Use not isdisjoint instead of intersection
While it reads slightly worse, the isdisjoint method will stop when it
finds a counterexample and returns a bool, rather than looping over the
entire iterable and constructing a new set.
bpo-37828: Fix default mock_name in unittest.mock.assert_called error (GH-16166)
In the format string for assert_called the evaluation order is incorrect and hence for mock's without name, 'None' is printed whereas it should be 'mock' like for other messages. The error message is ("Expected '%s' to have been called." % self._mock_name or 'mock').
Victor Stinner [Tue, 17 Sep 2019 08:08:19 +0000 (10:08 +0200)]
bpo-37531: regrtest main process uses shorter timeout (GH-16220)
When using multiprocesss (-jN), the main process now uses a timeout
of 60 seconds instead of the double of the --timeout value. The
buildbot server stops a job which does not produce any output in 1200
seconds.
Ivan Levkivskyi [Mon, 16 Sep 2019 22:04:06 +0000 (23:04 +0100)]
bpo-28556: Update the opening note in typing docs (GH-16204)
This PR replaces the old note mentioning that `typing` is a provisional module with a new one mentioning types are not enforced at runtime. I am not sure if there was any official announcement about making `typing` non-provisional, but _de-facto_ no new features were added during Python 3.7, and no backwards incompatible changes were made except for few small things that were considered bugs.