bpo-6135: Fix subprocess.check_output doc to mention changes in 3.6 (GH-5564) (GH-5573)
Fixes the documentation for `subprocess.check_output()` not mentioning that the encoding and errors parameters were added in 3.6.
(cherry picked from commit fc1ce810f1da593648b4d19e7d582a235ec1dd37)
bpo-32777: Fix _Py_set_inheritable async-safety in subprocess (GH-5560) (GH-5563)
Fix a rare but potential pre-exec child process deadlock in subprocess on POSIX systems when marking file descriptors inheritable on exec in the child process. This bug appears to have been introduced in 3.4 with the inheritable file descriptors support.
This also changes Python/fileutils.c `set_inheritable` to use the "slow" two `fcntl` syscall path instead of the "fast" single `ioctl` syscall path when asked to be async signal safe (by way of being asked not to raise exceptions). `ioctl` is not a POSIX async-signal-safe approved function.
[3.6] bpo-8722: Document __getattr__ behavior with AttributeError in property (GH-5542)
When `__getattr__` is implemented, attribute lookup will always fall back to that,
even if the initial failure comes from `__getattribute__` or a descriptor's `__get__`
method (including property methods).
(cherry picked from commit d1f318105b8781b01f3507d5cb0fd841b977d5f2)
bpo-32720: Fixed the definition for width and precision in format mini-language doc (GH-5482) (GH-5525)
Changed the definition of width and precision from "integer" to "digit+" in format mini-language doc.
(cherry picked from commit 8b5fa289fdb04b6b919cf95fa99246aa872e47a8)
Fix typos found by codespell in docs, docstrings, and comments.
Fixes for the following files were in post-3.6 code and not backported:
Lib/ctypes/_aix.py (new), Lib/test/test_concurrent_futures.py,
Modules/_asynciomodule.c, Modules/_pickle.c, Objects/obmalloc.c.
Barry Warsaw [Sat, 3 Feb 2018 04:21:14 +0000 (23:21 -0500)]
[3.6] bpo-32303 - Consistency fixes for namespace loaders (GH-5481) (#5504)
* Make sure ``__spec__.loader`` matches ``__loader__`` for namespace packages.
* Make sure ``__spec__.origin` matches ``__file__`` for namespace packages.
Terry Jan Reedy [Fri, 2 Feb 2018 22:37:30 +0000 (17:37 -0500)]
[3.6] bpo-32614: Modify re examples to use a raw string to prevent wa… …rning (GH-5265) (GH-5500)
Modify RE examples in documentation to use raw strings to prevent DeprecationWarning.
Add text to REGEX HOWTO to highlight the deprecation. Approved by Serhiy Storchaka.
bpo-32674: Improve the docstring for __import__ (GH-5339) (GH-5489)
Clarify that the level argument is used to determine whether to
perform absolute or relative imports: 0 is absolute, while a positive number
is the number of parent directories to search relative to the current module.
(cherry picked from commit 461d225b195eec5269f317323b41115516144c41)
bpo-32735: Fix typo in f-strings datetime format specifier example (GH-5464) (GH-5465)
The f-string example for using datetime format specifier does not match the given output.
Changed the format from %b to %B so it matches the output of "January".
(cherry picked from commit b299393cc372f3ecbef4304f8eaa4c7838e975ca)
Xiang Zhang [Wed, 31 Jan 2018 13:34:17 +0000 (21:34 +0800)]
[3.6] bpo-32583: Fix possible crashing in builtin Unicode decoders (GH-5325) (#5459)
When using customized decode error handlers, it is possible for builtin decoders
to write out-of-bounds and then crash..
(cherry picked from commit 2c7fd46e11333ef5e5cce34212f7d087694f3658)
[3.6] closes bpo-30117: fix lib2to3 ParserIdempotency test (GH-1242) (GH-5443)
Fix two (in my opinion) spurious failure conditions in the lib2to3.tests.test_parser.TestParserIdempotency test_parser test.
Use the same encoding found in the initial file to write a temp file for a diff. This retains the BOM if the encoding was initially utf-8-sig.
If the file cannot be parsed using the normal grammar, try again with no print statement which should succeed for valid files using future print_function
For case (1), the driver was correctly handling a BOM in a utf-8 file, but then the test was not writing a comparison file using 'utf-8-sig' to diff against, so the BOM got removed. I don't think that is the fault of the parser, and lib2to3 will retain the BOM.
For case (2), lib2to3 pre-detects the use of from __future__ import print_function or allows the user to force this interpretation with a -p flag, and then selects a different grammar with the print statement removed. That makes the test cases unfair to this test as the driver itself doesn't know which grammar to use. As a minimal fix, the test will try using a grammar with the print statement, and if that fails fall back on a grammar without it. A more thorough handling of the idempotency test would to be to parse all files using both grammars and ignore if one of the two failed but otherwise check both. I didn't think this was necessary but can change..
(cherry picked from commit 14e976e00e65bf343ba0fca016c3c9132a843daf)
Document the error return of PyLong_As* APIs. (GH-5396) (#5404)
Document the error return of PyLong_As* APIs.
A frequent Python C API usage error is neglecting to check the return
value and/or PyErr_Occurred().
(cherry picked from commit f5b04a360e44aa9733f7a92dd66d2292d6c52955)
Antoine Pitrou [Sun, 28 Jan 2018 17:42:31 +0000 (18:42 +0100)]
[3.6] bpo-32228: Reset raw_pos after unwinding the raw stream (GH-4858) (#5389)
Ensure that ``truncate()`` preserves the file position (as reported by ``tell()``) after writes longer than the buffer size..
(cherry picked from commit 059f58ce938d9c3f0286412a4efb1b9131339421)
Travis when merging changes from a pull request onto the target branch
does not perform a rebase, instead it does a simple merge which causes
the PR commits to retain their commit dates. This means that the commit
log can potentially look like:
PR merge <-- HEAD
normal master commit <- master
more commits from normal workflow
PR commit 1
another master commit
PR commit 2
Performing a git diff from PR commit 2 to master will accidentally
include files that should not be there.
Cheryl Sabella [Sat, 27 Jan 2018 02:45:24 +0000 (21:45 -0500)]
[3.6] bpo-27505: Retrofit module __class__ documentation from 3.7 (GH-5321)
The module `__class__` attribute documentation added to 3.7 for PEP 562
(dynamic module attributes) also applies to earlier versions. This backports
that subset of the new docs to the 3.6 branch so that it will appear in the
main online documentation and in the final 3.6 binary release.
bpo-32667: Fix tests when $PATH contains a file (GH-5322) (#5323)
Some tests failed when the PATH environment variable contained a path
to an existing file. Fix tests to ignore also NotADirectoryError, not
only FileNotFoundError and PermissionError.
(cherry picked from commit b31206a223955d614d7769f95fb979d60f77bf87)
bpo-32635: Fix a segfault when importing the crypt module with libxcrypt. (GH-5284) (#5296)
glibc is deprecating libcrypt in favor of libxcrypt, however python assumes
that crypt.h will always be included. This change makes the header inclusion
explicit when libxcrypt is present on the system.
(cherry picked from commit e768c86ef442ef89004089a8a34ce5909ffb90f2)
[3.6] bpo-32028: Fix suggestions for indented print statements (GH-5249)
The suggested replacement for print statements previously failed to account
for leading whitespace and hence could end up including unwanted text in
the proposed call to the print builtin.
Fix typo in thread_nt.h code comment (GH-5211) (GH-5226)
The comment for PyThread_allocate_lock says "It has too be implemented ...".
There was an extra "o" in ".. to be implemented.."
(cherry picked from commit 6027802ca7fae118bce6afead51d01a174600d40)
Use assertCountEqual instead of assertEqual. (GH-5223) (#5225)
This test doesn't care about order, the underlying filesystem APIs do not
guarantee directory listings on subsequent calls will be in the same order.
(cherry picked from commit 3941499d6c2a4d09f2ea476f84b9c4ddc4bdefa0)
Skip test_readline.test_nonascii() on C locale (GH-5203) (#5204)
bpo-29240: On FreeBSD, if the LC_CTYPE locale is "C" or "POSIX",
writing and reading non-ASCII bytes into/from a TTY works,
but readline or ncurses ignores non-ASCII bytes on read.
(cherry picked from commit c495e799ed376af91ae2ddf6c4bcc592490fe294)
Victor Stinner [Mon, 15 Jan 2018 22:43:24 +0000 (23:43 +0100)]
[3.6] bpo-32555: Fix locale encodings (#5193)
On FreeBSD and Solaris, os.strerror() now always decode the byte
string from the current locale encoding, rather than using
ASCII/surrogateescape in some cases.
Changes:
* Add _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx() which has an
additional current_locale parameter.
* PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize() and
* PyUnicode_EncodeLocale() now always use the current locale
* encoding, instead of using Py_DecodeLocale()/Py_EncodeLocale().
* Document encoding in Py_DecodeLocale() and Py_EncodeLocale()
documentations.
* Add USE_FORCE_ASCII define to not define
decode_ascii_surrogateescape() on Android.
Victor Stinner [Mon, 15 Jan 2018 22:23:47 +0000 (23:23 +0100)]
[3.6] bpo-31900: Fix localeconv() encoding for LC_NUMERIC (#4174) (#5192)
* Add _Py_GetLocaleconvNumeric() function: decode decimal_point and
thousands_sep fields of localeconv() from the LC_NUMERIC encoding,
rather than decoding from the LC_CTYPE encoding.
* Modify locale.localeconv() and "n" formatter of str.format() (for
int, float and complex to use _Py_GetLocaleconvNumeric()
internally.
Clarify that on Windows, path must be a directory.
On Unix, path can be a file or a directory.
(cherry picked from commit ee3b83547c6b0cac1da2cb44aaaea533a1d1bbc8)
`os.path.is*()` can return False if the file can't be accessed.
The behaviour is documented in details in `os.path.exists()`.
Link to `os.path.exists()` from `os.path.is*()`.
(cherry picked from commit b3dd18d4035803b50c65a434955966d1b3b1f363)
Improve enum.Flag code example (GH-5167) (GH-5171)
The code example that demonstrate how to use enum.Flag was missing
the import of enum.auto.
(cherry picked from commit 0f31c74fcfdec8f9e6157de2c366f2273de81677)
Lexical Analysis ref doc: Fix a typo in the string concatenation internal link (GH-5157) (GH-5160)
In lexical analysis reference documentation, the internal link to
the string literal concatenation section was written as`.. _string-catenation:`.
Changed that to `.. _string-concatenation:`.
(cherry picked from commit 3764bb075a73074f55568fc7c37adde6054eb59a)
Zachary Ware [Sun, 7 Jan 2018 03:16:33 +0000 (21:16 -0600)]
[3.6] Kill the AppVeyor file whitelist (GH-5122)
It's more trouble than it's worth, since AppVeyor only checks the HEAD commit of a PR rather than the full diff against the base branch to decide which files changed.
(cherry picked from commit 7f7de371f947dc38e67505601927e9bc58fa268a)
bpo-32452: clarify term 'brackets' in generator tutorial (GH-5079) (#5081)
Updates documentation for generator expressions in classes tutorial: Clarify usage of ambiguous term "brackets" by replacing with "square brackets". Updated subsequent lines to respect line breaks. (GH-5079)
(cherry picked from commit f190eb59e60e2ae7a7cbd396458389a7a076e0d3)
bpo-32440: Update the docs URL to https in help() (GH-5030) (GH-5031)
In pydoc.py, the reference to Python' documentation was in http.
The link has been updated to use https.
(cherry picked from commit e5681b9822c633c77ddfeb94585d58895e0ecff5)
bpo-24960: use pkgutil.get_data in lib2to3 to read pickled grammar files (GH-4977) (#4979)
This is more complicated than it should be because we need to preserve the
useful mtime-based regeneration feature that lib2to3.pgen2.driver.load_grammar
has. We only look for the pickled grammar file with pkgutil.get_data and only if
the source file does not exist.
(cherry picked from commit 8a5877165e993afb2633cd48da5222326d3f6e0e)
The built-in map() function collects function arguments lazily, but concurrent.futures.Executor.map() does so eagerly.
(cherry picked from commit a7a751dd7b08a5bb6cb399c1b2a6ca7b24aba51d)
Improve the F-strings and format specifier documentation (GH-4931) (GH-4933)
Mention that the format-specifier mini language in f-strings
is the same one used by str.format.
(cherry picked from commit f4e21a2a72f76d75a6cc6f74faf910a5f3108482)