bpo-37705: Improve the implementation of winerror_to_errno() (GH-15623)
winerror_to_errno() is no longer automatically generated.
Do not rely on the old _dosmapperr() function.
Add ERROR_NO_UNICODE_TRANSLATION (1113) -> EILSEQ.
(cherry picked from commit 19052a11314e7be7ba003fd6cdbb5400a5d77d96)
bpo-37936: Avoid ignoring files that we actually do track. (GH-15451)
There were about 14 files that are actually in the repo but that are
covered by the rules in .gitignore.
Git itself takes no notice of what .gitignore says about files that
it's already tracking... but the discrepancy can be confusing to a
human that adds a new file unexpectedly covered by these rules, as
well as to non-Git software that looks at .gitignore but doesn't
implement this wrinkle in its semantics. (E.g., `rg`.)
Several of these are from rules that apply more broadly than
intended: for example, `Makefile` applies to `Doc/Makefile` and
`Tools/freeze/test/Makefile`, whereas `/Makefile` means only the
`Makefile` at the repo's root.
And the `Modules/Setup` rule simply wasn't updated after 961d54c5c.
bpo-37445: Include FORMAT_MESSAGE_IGNORE_INSERTS in FormatMessageW() calls (GH-14462)
If FormatMessageW() is passed the FORMAT_MESSAGE_FROM_SYSTEM flag without FORMAT_MESSAGE_IGNORE_INSERTS, it will fail if there are insert sequences in the message definition.
(cherry picked from commit a6563650c835d50f7302971a5b145e94f9d0dc68)
Restart lines now always start with '=' and never end with ' ' and fill the width of the window unless that would require ending with ' ', which could be wrapped by itself and possible confusing the user.
(cherry picked from commit 38da805d563422cf1bb9cd9be24c73806840fe30)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
bpo-37380: subprocess: don't use _active on win (GH-14360) (GH-15707)
As noted by @eryksun in [1] and [2], using _cleanup and _active(in
__del__) is not necessary on Windows, since:
> Unlike Unix, a process in Windows doesn't have to be waited on by
> its parent to avoid a zombie. Keeping the handle open will actually
> create a zombie until the next _cleanup() call, which may be never
> if Popen() isn't called again.
This patch simply defines `subprocess._active` as `None`, for which we already
have the proper logic in place in `subprocess.Popen.__del__`, that prevents it
from trying to append the process to the `_active`. This patch also defines
`subprocess._cleanup` as a noop for Windows.
bpo-37902: IDLE: Add scrolling for IDLE browsers. (GH-15368)
Modify the wheel event handler so it can also be used for module, path, and stack browsers.
Patch by George Zhang.
(cherry picked from commit 2cd902585815582eb059e3b40e014ebe4e7fdee7)
Fixes a case in which email._header_value_parser.get_unstructured hangs the system for some invalid headers. This covers the cases in which the header contains either:
- a case without trailing whitespace
- an invalid encoded word
closes bpo-37966: Fully implement the UAX GH-15 quick-check algorithm. (GH-15558)
The purpose of the `unicodedata.is_normalized` function is to answer
the question `str == unicodedata.normalized(form, str)` more
efficiently than writing just that, by using the "quick check"
optimization described in the Unicode standard in UAX GH-15.
However, it turns out the code doesn't implement the full algorithm
from the standard, and as a result we often miss the optimization and
end up having to compute the whole normalized string after all.
Implement the standard's algorithm. This greatly speeds up
`unicodedata.is_normalized` in many cases where our partial variant
of quick-check had been returning MAYBE and the standard algorithm
returns NO.
At a quick test on my desktop, the existing code takes about 4.4 ms/MB
(so 4.4 ns per byte) when the partial quick-check returns MAYBE and it
has to do the slow normalize-and-compare:
$ build.base/python -m timeit -s 'import unicodedata; s = "\uf900"*500000' \
-- 'unicodedata.is_normalized("NFD", s)'
50 loops, best of 5: 4.39 msec per loop
With this patch, it gets the answer instantly (58 ns) on the same 1 MB
string:
$ build.dev/python -m timeit -s 'import unicodedata; s = "\uf900"*500000' \
-- 'unicodedata.is_normalized("NFD", s)' 5000000 loops, best of 5: 58.2 nsec per loop
This restores a small optimization that the original version of this
code had for the `unicodedata.normalize` use case.
With this, that case is actually faster than in master!
$ build.base/python -m timeit -s 'import unicodedata; s = "\u0338"*500000' \
-- 'unicodedata.normalize("NFD", s)'
500 loops, best of 5: 561 usec per loop
$ build.dev/python -m timeit -s 'import unicodedata; s = "\u0338"*500000' \
-- 'unicodedata.normalize("NFD", s)'
500 loops, best of 5: 512 usec per loop
(cherry picked from commit 2f09413947d1ce0043de62ed2346f9a2b4e5880b)
bpo-38022: IDLE: upgrade help.html to sphinx 2.x HTML5 output (GH-15664)
The HTML5 output from Sphinx 2.x adds '<p>' tags within list elements. Using a new prevtag attribute, ignore these instead of emitting unwanted '\n\n'.
Also stop looking for 'first' classes on tags (no longer present) and fix the bug of double-spacing instead of single spacing after <pre> blocks.
(cherry picked from commit 580bdb0ece681537eadb360f0c796123ead7a559)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
[3.8] bpo-21315: Fix parsing of encoded words with missing leading ws (GH-13425) (GH-15655)
* [bpo-21315](https://bugs.python.org/issue21315): Fix parsing of encoded words with missing leading ws.
Because of missing leading whitespace, encoded word would get parsed as
unstructured token. This patch fixes that by looking for encoded words when
splitting tokens with whitespace.
Missing trailing whitespace around encoded word now register a defect
instead.
Original patch suggestion by David R. Murray on [bpo-21315](https://bugs.python.org/issue21315).
(cherry picked from commit 66c4f3f38b867d8329b28c032bb907fd1a2f22d2)
Extending the hover delay in test_tooltip should avoid spurious test_idle failures.
One longer delay instead of two shorter delays results in a net speedup.
(cherry picked from commit 132acaba5a7f01373ca624b1a5975b190fe866f5)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
Fix a ctypes regression of Python 3.8. When a ctypes.Structure is
passed by copy to a function, ctypes internals created a temporary
object which had the side effect of calling the structure finalizer
(__del__) twice. The Python semantics requires a finalizer to be
called exactly once. Fix ctypes internals to no longer call the
finalizer twice.
Create a new internal StructParam_Type which is only used by
_ctypes_callproc() to call PyMem_Free(ptr) on Py_DECREF(argument).
StructUnionType_paramfunc() creates such object.
(cherry picked from commit 96b4087ce784ee7434dffdf69c475f5b40543982)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
when built on non-Windows system without fd system call support,
like older versions of macOS.
(cherry picked from commit 7fcc2088a50a4ecb80e5644cd195bee209c9f979)
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593)
With `symtable_visit_expr` now correctly adjusting the recursion depth for named
expressions, `symtable_handle_namedexpr` should be leaving it alone.
Also adds a new check to `PySymtable_BuildObject` that raises `SystemError`
if a successful first symbol analysis pass fails to keep the stack depth
accounting clean.
(cherry picked from commit 06145230c833c3db5dab8858e11bcd550a37c57f)
bsiem [Thu, 29 Aug 2019 04:47:15 +0000 (06:47 +0200)]
[3.8] bpo-37482: Fix email address name with encoded words and special chars (GH-14561) (GH-15380)
Special characters in email address header display names are normally
put within double quotes. However, encoded words (=?charset?x?...?=) are
not allowed withing double quotes. When the header contains a word with
special characters and another word that must be encoded, the first one
must also be encoded.
In the next example, the display name in the From header is quoted and
therefore the comma is allowed; in the To header, the comma is not
within quotes and not encoded, which is not allowed and therefore
rejected by some mail servers.
From: "Foo Bar, France" <foo@example.com>
To: Foo Bar, =?utf-8?q?Espa=C3=B1a?= <foo@example.com>
bpo-37936: Remove some .gitignore rules that were intended locally. (GH-15542)
These appeared in commit c5ae169e1. The comment on them, as well as
the presence among them of a rule for the .gitignore file itself,
indicate that the author intended these lines to remain only in their
own local working tree -- not to get committed even to their own repo,
let alone merged upstream.
They did nevertheless get committed, because it turns out that Git
takes no notice of what .gitignore says about files that it's already
tracking... for example, this .gitignore file itself.
Give effect to these lines' original intention, by deleting them. :-)
Git tip, for reference: the `.git/info/exclude` file is a handy way
to do exactly what these lines were originally intended to do. A
related handy file is `~/.config/git/ignore`. See gitignore(5),
aka `git help ignore`, for details.
Declare Py_BytesMain() in Include/pylifecycle.h, rather in
Include/internal/pycore_pylifecycle.h.
(cherry picked from commit 9a943b4ce13fac26873b8100e89c818c5c47ac4b)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
[3.8] bpo-36917: Add default implementation of ast.NodeVisitor.visit_Constant(). (GH-15490) (GH-15509)
It emits a deprecation warning and calls corresponding method
visit_Num(), visit_Str(), etc.
(cherry picked from commit c3ea41e9bf100a5396b851488c3efe208e5e2179)
Nick Coghlan [Sun, 25 Aug 2019 14:41:47 +0000 (00:41 +1000)]
[3.8] bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15491)
- drop TargetScopeError in favour of raising SyntaxError directly
as per the updated PEP 572
- comprehension iteration variables are explicitly local, but
named expression targets in comprehensions are nonlocal or
global. Raise SyntaxError as specified in PEP 572
- named expression targets in the outermost iterable of a
comprehension have an ambiguous target scope. Avoid resolving
that question now by raising SyntaxError. PEP 572
originally required this only for cases where the bound name
conflicts with the iteration variable in the comprehension,
but CPython can't easily restrict the exception to that case
(as it doesn't know the target variable names when visiting
the outermost iterator expression)
These were caused by keeping around a reference to the Squeezer
instance and calling it's load_font() upon config changes, which
sometimes happened even if the shell window no longer existed.
This change completely removes that mechanism, instead having the
editor window properly update its width attribute, which can then
be used by Squeezer.
(cherry picked from commit d4b4c00b57d24f6ee2cf3a96213406bb09953df3)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
* fix Path._add_implied_dirs to include all implied directories
* fix Path._add_implied_dirs to include all implied directories
* Optimize code by using sets instead of lists
* 📜🤖 Added by blurb_it.
* fix Path._add_implied_dirs to include all implied directories
* Optimize code by using sets instead of lists
* 📜🤖 Added by blurb_it.
* Add tests to zipfile.Path.iterdir() fix
* Update test for zipfile.Path.iterdir()
* remove whitespace from test file
* Rewrite NEWS blurb to describe the user-facing impact and avoid implementation details.
* remove redundant [] within set comprehension
* Update to use unique_everseen to maintain order and other suggestions in review
* remove whitespace and add back add_dirs in tests
* Add new standalone function parents using posixpath to get parents of a directory
* removing whitespace (sorry)
* Remove import pathlib from zipfile.py
* Rewrite _parents as a slice on a generator of the ancestry of a path.
* Remove check for '.' and '/', now that parents no longer returns those.
* Separate calculation of implied dirs from adding those
* Re-use _implied_dirs in tests for generating zipfile with dir entries.
* Replace three fixtures (abcde, abcdef, abde) with one representative example alpharep.
* Simplify implementation of _implied_dirs by collapsing the generation of parent directories for each name.
(cherry picked from commit a4e2991bdc993b60b6457c8a38d6e4a1fc845781)
Serhiy Storchaka [Sat, 24 Aug 2019 10:41:53 +0000 (13:41 +0300)]
[3.8] bpo-37830: Fix compilation of break and continue in finally. (GH-15320) (GH-15456)
Fix compilation of "break" and "continue" in the
"finally" block when the corresponding "try" block
contains "return" with a non-constant value.
(cherry picked from commit ef61c524ddeeb56da3858b86e349e7288d68178e)
projects / python / log