Petr Viktorin [Tue, 10 Sep 2019 11:21:09 +0000 (12:21 +0100)]
bpo-37499: Test various C calling conventions (GH-15776)
Add functions with various calling conventions to `_testcapi`, expose them as module-level functions, bound methods, class methods, and static methods, and test calling them and introspecting them through GDB.
bpo-38076: Make struct module PEP-384 compatible (#15805)
* PEP-384 _struct
* More PEP-384 fixes for _struct
Summary: Add a couple of more fixes for `_struct` that were previously missed such as removing `tp_*` accessors and using `PyBytesWriter` instead of calling `PyBytes_FromStringAndSize` with `NULL`. Also added a test to confirm that `iter_unpack` type is still uninstantiable.
Neil Schemenauer [Tue, 10 Sep 2019 09:44:20 +0000 (02:44 -0700)]
bpo-37725: have "make clean" remove PGO task data (#15033)
Change "clean" makefile target to also clean the program guided
optimization (PGO) data. Previously you would have to use "make
clean" and "make profile-removal", or "make clobber".
bpo-38043: Move unicodedata.normalize tests into test_unicodedata. (GH-15712)
Having these in a separate file from the one that's named after the
module in the usual way makes it very easy to miss them when looking
for tests for these two functions.
(In fact when working recently on is_normalized, I'd been surprised to
see no tests for it here and concluded the function had evaded being
tested at all. I'd gone as far as to write up some tests myself
before I spotted this other file.)
Mostly this just means moving all the one file's code into the other,
and moving code from the module toplevel to inside the test class to
keep it tidily separate from the rest of the file's code.
There's one substantive change, which reduces by a bit the amount of
code to be moved: we drop the `x > sys.maxunicode` conditional and all
the `RangeError` logic behind it. Now if that condition ever occurs
it will cause an error at `chr(x)`, and a test failure. That's the
right result because, since PEP 393 in Python 3.3, there is no longer
such a thing as an "unsupported character".
Cut tricky `goto` that isn't needed, in _PyBytes_DecodeEscape. (GH-15825)
This is the sort of `goto` that requires the reader to stare hard at
the code to unpick what it's doing.
On doing so, the answer is... not very much!
* It jumps from the bottom of the loop to almost the top; the effect
is to bypass the loop condition `s < end` and also the
`if`-condition `*s != '\\'`, acting as if both are true.
* We've just decremented `s`, after incrementing it in the `switch`
condition. So it has the same value as when `s == end` failed.
Before that was another increment... and before that we had
`s < end`. So `s < end` true, then increment, then `s == end`
false... that means `s < end` is still true.
* Also this means `s` points to the same character as it did for the
`switch` condition. And there was a `case '\\'`, which we didn't
hit -- so `*s != '\\'` is also true.
* That means this has no effect on the behavior! The most it might do
is an optimization -- we get to skip those two checks, because (as
just proven above) we know they're true.
* But gosh, this is the *invalid escape sequence* path. This does not
seem like the kind of code path that calls for extreme optimization
tricks.
So, take the `goto` and the label out.
Perhaps the compiler will notice the exact same facts we showed above,
and generate identical code. Or perhaps it won't! That'll be OK.
But then, crucially, if some future edit to this loop causes the
reasoning above to *stop* holding true... the compiler will adjust
this jump accordingly. One of us fallible humans might not.
bpo-36502: Update link to UAX #44, the Unicode doc on the UCD. (GH-15301)
The link we have points to the version from Unicode 6.0.0, dated 2010.
There have been numerous updates to it since then:
https://www.unicode.org/reports/tr44/#Modifications
Change the link to one that points to the current version. Also, use HTTPS.
bpo-35941: Fix performance regression in new code (GH-12610)
Accumulate certificates in a set instead of doing a costly list contain
operation. A Windows cert store can easily contain over hundred
certificates. The old code would result in way over 5,000 comparison
operations
Signed-off-by: Christian Heimes <christian@python.org>
In debug mode, visit_decref() now calls _PyObject_IsFreed() to ensure
that the object is not freed. If it's freed, the program fails with
an assertion error and Python dumps informations about the freed
object.
bpo-37758: Cut always-constant conditionals on sys.maxunicode. (GH-15302)
Since PEP 393 in Python 3.3, this value is always 0x10ffff, the
maximum codepoint in Unicode; there's no longer such a thing as a
UCS-2 build of Python, which couldn't properly represent some
characters.
There are a couple of spots left where we still condition on the value
of this constant. Take them out.
Victor Stinner [Mon, 9 Sep 2019 14:55:58 +0000 (16:55 +0200)]
bpo-38006: Avoid closure in weakref.WeakValueDictionary (GH-15641)
weakref.WeakValueDictionary defines a local remove() function used as
callback for weak references. This function was created with a
closure. Modify the implementation to avoid the closure.
Mark files as executable that are meant as scripts. (GH-15354)
This is the converse of GH-15353 -- in addition to plenty of
scripts in the tree that are marked with the executable bit
(and so can be directly executed), there are a few that have
a leading `#!` which could let them be executed, but it doesn't
do anything because they don't have the executable bit set.
Here's a command which finds such files and marks them. The
first line finds files in the tree with a `#!` line *anywhere*;
the next-to-last step checks that the *first* line is actually of
that form. In between we filter out files that already have the
bit set, and some files that are meant as fragments to be
consumed by one or another kind of preprocessor.
bpo-26185: Fix repr() on empty ZipInfo object (#13441)
* bpo-26185: Fix repr() on empty ZipInfo object
It was failing on AttributeError due to inexistant
but required attributes file_size and compress_size.
They are now initialized to 0 in ZipInfo.__init__().
* Remove useless hasattr() in ZipInfo._open_to_write()
* Completely remove file_size setting in _open_to_write().
bpo-37702: Fix SSL's certificate-store leak on Windows (GH-15632)
ssl_collect_certificates function in _ssl.c has a memory leak.
Calling CertOpenStore() and CertAddStoreToCollection(), a store's refcnt gets incremented by 2.
But CertCloseStore() is called only once and the refcnt leaves 1.
bpo-37705: Improve the implementation of winerror_to_errno() (GH-15623)
winerror_to_errno() is no longer automatically generated.
Do not rely on the old _dosmapperr() function.
Add ERROR_NO_UNICODE_TRANSLATION (1113) -> EILSEQ.
bpo-37936: Avoid ignoring files that we actually do track. (GH-15451)
There were about 14 files that are actually in the repo but that are
covered by the rules in .gitignore.
Git itself takes no notice of what .gitignore says about files that
it's already tracking... but the discrepancy can be confusing to a
human that adds a new file unexpectedly covered by these rules, as
well as to non-Git software that looks at .gitignore but doesn't
implement this wrinkle in its semantics. (E.g., `rg`.)
Several of these are from rules that apply more broadly than
intended: for example, `Makefile` applies to `Doc/Makefile` and
`Tools/freeze/test/Makefile`, whereas `/Makefile` means only the
`Makefile` at the repo's root.
And the `Modules/Setup` rule simply wasn't updated after 961d54c5c.
bpo-37445: Include FORMAT_MESSAGE_IGNORE_INSERTS in FormatMessageW() calls (GH-14462)
If FormatMessageW() is passed the FORMAT_MESSAGE_FROM_SYSTEM flag without FORMAT_MESSAGE_IGNORE_INSERTS, it will fail if there are insert sequences in the message definition.
Restart lines now always start with '=' and never end with ' ' and fill the width of the window unless that would require ending with ' ', which could be wrapped by itself and possible confusing the user.