Fix #80266: parse_url silently drops port number 0
As of commit 81b2f3e[1], `parse_url()` accepts URLs with a zero port,
but does not report that port, what is wrong in hindsight.
Since the port number is stored as `unsigned short` there is no way to
distinguish between port zero and no port. For BC reasons, we thus
introduce `parse_url_ex2()` which accepts an output parameter that
allows that distinction, and use the new function to fix the behavior.
The introduction of `parse_url_ex2()` has been suggested by Nikita.
Nikita Popov [Wed, 4 Nov 2020 13:51:44 +0000 (14:51 +0100)]
Assert that references are not persisted
There should not be any need to persist references, and it's unlikely
that persisting a reference will behave correctly at runtime, because
we don't have a concept of an immutable reference.
Nikita Popov [Tue, 3 Nov 2020 15:45:13 +0000 (16:45 +0100)]
Don't disable early binding during preloading script
We should only disable early binding during the opcache_compile_file()
calls, not inside the preloading script or anything it includes.
The right condition to check for is whether we compile the file
without execution, as declaring classes is "execution".
Nikita Popov [Tue, 3 Nov 2020 13:15:05 +0000 (14:15 +0100)]
Fix variance checks on resolved union types
This is a bit annoying: When preloading is used, types might be
resolved during inheritance checks, so we need to deal with CE
types rather than just NAME types everywhere.
Nikita Popov [Tue, 3 Nov 2020 10:50:14 +0000 (11:50 +0100)]
Don't ignore internal classes during preloading
When preloading, it's fine to make use of internal class information,
as we do not support Windows. It is also necessary to allow proper
variance checks against internal classes.
Alex Dowad [Sun, 18 Oct 2020 15:49:57 +0000 (17:49 +0200)]
Fix mbstring support for ARMSCII-8
- Identify filter was completely wrong.
- Respect `mb_substitute_character` rather than converting invalid bytes to
Unicode 0xFFFD (generic replacement character).
- Don't convert Unicode 0xFFFD to a valid ARMSCII-8 character.
- When converting ARMSCII-8 to ARMSCII-8, don't pass invalid bytes through
silently.
Alex Dowad [Sat, 29 Aug 2020 17:12:28 +0000 (19:12 +0200)]
Optimize (AND FIX) mb_check_encoding (cut execution time by 50%+)
Previously, `mb_check_encoding` did an awful lot of unneeded work. In order to
determine whether a string was valid or not, it would convert the whole string
into wchar (code points), which required dynamically allocating a (potentially
large) buffer. Then it would turn right around and convert that big 'ol buffer
of code points back to the original encoding again. Finally, it would check
whether any invalid bytes were detected during that long and onerous process.
The thing is, mbstring _already_ has machinery for detecting whether a string
is valid in a certain encoding or not, and it doesn't require copying any data
around or allocating buffers. Better yet, it can fail fast when an invalid byte
is found. Why not use it? It's sure a lot faster!
Further, the legacy code was also badly broken. Why? Because aside from
checking whether illegal characters were detected, it would also check whether
the conversion to and from wchars was lossless. But, some encodings have
more than one valid encoding for the same character. In such cases, it is
not possible to make the conversion to and from wchars lossless for every
valid character. So `mb_check_encoding` would actually reject good strings
in a lot of encodings!
Alex Dowad [Sun, 18 Oct 2020 13:30:03 +0000 (15:30 +0200)]
Fix mbstring support for CP1254 encoding
One funny thing: while the original author used Unicode 0xFFFD (generic
replacement character) for invalid bytes in CP1251 and CP1252, for CP1254
they used 0xFFFE, which is not a valid Unicode codepoint at all, but is a
reversed byte-order mark. Probably this was by mistake.
Anyways,
- Fixed identify filter, which was completely wrong.
- Don't convert Unicode 0xFFFE to a random (but valid) CP1254 byte.
- When converting CP1254 to CP1254, don't pass invalid bytes through silently.
Alex Dowad [Sun, 18 Oct 2020 13:12:11 +0000 (15:12 +0200)]
Fix mbstring support for CP1251 encoding
- Identify filter was as wrong as wrong can be.
- Invalid CP1251 byte 0x98 was converted to Unicode 0xFFFD (generic
replacement character), rather than respecting `mb_substitute_character`.
- Unicode 0xFFFD was converted to some random CP1251 byte.
- When converting CP1251 to CP1251, don't pass invalid bytes through silently.
Anatol Belski [Sun, 30 Aug 2020 12:14:04 +0000 (14:14 +0200)]
hash: Add MurmurHash3 with streaming support
The implementation is based on the upstream PMurHash. The following
variants are implemented
- murmur3a, 32-bit hash
- murmur3c, 128-bit hash for x86
- murmur3f, 128-bit hash for x64
The custom seed support is not targeted by this implementation. It will
need a major change to the API, so then custom arguments can be passed
through `hash_init`. For now, the starting hash is always zero.
Fixes bug #68109, closes #6059
Signed-off-by: Anatol Belski <ab@php.net> Co-Developed-by: Michael Wallner <mike@php.net> Signed-off-by: Michael Wallner <mike@php.net>
Alex Dowad [Sun, 18 Oct 2020 12:41:40 +0000 (14:41 +0200)]
Add test suite for CP1252 encoding
Also remove a bogus test (bug62545.phpt) which wrongly assumed that all invalid
characters in CP1251 and CP1252 should map to Unicode 0xFFFD (REPLACEMENT
CHARACTER).
mbstring has an interface to specify what invalid characters should be
replaced with; it's called `mb_substitute_character`. If a user wants to see
the Unicode 'replacement character', they can specify that using
`mb_substitute_character`. But if they specify something else, we should
follow that.
Alex Dowad [Sun, 18 Oct 2020 12:56:32 +0000 (14:56 +0200)]
Fix mbstring support for CP1252 encoding
It's a bit surprising how much was broken here.
- Identify filter was utterly and completely wrong.
- Instead of handling invalid CP1252 bytes as specified by
`mb_substitute_character`, it would convert them to Unicode 0xFFFD
(generic replacement character).
- When converting ISO-8859-1 to CP1252, invalid ISO-8859-1 bytes would
be passed through silently.
- Unicode codepoints from 0x80-0x9F were converted to CP1252 bytes 0x80-0x9F,
which is wrong.
- Unicode codepoint 0xFFFD was converted to CP1252 0x9F, which is very wrong.
Also clean up some unneeded code, and make the conversion table consistent with
others by using zero as a 'invalid' marker, rather than 0xFFFD.
Alex Dowad [Thu, 29 Oct 2020 21:10:04 +0000 (23:10 +0200)]
Convert numeric string array keys to integers correctly in JITted code
While fixing bugs in mbstring, one of my new test cases failed with a strange
error message stating: 'Warning: Undefined array key 1...', when clearly the
array key had been set properly.
GDB'd that sucker and found that JIT'd PHP code was calling directly into
`zend_hash_add_new` (which was not converting the numeric string key to an
integer properly). But where was that code coming from? I examined the disasm,
looked up symbols to figure out where call instructions were going, then grepped
the codebase for those function names. It soon became clear that the disasm I
was looking at was compiled from `zend_jit_fetch_dim_w_helper`.
Nikita Popov [Wed, 21 Oct 2020 13:01:47 +0000 (15:01 +0200)]
Add --repeat testing mode
This testing mode executes the test multiple times in the same
process (but in different requests). It is primarily intended to
catch tracing JIT bugs, but also catches state leaks across
requests.
Nikita Popov [Fri, 30 Oct 2020 10:11:16 +0000 (11:11 +0100)]
Fixed bug #80290
Dropping the dtor arg args[3] rather than using STR_COPY: Since
PHP 8, we no longer support separation in call_user_function(),
so we also don't need to worry about things like arguments being
replaced with references.