Alex Dowad [Sun, 18 Oct 2020 13:30:03 +0000 (15:30 +0200)]
Fix mbstring support for CP1254 encoding
One funny thing: while the original author used Unicode 0xFFFD (generic
replacement character) for invalid bytes in CP1251 and CP1252, for CP1254
they used 0xFFFE, which is not a valid Unicode codepoint at all, but is a
reversed byte-order mark. Probably this was by mistake.
Anyways,
- Fixed identify filter, which was completely wrong.
- Don't convert Unicode 0xFFFE to a random (but valid) CP1254 byte.
- When converting CP1254 to CP1254, don't pass invalid bytes through silently.
Alex Dowad [Sun, 18 Oct 2020 13:12:11 +0000 (15:12 +0200)]
Fix mbstring support for CP1251 encoding
- Identify filter was as wrong as wrong can be.
- Invalid CP1251 byte 0x98 was converted to Unicode 0xFFFD (generic
replacement character), rather than respecting `mb_substitute_character`.
- Unicode 0xFFFD was converted to some random CP1251 byte.
- When converting CP1251 to CP1251, don't pass invalid bytes through silently.
Anatol Belski [Sun, 30 Aug 2020 12:14:04 +0000 (14:14 +0200)]
hash: Add MurmurHash3 with streaming support
The implementation is based on the upstream PMurHash. The following
variants are implemented
- murmur3a, 32-bit hash
- murmur3c, 128-bit hash for x86
- murmur3f, 128-bit hash for x64
The custom seed support is not targeted by this implementation. It will
need a major change to the API, so then custom arguments can be passed
through `hash_init`. For now, the starting hash is always zero.
Fixes bug #68109, closes #6059
Signed-off-by: Anatol Belski <ab@php.net> Co-Developed-by: Michael Wallner <mike@php.net> Signed-off-by: Michael Wallner <mike@php.net>
Alex Dowad [Sun, 18 Oct 2020 12:41:40 +0000 (14:41 +0200)]
Add test suite for CP1252 encoding
Also remove a bogus test (bug62545.phpt) which wrongly assumed that all invalid
characters in CP1251 and CP1252 should map to Unicode 0xFFFD (REPLACEMENT
CHARACTER).
mbstring has an interface to specify what invalid characters should be
replaced with; it's called `mb_substitute_character`. If a user wants to see
the Unicode 'replacement character', they can specify that using
`mb_substitute_character`. But if they specify something else, we should
follow that.
Alex Dowad [Sun, 18 Oct 2020 12:56:32 +0000 (14:56 +0200)]
Fix mbstring support for CP1252 encoding
It's a bit surprising how much was broken here.
- Identify filter was utterly and completely wrong.
- Instead of handling invalid CP1252 bytes as specified by
`mb_substitute_character`, it would convert them to Unicode 0xFFFD
(generic replacement character).
- When converting ISO-8859-1 to CP1252, invalid ISO-8859-1 bytes would
be passed through silently.
- Unicode codepoints from 0x80-0x9F were converted to CP1252 bytes 0x80-0x9F,
which is wrong.
- Unicode codepoint 0xFFFD was converted to CP1252 0x9F, which is very wrong.
Also clean up some unneeded code, and make the conversion table consistent with
others by using zero as a 'invalid' marker, rather than 0xFFFD.
Alex Dowad [Thu, 29 Oct 2020 21:10:04 +0000 (23:10 +0200)]
Convert numeric string array keys to integers correctly in JITted code
While fixing bugs in mbstring, one of my new test cases failed with a strange
error message stating: 'Warning: Undefined array key 1...', when clearly the
array key had been set properly.
GDB'd that sucker and found that JIT'd PHP code was calling directly into
`zend_hash_add_new` (which was not converting the numeric string key to an
integer properly). But where was that code coming from? I examined the disasm,
looked up symbols to figure out where call instructions were going, then grepped
the codebase for those function names. It soon became clear that the disasm I
was looking at was compiled from `zend_jit_fetch_dim_w_helper`.
Nikita Popov [Wed, 21 Oct 2020 13:01:47 +0000 (15:01 +0200)]
Add --repeat testing mode
This testing mode executes the test multiple times in the same
process (but in different requests). It is primarily intended to
catch tracing JIT bugs, but also catches state leaks across
requests.
Nikita Popov [Fri, 30 Oct 2020 10:11:16 +0000 (11:11 +0100)]
Fixed bug #80290
Dropping the dtor arg args[3] rather than using STR_COPY: Since
PHP 8, we no longer support separation in call_user_function(),
so we also don't need to worry about things like arguments being
replaced with references.
divinity76 [Tue, 27 Oct 2020 15:19:58 +0000 (16:19 +0100)]
Use constant size string in hash bench.php
I don't like the previous behaviour where the bytes to hash change
every time the code changes, that may make it difficult to compare
hash() performance changes over time.
Use a fixed number instead, and allow passing an override for a
different length.
Fix bug #72413: Segfault with get_result and PS cursors
We cannot simply switch to use_result here, because the fetch_row
methods in get_result mode and in use_result/store_result mode
are different: In one case it accepts a statement, in the other
a return value zval. Thus, doing a switch to use_result results
in a segfault when trying to fetch a row.
Actually supporting get_result with cursors would require adding
cursor support in mysqlnd_result, not just mysqlnd_ps. That would
be a significant amount of effort and, given the age of the issue,
does not appear to be particularly likely to happen soon.
As such, we simply generate an error when using get_result()
with cursors, which is much better than causing a segfault.
Instead, parameter binding needs to be used.
Nikita Popov [Thu, 29 Oct 2020 13:07:08 +0000 (14:07 +0100)]
Handle errors during PDO row fetch
The EOF flag also gets set on error, so we always end up ignoring
errors here.
However, we should only check errors for unbuffered results. For
buffered results, this function is guaranteed not to error, and
querying the errno may return an unrelated error.
Fix #44618: Fetching may rely on uninitialized data
Unless `SQLGetData()` returns `SQL_SUCCESS` or `SQL_SUCCESS_WITH_INFO`,
the `StrLen_or_IndPtr` output argument is not guaranteed to be properly
set. Thus we handle retrieval failure other than `SQL_ERROR` by
yielding `false` for those column values and raising a warning.
Nikita Popov [Wed, 28 Oct 2020 16:12:35 +0000 (17:12 +0100)]
Fixed bug #65825
Set error_info when we fail to read a packet, instead of throwing
a warning. Additionally we also need to populate the right
error_info in rowp_read -- we'll later take the error from the
packet, not the connection.
No test case, as this is hard to reliably test. I'm using the
test case from:
https://github.com/php/php-src/pull/2131#issuecomment-538374838
projects / php / log