Use constants referring to the signed integer types when setting SSIZE_T_MAX.
In practice, the sizes of the signed and unsigned integer types will
almost cetainly be the same, but this is more correct.
Pointed out in issue #638.
In the json_tokener_state_number case, explicitly adjust what "number" characters are allowed based on the exact micro-state that we're in, and check for invalid following characters in a different way, to allow a valid json_type_number object to be returned at the top level.
This causes previously failing strings like "123-456" to return a valid json_object with the appropriate value. If you care about the trailing content, call json_tokener_parse_ex() and check the parse end point with json_tokener_get_parse_end().
Fix incremental parsing of invalid numbers with exponents, such as "0e+-" and "12.3E12E12", while still allowing "0e+" in non-strict mode.
Deprecate the json_parse_double() function from json_util.h
Issue #616: Change the parsing of surrogate pairs in unicode escapes so it uses a couple of additional states instead of assuming the low surrogate is already present, to ensure that we correctly handle various cases of incremental parsing.
In test_parse, fix lengths passed during a couple of incremental tests.
Add an optional way to use an incremental chunk size ($TEST_PARSE_CHUNKSIZE)
for all of the single_basic_parse tests, in additional to the regular way.
Rearrange the json_tokener_state_escape_unicode case in json_tokener to simplify the code slightly and make it a bit easier to understand.
While here, drop the utf8_replacement_char that is unnecesarily added if we run out of input in the middle of a unicode escape. No other functional changes (yet).
Add json_object_array_shrink() (and array_list_shrink()) and use it in json_tokener to minimize the amount of memory used. This results in a 39%-50% reduction in memory use (peak RSS, peak heap usage) on the jc-bench benchmark and 9% shorter runtime.
Also add the json_object_new_array_ext, array_list_new2, and array_list_shrink functions.
Micah Snyder [Sun, 14 Jun 2020 16:01:48 +0000 (12:01 -0400)]
Issue #508: `-fPIC` to link libjson-c.a with libs
json-c has symbol collisions with other popular C JSON libraries.
To prevent random crashes in downstream applications that may use a
library which depends on json-c, the solution is to statically link
libjson-c.a into those libraries.
`-fPIC`/`-fPIE` is required when building a `.a` so it can be linked
into a `.so`.
Drop the _delete field from struct json_object and call the type-specific delete functions directly from json_object_put. (Thanks @dota17 for the suggestion in PR #632!)
The split of json_object into type-specific sub-structures is now functionally complete.
Remove all of the temporary defines, structures, old compat fuctions, extra fields, etc... that were needed during the conversion to a split set of json_object_* structures.
Kick json_type_string out of struct json_object.
The default is now that string data is stored inline at the end of json_object, though to allow for json_object_set_string() to set a _longer_ string, we still need to allow for the possibility of a separate char * pointer.
All json types have been split out now, next step it cleanup.
Kick json_type_int and json_type_double out of struct json_object.
Clean up the code in json_object_new_* a bit by dropping unnecesary json_object_base variables.
Start splitting struct json_object into multiple sub-types, as descibed at https://github.com/json-c/json-c/wiki/Proposal:-struct-json_object-split
The current changes split out _only_ json_type_object, and thus have a number of hacks
to allow the code to continue to build and work.
Originally mentioned in issue #535.
When complete, this will probably invalidate #552.
This is likely to cause notable conflicts in any other significant un-merged
changes, such as PR#620.
A custom double formatter can lead to truncation of the rest of the
JSON document.
If a custom formatter completely fills the buffer used by snprintf
with a trailing dot or comma and the formatting option
JSON_C_TO_STRING_NOZERO has been specified, then an iterator moves
past the ending '\0' (off-by-one buffer overflow) to set an
additional '\0' and adds the first '\0' into the printbuf.
Since '\0' will eventually be considered the terminating character
of the complete printbuf result, all trailing characters are lost.
This leads to an incomplete JSON string as can be seen with the
test case.
The off-by-one can be noticed if compiled with address sanitizer.
Since this is a very special case and a malformed formatter could
do way more harm and is the responsibility of the user of this
library, this is just a protective measure to keep json-c code as
robust as possible.
Revert part of PR#606 and use isnan/isinf again, but provide macro implementations of those in math_compat.h is needed, as it seems to be on AIX and IBM i systems.
The data structures linkhash and printbuf are limited to 2 GB in size
due to a signed integer being used to track their current size.
If too much data is added, then size variable can overflow, which is
an undefined behaviour in C programming language.
Assuming that a signed int overflow just leads to a negative value,
like it happens on many sytems (Linux i686/amd64 with gcc), then
printbuf is vulnerable to an out of boundary write on 64 bit systems.
Protect array_list_del_idx against size_t overflow.
If the assignment of stop overflows due to idx and count being
larger than SIZE_T_MAX in sum, out of boundary access could happen.
It takes invalid usage of this function for this to happen, but
I decided to add this check so array_list_del_idx is as safe against
bad usage as the other arraylist functions.
Issue #589: drop the rdrand test loops to just 3, tweak comments and add some links to bug reports, and decrease the nesting level of the has_rdrand() function.
Tudor Brindus [Sat, 2 May 2020 01:09:22 +0000 (21:09 -0400)]
Detect broken RDRAND during initialization
Some CPUs advertise RDRAND in CPUID, but return 0xFFFFFFFF
unconditionally. To avoid locking up later, test RDRAND during
initialization, and if it returns 0xFFFFFFFF, mark it as nonexistent.
Extend the CMakeLists.txt in the apps directory to be usable as a standalone build, to link against other versions of json-c.
Tweak json_parse.c slightly to allow it to build against older json-c versions.
Add an apps directory, and a json_parse program to parse an input file and report on memory usage.
This is intended to provide a way, during development, to test out the memory
and performance impacts of a change.
Add a JSON_TOKENER_ALLOW_TRAILING_CHARS flag for json_tokener_set_flags() to allow multiple objects to be parsed from input even when JSON_TOKENER_STRICT is set.
projects / json-c / log