Erik Torres [Wed, 22 Sep 2021 22:51:08 +0000 (22:51 +0000)]
ICU-21581 BRSRC 70.1 Version update and regenerate configure for v70.1
In this PR, I am updating the version number from 70.0.1 for the BRS task.
Previously, we had frontloaded part of this, so the diff in this PR are not as numerous.
It has also been decided that we should differentiate frontloaded tasks and RC tasks by having the version numbers being:
Frontload version number: XX.X.X (70.0.1 -> Major.minor.patch)
RC/GA version number: XX.X (70.1 -> Major.minor)
I've added some documentation for this, for future releases :)
Andy Heninger [Wed, 15 Sep 2021 21:51:48 +0000 (14:51 -0700)]
ICU-21662 UVector cleanup in rbtz.cpp
Revise uses of UVector in rbtz.cpp to better handle memory allocation failures.
This is one of an ongoing series of commits to address similar problems with
UVector usage throughout ICU.
The changes include
- Use LocalPointers and UVector deleter functions to simplify OOM checking and recovery.
- Fix RuleBasedTimeZone::addTransitionRule(rule) to have standard ICU adopt behavior
when errors occur, meaning automatic deletion of the incoming rule. This simplifies
both the implementation of the function and the code at the call sites.
- Update addTransitionRule() call sites. Includes modifying the Dangi calendar initializtion
to not silently ignore errors.
- struct Transition is changed to derive from UMemory, which allows the use of LocalPointers.
Steven R. Loomis [Sat, 18 Sep 2021 00:51:23 +0000 (19:51 -0500)]
ICU-21756 icu4j: port UnicodeKnownIssues.java from CLDR
- port of CLDR-14588
- use a fake Consumer<String>
- currently logs after each test class, not ideal but better
- Formerly ICU-12589 but that is not as related
- add unit test
Andy Heninger [Sat, 11 Sep 2021 23:57:03 +0000 (16:57 -0700)]
ICU-21662 UVector cleanup in basictz.cpp
Revise uses of UVector in basictz.cpp to better handle memory allocation failures.
This is one of an ongoing series of commits to address similar problems with
UVector usage throughout ICU.
The changes primarily involve switching to the use of LocalPointers for
the tracking of memory ownership, and to simplify cleanup in case of errors.
In the function BasicTimeZone::getTimeZoneRulesAfter(), also switched some additional
allocated memory to use LocalPointer or LocalMemory, for consistency in
memory handling.
Andy Heninger [Fri, 10 Sep 2021 21:59:46 +0000 (14:59 -0700)]
ICU-21662 UVector Error Handling in Regex
Clean up some oversights from commit 0ec329c. This was triggered by fuzz testing finding
a memory leak with the original commit, see https://oss-fuzz.com/testcase-detail/4656781834452992
I was unable to replicate the fuzzing failure, but reviewing the nearby code showed
some likely problems.
In this commit,
- Fix UStack::pop() to not delete the popped element when a deleter function is present.
This was a bug, but because there were no stacks with deleters, was not causing trouble.
- Change RegexCompile::compileSet() to delete the set if it cannot be added to the internal
vector of sets. I suspect this is the cause of the fuzzing leak - 0ec329c changed the
behavior of UVector in the presence of errors.
- Changed RegexCompile::fSetStack to use an object deleter function. This fixes the
leak checking at the points new elements are pushed onto this stack.
Rich Gillam [Fri, 6 Aug 2021 21:00:41 +0000 (14:00 -0700)]
ICU-13353 Fixed several problems preventing horizontal resource inheritance from working as intended, and added a
Java version of a unit test I'd previously only added on the C++ side.
ICU-21735 Fixed two old references to UNUM_CURRENCY_CASH, one in a comment, one in a unit-test failure message--
to refer to the now-correct UNUM_CASH_CURRENCY.
Andy Heninger [Thu, 2 Sep 2021 23:09:04 +0000 (16:09 -0700)]
ICU-21662 Improve UVector error handling.
This is the next installment of UVector error handling cleanup. It includes:
- Revise UStack to follow the conventions of UVector, to leave the stack
unmodified when there is an incoming error code. And, for stacks with a
deleter function, to delete the incoming object if it cannot be
succesfully pushed.
- Review all useage of UStack in ICU; adjust call sites as needed.
- Review all uses of UVector::addElementX() in the implementation of
- Regular Expressions
- Break Iteration
- toolutil/xmlparser
changing to the updated functions, and adjusting call sites as needed.
Markus Scherer [Fri, 3 Sep 2021 21:53:25 +0000 (14:53 -0700)]
ICU-21652 add emoji properties of strings
- 7 new properties: API constants & property names
- u_stringHasBinaryProperty(s, property) & UCharacter.hasBinaryProperty(s, property)
- two additional source data files
- new genprops part for writing new binary data file uemoji.icu
- data for existing emoji properties moved from uprops.icu (hardcoded in C++) to uemoji.icu (always loaded)
- new EmojiProps implementation
Rich Gillam [Tue, 17 Aug 2021 23:28:32 +0000 (16:28 -0700)]
ICU-21460 Changed the ULocale initializers to allow locale IDs that use BCP47 syntax, but with '_' as a field delimiter.
(APIs that specifically require BCP47 syntax are unaffected-- they still require '-').
Andy Heninger [Wed, 28 Jul 2021 18:20:53 +0000 (11:20 -0700)]
ICU-21662 Improve UVector error handling.
- Add updated versions of UVector::addElement() and ensureCapacity() that respect
incoming errors.
Follow on to c26aebe, which renamed the original versions.
- Add UVector::adoptElement() as a replacement for addElement() when the vector
has a deleter function set, meaning that it adopts ownership of its elements.
The intent is to make the behavior clearer at the call sites when looking
at unfamiliar code.
- Make all functions with an incoming failure, as indicated by a UErrorCode parameter,
leave the vector unchanged.
- Change all functions that store object pointers into the vector such that,
when the store cannot be completed for any reason _and_ the vector has a deleter function,
then the incoming object is deleted.
This change can simplify the error handling code around calls to the affected functions
(addElement() and insertElementAt(), in particular)
- Add index bounds checking on functions where it was possible - that is, on functions
with both U_ErrorCode and index parameters.
- Changed to more modern C++ idioms in some parts of the UVector implementation.
- Review & update as required all uses of the UVector functions
setElementAt(), insertElementAt(), setSize(), sortedInsert()
these being the functions with changed behavior on error conditions
(aside from addElement()).
This PR will be followed by more, switching call sites in various ICU services
from UVector::addElementX() (old behavior on errors)
to UVector::addElement() (new behavior on errors)
Markus Scherer [Thu, 26 Aug 2021 21:44:49 +0000 (14:44 -0700)]
ICU-20769 refactor UResourceBundle.fResData & init_resb_result()
- replace UResourceBundle.fResData with access via fData
- change ResourceDataValue.resData into a pointer
- move C tests to C++ that stack-allocate now-C++ UResourceBundle
- init_resb_result():
- remove redundant ResourceData parameter
- simplify memory management, and various small simplifications
- /LOCALE/path reuse valid locale, no ures_open()
- pull alias-fetching code into separate function getAliasTargetAsResourceBundle()
- `const char *` for keyPath pieces
- s/noAlias/recursionDepth/
- getAllItemsWithFallback() set correct parentBundle.fValidLocaleDataEntry
Rich Gillam [Sat, 28 Aug 2021 00:03:22 +0000 (17:03 -0700)]
ICU-21671 Corrected a bug in SimpleDateFormat.subParse() that was causing us to always parse quarter names that begin
with numbers as though the number was the whole quarter name.
Jeff Genovy [Fri, 27 Aug 2021 02:23:16 +0000 (19:23 -0700)]
ICU-21710 Additional clean up after removing BOYER_MOORE code from usearch.cpp
Changes:
- We can completely remove the shift tables and related fields from
data structs.
- Creation of UStringSearch objects should be faster now,
as it doesn't waste time computing the unused shift tables.
- The sizeof(UStringSearch) is decreased from 5240 to 3192, so
this should help to reduce memory for applications that create many string search objects.
Note regarding the comments on initialize(). It actually does not set illegal argument error if pattern is all ignoreables. Added a test case for it.
Fredrik Roubert [Thu, 26 Aug 2021 13:15:30 +0000 (15:15 +0200)]
ICU-20973 Use standard keywords true & false to initialize type bool.
Now when all equality operators return standard bool (commit 633438f),
it no longer makes any sense to use the ICU4C constants TRUE & FALSE
or local variables of type UBool for their return value.
Fredrik Roubert [Wed, 25 Aug 2021 16:11:20 +0000 (18:11 +0200)]
ICU-20973 Rewrite polymorphic CacheKeyBase equality operators for C++20.
The existing polymorphic equality operators that use different types for
the `this` and `other` objects are ambiguous with C++20 resolution rules
that require equality for reversed arguments.
In order to resolve that, while also possibly making the implementation
somewhat simpler overall, the implementation classes (LocaleCacheKey
and DateFmtBestPatternKey) now get normal (non-polymorphic) equality
operators that are trivially non-ambiguous (and as a bonus also don't
need any type casts), while the dynamic type checking logic is moved
into protected helper functions, which in the end are invoked
(without any ambiguity) by friend operators in the base class.
This way, all equality testing of cache key objects ends up taking one
of these two possible paths:
1. Both sides of the equality operator are of the same implementation
type (ie. LocaleCacheKey or DateFmtBestPatternKey):
The type specific equality operator is called directly, comparing the
relevant attributes of the two objects directly.
2. The two sides of the equality operator are either of different types
or of some base class type:
The friend equality operators of CacheKeyBase call the virtual helper
function to figure out whether the two objects are actually of the
same type and if they are and this type is an implementation type
then does the necessary type cast to get to 1.