Cherry-pick cldrbug 11492: Bad symbols for NaN in sv, ksh, kl, se locales.
Cherry-pick cldrbug 11491: sd, month name for July uses character not in exemplars.
Updated the various ICU4J *.jar files as well.
- Changes Java DecimalFormat boolean get* methods to is*.
- Makes the new draft methods non-virtual.
- Removes obsolete template class in header file.
- Adds proper U_HIDE tags in unum.h and decimfmt.h
- Adds additional logic to NumberStringBuilder.
- Extends logic of number::impl::Field type.
- Adds tests for RBNF support.
- Adds tests from ftang's original PR.
- widen API from LocalePriorityList to Iterable
- merge getBestMatch(multiple locales) and getBestMatch(single locale) into one function
- process desired locales incrementally, create fewer objects
- reject poor matches early: use bestDistance-demotion for threshold
- add API for java.util.Locale, convert incrementally
- new feature: tracks indexes of supported and desired locales which eliminates conversion of result objects in wrappers around getBestMatch() as shown by the java.util.Locale API here
- simpler data structures, more serialization-friendly (easier to port to C++)
- e.g., use a BytesTrie each for likelySubtags & locale distance, instead of layers of TreeMap
- un-hardcode locale matcher data; use modern resource bundle functions
- split builder code & runtime code into separate classes
- move LSR to simple top-level value class, cache regionIndex in LSR
- simpler handling of private use languages and pseudolocales
- simplify RegionMapper
- LocaleDistance builder: move the node distance into the DistanceTable, remove DistanceNode
- support distance rules with region codes, not just with variables
- enforce & use distance rule constraints:
- no rule with *,supported or desired,*
- no rule with language * and script/region non-*
- distance trie collapse a (desired, supported)=(ANY, ANY) pair into a single *
- look up each desired language only once for all supported LSRs
- remove layers-of-Maps compaction (trie builder compacts)
- remove unused XML printing
- remove other unused code
- make XLocaleMatcherTest.testPerf() exercise locale distance lookup code
- Wires up FormattedNumber[Range] in applicable languages.
- Adds new header files and tests, with minor cleanup to old tests.
- Adds code to guarantee terminating NUL in FormattedNumber[Range].
- Cleanup of API docs for inherited methods in FormattedNumber[Range].
Adds some plumbing to allow MutablePatternModifier to set fields, and otherwise builds upon the infrastructure from the previous commit to add the MEASURE_UNIT field.
- fastpath for UnicodeSet.add(new last range)
- fewer UnicodeSet memory allocations:
initial internal list array, exponential array growth,
allocate strings list/set only when first one is added
- faster CodePointTrie.getRange(): fewer calls to filter function
- revert UnicodeSet(intprop=value) from trie ranges to range starts + lookup
- cache per-int-prop range starts: fewer lookups
* ICU-20205 Add locale test for RelativeDateTimeFormatter.
* ICU-20205 Fix error in pt relative date data. Improve error handling in code.
* ICU-20205 Add instantiation test & regen data from ICU4C
* ICU-20205 Added DateFormatSymbols error check per jefgen's comments.
* Add Automatic-Module-Name to META-INF/MANIFEST.MF
* Added com.ibm.icu.number in Export-Package list
* Add Automatic-Module-Name to charset and localespi
- Updated API status of java.lang.Object method overrides (equals/hashCode/toString/clone) to match status of declaring class. There are some API comments that is insufficient, or incorrect, or not appropriate after the change. Most of these issues were fixed.
- APIStatusConsistencyChecker and ant tasks invoking the check, designed for checking this requirement. For now, Normalizer#clone() does not satisfy the requirement, but unchanged on purpose. The tool can accept exception.
- DeprecatedAPIChecker had a problem for handling non-static inner class's constructor. CodePointMap$StringIterator is the very first instance of such class in ICU4J. The problem was fixed by removing the implicit param scanned by reflection.
- This commit includes a lot of changes made by Eclipse project configuration - removing spaces in blank lines/end of statement, and removeal of redundant generics type declaration.
* ICU-20140 Allow duplicated keys in U-extension per RFC 6067
RFC 6067 [1] does allow duplicate keywords, but ICU4C's
uloc_forLanguageCode rejects it as invalid.
Change it to accept duplicate keywords and honor only the
1st one while ignoring subsequent ones per RFC 6067.
[1] Unicode extension to BCP 47:
https://tools.ietf.org/html/rfc6067
* ICU-20140 Add ICU4J test and tweak ICU4C test
ICU4J test diverges from ICU4C tests:
1. Handling of duplicate variants in ICU4J seem to be wrong:
https://unicode-org.atlassian.net/browse/ICU-20148
2. ULocale.forLanguageTag only throws NullPointException so
that ICU4C's test for duplicate attributes cannot be ported.
- Follow the spec to calculate the mimimum significant digits in engineering notation
- The bug is regression since ICU 58. The new test still passes on
ICU58-based DecimalFormat
- Maximum significant digits is not changed
* Updated era data format in supplementalData.
* Include tentative era names in data. Implemented Japanese era loaded from CLDR data in ICU4J.
* ICU4C implementation, ICU4C refactoring. WIP.
* VS project updates and some bug fixes
Also added API comments.
* Review feedback and bug fixes
- NULL to nullptr
- use of LocalUResourceBundlePointer
- TYPO "name" to "named"
- env var checking stricmp() == 0
* API comment correction based on feedback
* Duplicate the comment in ucal.h to calendar.h
* Fixed spelling errors in API comment
* ICU-20098 Fix the validty check for extlang in uloc_forLanguageTag
BCP 47 has the following for language. extlang subtag can only be
preceded by 2*3ALPHA. Add a check for the length of language subtag
before extlang subtag.
language = 2*3ALPHA ; shortest ISO 639 code
["-" extlang] ; sometimes followed by
; extended language subtags
/ 4ALPHA ; or reserved for future use
/ 5*8ALPHA ; or registered language subtag
extlang = 3ALPHA ; selected ISO 639 codes
*2("-" 3ALPHA) ; permanently reserved}}
With this change, 'hant-cmn-CN' would drop '-cmn-CN' keeping only
'hant'.
* ICU-20098 Fix the validty check for extlang for ICU4J
* ICU-20098 Fix the compiler failure for ICU4J
* ICU-20098 Fix a compile error and test.
* ICU-20098 Add a test for invalid private use singleton
ICU4C's check for private use singleton subtag ('x') is wrong and
treats invalid language tags as valid.
ICU4J's check is correct and does not require any change.
Fix that and add tests to both ICU4C and ICU4J.
By extracting this test case into a separate class and separating the
complex and time consuming building of the UnicodeSet disorderedMarks
data structure into an @BeforeClass shared setup method it becomes
possible to test the 18 different transliteration rules as a
parameterized test case.
This will lower the running time per test case and also aid in debugging
as it will make it immediately obvious which transliteration rules cause
test failures (and which don't).
When running in Android, java.util.TimeZone.getDefault() will only
return a TimeZone object for a time zone that actually exists, so when
passing a bogus test TimeZone object to setDefault() and afterwards
calling getDefault() it won't return that same time zone.
Changing our tests to instead use a real existing (but otherwise unused)
time zone instead makes the tests work in Android as well.
Neither of these tests was testing the ability to set an arbitrarily
made up time zone as the default, so no test coverage is lost.
* ICU-13530 test & fix cptrie.getRange() when small null data block matches the start of a non-null fast data block
* ICU-13530 fix test bug
* ICU-13530 test & fix bug calculating worst-case data array capacity at start of compaction
* ICU-13530 docs: clarify buildImmutable() discards upper bits, trie then returns unsigned ints; range iteration slower than if ranges were stored directly
* ICU-13530 accept feedback on docs
Different implementations of BigDecimal.stripTrailingZeros(), in
different versions of the JDK (and different versions of Android), have
differences in their handling of zero. To avoid this, ICU4J can return
BigDecimal.ZERO for any value that is equal to zero, instead of calling
BigDecimal.stripTrailingZeros() in this problematic case.
* ICU-13530 copy C/C++ files UTrie2 -> UTrie3
X-SVN-Rev: 40754
* ICU-13530 UTrie3 new files copied from UTrie2: rename types/functions/macros
X-SVN-Rev: 40755
* ICU-13530 debug-print building each UTrie2
X-SVN-Rev: 40756
* ICU-13530 remove two-byte-UTF-8 errorValue block; move highValue from end of data array into header; add errorValue to header
X-SVN-Rev: 40762
* ICU-13530 UTrie3 U16_NEXT/PREV: errorValue for unpaired surrogates
X-SVN-Rev: 40763
* ICU-13530 no more separate values for lead surrogate code units
X-SVN-Rev: 40764
* ICU-13530 change from 11:5 trie bits to 10:6 for simpler UTF-8 code
X-SVN-Rev: 40766
* ICU-13530 UTrie2 build UTrie3 as well, print sizes
X-SVN-Rev: 40767
* ICU-13530 debug-print countSame, sumOverlaps, countInitial
X-SVN-Rev: 40768
* ICU-13530 debug-print whether trie is for CanonIterData
X-SVN-Rev: 40769
* ICU-13530 no index-shift for BMP data, no separate index-2 for 2-byte UTF-8; builder changes incomplete
X-SVN-Rev: 40777
* ICU-13530 remove errorValue and highStart from UNewTrie3
X-SVN-Rev: 40778
* ICU-13530 rewrite UTrie3 builder code
X-SVN-Rev: 40783
* ICU-13530 UTrie3 bug fixes
X-SVN-Rev: 40788
* ICU-13530 fully re-inline _UTRIE3_U8_NEXT()
X-SVN-Rev: 40790
* ICU-13530 find most common all-same data block for dataNullBlock and initialValue
X-SVN-Rev: 40792
* ICU-13530 UTrie3 iterator functions take start and return the end of a range, rather than callback call for each range
X-SVN-Rev: 40800
* ICU-13530 mask off unused data value bits before building a UTrie3 with values less than 32 bits wide
X-SVN-Rev: 40803
* ICU-13530 split utrie3builder.h out of utrie3.h
X-SVN-Rev: 40804
* ICU-13530 separate types UTrie3 vs. UTrie3Builder, implement builder as wrapper over C++ class Trie3Builder in .cpp
X-SVN-Rev: 40809
* ICU-13530 function to make a UTrie3Builder from a UTrie3
X-SVN-Rev: 40810
* ICU-13530 debug-print some data; some cleanup
X-SVN-Rev: 40865
* ICU-13530 BMP 10:6 but supplementary 10:6:4
X-SVN-Rev: 40984
* ICU-13530 move errorValue & highValue to the end of the data table, minimal padding to 4 bytes
X-SVN-Rev: 41011
* ICU-13530 index-1 table gap of index-2 null blocks
X-SVN-Rev: 41018
* ICU-13530 test with more than 128k compacted data
X-SVN-Rev: 41034
* ICU-13530 supplementary bits 11:5:4 saves a little space
X-SVN-Rev: 41039
* ICU-13530 supplementary bits 6:5:5:4 instead of gap: about same size but simpler
X-SVN-Rev: 41050
* ICU-13530 remove unnecessary utrie3_clone(built trie)
X-SVN-Rev: 41058
* ICU-13530 remove unnecessary UTrie3StringIterator
X-SVN-Rev: 41059
* ICU-13530 back to UTRIE3_GET...() macros *returning* data values
X-SVN-Rev: 41060
* ICU-13530 fast vs. small
X-SVN-Rev: 41066
* ICU-13530 always load NFC data, add simple normalization performance test
X-SVN-Rev: 41110
* ICU-13530 change normalization main trie to UTrie3 with special values for lead surrogates; forbid non-inert surrogate code *points* because unable to store values different from code *units*; runtime code work around that for code point lookup and iteration; adjust UTS 46 for normalization no longer mapping unpaired surrogates to U+FFFD
X-SVN-Rev: 41122
* ICU-13530 simplenormperf bug fix and NFC base line
X-SVN-Rev: 41126
* ICU-13530 move normalization getRange skipping lead surrogates to API getRangeSkipLead()
X-SVN-Rev: 41182
* ICU-13530 switch CanonIterData and gennorm2 Norms to UTrie3
X-SVN-Rev: 41183
* ICU-13530 remove unused overwrite parameter from setRange()
X-SVN-Rev: 41184
* ICU-13530 getRange skip lead -> fixed surrogates
X-SVN-Rev: 41219
* ICU-13530 minor cleanup
X-SVN-Rev: 41221
* ICU-13530 UTS 46 code map unpaired surrogates to U+FFFD before normalization
X-SVN-Rev: 41224
* ICU-13530 minor internal-docs cleanup
X-SVN-Rev: 41225
* ICU-13530 rename UTrie3 to UCPTrie, and other name changes
X-SVN-Rev: 41226
* ICU-13530 add 8-bit data option; add type-any & valueBits-any for fromBinary(); macros consistently source type then data width
X-SVN-Rev: 41234
* ICU-13530 scrub the API docs for the proposal
X-SVN-Rev: 41319
* ICU-13530 tag internal definitions as such, or move them to an internal header
X-SVN-Rev: 41320
* ICU-13530 Java API skeleton
X-SVN-Rev: 41326
* ICU-13530 API feedback: ValueWidth, MutableCodePointTrie, base CodePointMap, ...
X-SVN-Rev: 41382
* ICU-13530 add UCPTrie valueWidth field and padding, and combine data pointers into a union
X-SVN-Rev: 41408
* ICU-13530 switch some macros to using dataAccess parameter: separate index vs. data lookups, no macro variant for each value width
X-SVN-Rev: 41409
* ICU-13530 StringIterator is no longer a java.util.Iterator (bad fit)
X-SVN-Rev: 41455
* ICU-13530 CodePointTrie.java code complete
X-SVN-Rev: 41518
* ICU-13530 finish Java port incl test; keep C++ parallel
* ICU-13530 adjust API for feedback: rename HandleValue to FilterValue, change getRange+getRangeFixedSurr(bool allSurr) to enum RangeOption+getRange(enum option); change remaining C macros to use dataAccess for 16/32/8-bit value widths; fix/clarify some API docs
* ICU-13530 add javadoc
* ICU-13530 document UCPTrie binary data format
* ICU-13530 update .nrm formatVersion 3->4, document change in surrogate handling with new trie
* ICU-13530 re-hardcode NFC data
* move trie swapper code into new file; add new files to Windows project files; turn off trie debugging
* ICU-13530 minor cleanup
* ICU-13530 test more range starts; fix a C test leak
* ICU-13530 regenerate Java data from scratch
* ICU-13530 review feedback changes: API docs typos, more @internal, C++11 field initializers, fix potential leak in MutableCodePointTrie::fromUCPTrie()
* ICU-13530 rename interface FilterValue to ValueFilter
Simplified redundant code and removed unnecessary code path for Java 7 and later runtime in various places. Also cleaned up stale comments.
There is one test code change in com.ibm.icu.dev.test.localespi.NumberFormatTest - number keyword in test case was changed to Arab to arab. This test case was skipped with Java 6 runtime. It looks this code was note tested on Java 7 and later. @number=Arab does not work because it's case sensitive and must be all lower case letters.
Previously, ULocale had to work on Java 6, therefore, Java 7 specific Locale methods needed to be invoked by reflection. For the next release, we can directly call Java 7 Locale APIs, so rather complicated reflection implementation were replaced with simple API calls.
Previously, some developers accidentally introduced Java 7 Objects class utility methods. At that time, we once added Java 6 compatible implementation in com.ibm.icu.impl.Utility class. Now, we use Java 7 as the minimum supported Java version, so we can use Java 7 Objects class and methods.
There are some extra changes dropping generics type from constructor. For example from `ArrayList<String> output = new ArrayList<String>();` to `ArrayList<String> output = new ArrayList<>();`. The updated syntax is allowed since Java 7. We have eclipse project configured to normalize such expression, therefore, files touched by this commit were automatically updated. These changes are not directly related to Java 7 Objects method replacement.
Includes all line break tailorings.
Corresponding updates to monkey test rules.
State table builder, fix missed table optimization, uncovered by new rule.
At the moment, CLDR pseudolocale region codes don't have display names.
As long as that is so, ICU must not require them to exist in order to
not cause bogus test failures when building with pseudolocales.
Both the number of installed locales and the name of the last locale
change with updates to and customizations of CLDR/ICU data so test
expectations on these values will cause bogus test failures.
There are 0x10FFFF code points and this test allocates memory for every
code point tested. Testing every single one of them is therefore memory
intensive and slow, without adding any real value to the test.
Instead an equivalent test coverage can be achieved by testing a much
smaller set of ranges of code points that cover all categories.
X-SVN-Rev: 41547
