Commit Graph

8143 Commits

Author SHA1 Message Date
Yoshito Umaoka
45cdda61a6
ICU-12973 CLDR Japanese Era data and tentative Japanese new era support (#111)
* Updated era data format in supplementalData.

* Include tentative era names in data. Implemented Japanese era loaded from CLDR data in ICU4J.

* ICU4C implementation, ICU4C refactoring. WIP.

* VS project updates and some bug fixes

Also added API comments.

* Review feedback and bug fixes

- NULL to nullptr
- use of LocalUResourceBundlePointer
- TYPO "name" to "named"
- env var checking stricmp() == 0

* API comment correction based on feedback

* Duplicate the comment in ucal.h to calendar.h

* Fixed spelling errors in API comment
2018-09-27 14:27:39 -07:00
jungshik
5cfdf4a519
ICU-20098 Fix BCP47 validity check for extlang and privateuse singleton (#102)
* ICU-20098 Fix the validty check for extlang in uloc_forLanguageTag

BCP 47 has the following for language. extlang subtag can only be
preceded by 2*3ALPHA. Add a check for the length of language subtag
before extlang subtag.

language      = 2*3ALPHA            ; shortest ISO 639 code
                 ["-" extlang]       ; sometimes followed by
                                     ; extended language subtags
               / 4ALPHA              ; or reserved for future use
               / 5*8ALPHA            ; or registered language subtag

 extlang       = 3ALPHA              ; selected ISO 639 codes
                 *2("-" 3ALPHA)      ; permanently reserved}}

With this change, 'hant-cmn-CN' would drop '-cmn-CN' keeping only
'hant'.

* ICU-20098 Fix the validty check for extlang for ICU4J

* ICU-20098 Fix the compiler failure for ICU4J

* ICU-20098 Fix a compile error and test.

* ICU-20098 Add a test for invalid private use  singleton

ICU4C's check for private use singleton subtag ('x') is wrong and
treats invalid language tags as valid.

ICU4J's check is correct and does not require any change.

Fix that and add tests to both ICU4C and ICU4J.
2018-09-27 14:27:39 -07:00
David Corbett
378b04c1b2
ICU-12979 Fix \Q...\E in UnicodeRegex#transform (#23) 2018-09-27 14:27:39 -07:00
Markus Scherer
82f0f480d4
ICU-20086 C++ sets & maps for Unicode properties (#93)
also create ucpmap.h from renamed parts of ucptrie.h
2018-09-27 14:27:39 -07:00
Andy Heninger
740b24118f
ICU-8151 Simplify Finnish Line Break Tailoring, move to root. (#99) 2018-09-27 14:27:39 -07:00
Victor Chang
ac0c7bbe3c
ICU-13808 Document ArithmeticException thrown by DecimalFormat
- Document the exception in the following methods
setMultiplier, setMathContext, setMathContextICU
- Add test to check the documented behavior
2018-09-27 14:27:39 -07:00
Markus Scherer
d2ec8987a7
ICU-8966 ICU-12850 add API/data/code for text layout properties InPC, InSC, vo (#92)
ICU-8966: Indic_Positional_Category & Indic_Syllabic_Category

ICU-12850: Vertical_Orientation
2018-09-27 14:27:39 -07:00
gnrunge
82cad0d03c
ICU-20092 Merge pull request #75 from gnrunge/langmatcher
ICU-20092 Integrates new languagematcher enhancements into ICU4J.
2018-09-27 14:27:39 -07:00
Fredrik Roubert
d2b2d2409a
ICU-20006 Parameterize the TestSourceTargetSet2 test case.
By extracting this test case into a separate class and separating the
complex and time consuming building of the UnicodeSet disorderedMarks
data structure into an @BeforeClass shared setup method it becomes
possible to test the 18 different transliteration rules as a
parameterized test case.

This will lower the running time per test case and also aid in debugging
as it will make it immediately obvious which transliteration rules cause
test failures (and which don't).
2018-09-27 14:27:39 -07:00
Norbert Runge
c854dd0d54
ICU-20092 Integrates new languagematcher enhancements into ICU4J. 2018-09-27 14:27:39 -07:00
Fredrik Roubert
f71ecc6b18
ICU-13483 Use a real time zone to make tests work in Android.
When running in Android, java.util.TimeZone.getDefault() will only
return a TimeZone object for a time zone that actually exists, so when
passing a bogus test TimeZone object to setDefault() and afterwards
calling getDefault() it won't return that same time zone.

Changing our tests to instead use a real existing (but otherwise unused)
time zone instead makes the tests work in Android as well.

Neither of these tests was testing the ability to set an arbitrarily
made up time zone as the default, so no test coverage is lost.
2018-09-27 14:27:39 -07:00
Andy Heninger
03c6e86c6c
ICU-13334 add test for reported word break behavior issue, not reproducable. (#74) 2018-09-27 14:27:38 -07:00
Markus Scherer
000c266045
ICU-13530 fix bugs, add tests, clarify docs (#83)
* ICU-13530 test & fix cptrie.getRange() when small null data block matches the start of a non-null fast data block

* ICU-13530 fix test bug

* ICU-13530 test & fix bug calculating worst-case data array capacity at start of compaction

* ICU-13530 docs: clarify buildImmutable() discards upper bits, trie then returns unsigned ints; range iteration slower than if ranges were stored directly

* ICU-13530 accept feedback on docs
2018-09-27 14:27:38 -07:00
Markus Scherer
852fb5b51f
ICU-13530 CodePointMap docs fix s/trie/map/ (#73) 2018-09-27 14:27:38 -07:00
Shane
8cad57b56c
ICU-13696 Parse local currency ISO codes as case-insensitive. (#62) 2018-09-27 14:27:38 -07:00
Andy Heninger
c34984bd6f
ICU-13664 RBBI, add regression test for reported problem (already fixed.) 2018-09-27 14:27:38 -07:00
Fredrik Roubert
da7bd533ab
ICU-20000 Workaround for BigDecimal.stripTrailingZeros() differences. (#57)
Different implementations of BigDecimal.stripTrailingZeros(), in
different versions of the JDK (and different versions of Android), have
differences in their handling of zero. To avoid this, ICU4J can return
BigDecimal.ZERO for any value that is equal to zero, instead of calling
BigDecimal.stripTrailingZeros() in this problematic case.
2018-09-27 14:27:38 -07:00
Andy Heninger
ffedbf99cb
ICU-13413 Java 7 for Javadoc build. (#68) 2018-09-27 14:27:38 -07:00
Fredrik Roubert
acfc55d0a5
ICU-13776 Use OpenType.DIRECT for loading the keyTypeData resources. 2018-09-27 14:27:38 -07:00
Markus Scherer
fe3eb3ed5c
ICU-13530 add UCPTrie/CodePointTrie, switch normalization to use it (#48)
* ICU-13530 copy C/C++ files UTrie2 -> UTrie3

X-SVN-Rev: 40754

* ICU-13530 UTrie3 new files copied from UTrie2: rename types/functions/macros

X-SVN-Rev: 40755

* ICU-13530 debug-print building each UTrie2

X-SVN-Rev: 40756

* ICU-13530 remove two-byte-UTF-8 errorValue block; move highValue from end of data array into header; add errorValue to header

X-SVN-Rev: 40762

* ICU-13530 UTrie3 U16_NEXT/PREV: errorValue for unpaired surrogates

X-SVN-Rev: 40763

* ICU-13530 no more separate values for lead surrogate code units

X-SVN-Rev: 40764

* ICU-13530 change from 11:5 trie bits to 10:6 for simpler UTF-8 code

X-SVN-Rev: 40766

* ICU-13530 UTrie2 build UTrie3 as well, print sizes

X-SVN-Rev: 40767

* ICU-13530 debug-print countSame, sumOverlaps, countInitial

X-SVN-Rev: 40768

* ICU-13530 debug-print whether trie is for CanonIterData

X-SVN-Rev: 40769

* ICU-13530 no index-shift for BMP data, no separate index-2 for 2-byte UTF-8; builder changes incomplete

X-SVN-Rev: 40777

* ICU-13530 remove errorValue and highStart from UNewTrie3

X-SVN-Rev: 40778

* ICU-13530 rewrite UTrie3 builder code

X-SVN-Rev: 40783

* ICU-13530 UTrie3 bug fixes

X-SVN-Rev: 40788

* ICU-13530 fully re-inline _UTRIE3_U8_NEXT()

X-SVN-Rev: 40790

* ICU-13530 find most common all-same data block for dataNullBlock and initialValue

X-SVN-Rev: 40792

* ICU-13530 UTrie3 iterator functions take start and return the end of a range, rather than callback call for each range

X-SVN-Rev: 40800

* ICU-13530 mask off unused data value bits before building a UTrie3 with values less than 32 bits wide

X-SVN-Rev: 40803

* ICU-13530 split utrie3builder.h out of utrie3.h

X-SVN-Rev: 40804

* ICU-13530 separate types UTrie3 vs. UTrie3Builder, implement builder as wrapper over C++ class Trie3Builder in .cpp

X-SVN-Rev: 40809

* ICU-13530 function to make a UTrie3Builder from a UTrie3

X-SVN-Rev: 40810

* ICU-13530 debug-print some data; some cleanup

X-SVN-Rev: 40865

* ICU-13530 BMP 10:6 but supplementary 10:6:4

X-SVN-Rev: 40984

* ICU-13530 move errorValue & highValue to the end of the data table, minimal padding to 4 bytes

X-SVN-Rev: 41011

* ICU-13530 index-1 table gap of index-2 null blocks

X-SVN-Rev: 41018

* ICU-13530 test with more than 128k compacted data

X-SVN-Rev: 41034

* ICU-13530 supplementary bits 11:5:4 saves a little space

X-SVN-Rev: 41039

* ICU-13530 supplementary bits 6:5:5:4 instead of gap: about same size but simpler

X-SVN-Rev: 41050

* ICU-13530 remove unnecessary utrie3_clone(built trie)

X-SVN-Rev: 41058

* ICU-13530 remove unnecessary UTrie3StringIterator

X-SVN-Rev: 41059

* ICU-13530 back to UTRIE3_GET...() macros *returning* data values

X-SVN-Rev: 41060

* ICU-13530 fast vs. small

X-SVN-Rev: 41066

* ICU-13530 always load NFC data, add simple normalization performance test

X-SVN-Rev: 41110

* ICU-13530 change normalization main trie to UTrie3 with special values for lead surrogates; forbid non-inert surrogate code *points* because unable to store values different from code *units*; runtime code work around that for code point lookup and iteration; adjust UTS 46 for normalization no longer mapping unpaired surrogates to U+FFFD

X-SVN-Rev: 41122

* ICU-13530 simplenormperf bug fix and NFC base line

X-SVN-Rev: 41126

* ICU-13530 move normalization getRange skipping lead surrogates to API getRangeSkipLead()

X-SVN-Rev: 41182

* ICU-13530 switch CanonIterData and gennorm2 Norms to UTrie3

X-SVN-Rev: 41183

* ICU-13530 remove unused overwrite parameter from setRange()

X-SVN-Rev: 41184

* ICU-13530 getRange skip lead -> fixed surrogates

X-SVN-Rev: 41219

* ICU-13530 minor cleanup

X-SVN-Rev: 41221

* ICU-13530 UTS 46 code map unpaired surrogates to U+FFFD before normalization

X-SVN-Rev: 41224

* ICU-13530 minor internal-docs cleanup

X-SVN-Rev: 41225

* ICU-13530 rename UTrie3 to UCPTrie, and other name changes

X-SVN-Rev: 41226

* ICU-13530 add 8-bit data option; add type-any & valueBits-any for fromBinary(); macros consistently source type then data width

X-SVN-Rev: 41234

* ICU-13530 scrub the API docs for the proposal

X-SVN-Rev: 41319

* ICU-13530 tag internal definitions as such, or move them to an internal header

X-SVN-Rev: 41320

* ICU-13530 Java API skeleton

X-SVN-Rev: 41326

* ICU-13530 API feedback: ValueWidth, MutableCodePointTrie, base CodePointMap, ...

X-SVN-Rev: 41382

* ICU-13530 add UCPTrie valueWidth field and padding, and combine data pointers into a union

X-SVN-Rev: 41408

* ICU-13530 switch some macros to using dataAccess parameter: separate index vs. data lookups, no macro variant for each value width

X-SVN-Rev: 41409

* ICU-13530 StringIterator is no longer a java.util.Iterator (bad fit)

X-SVN-Rev: 41455

* ICU-13530 CodePointTrie.java code complete

X-SVN-Rev: 41518

* ICU-13530 finish Java port incl test; keep C++ parallel

* ICU-13530 adjust API for feedback: rename HandleValue to FilterValue, change getRange+getRangeFixedSurr(bool allSurr) to enum RangeOption+getRange(enum option); change remaining C macros to use dataAccess for 16/32/8-bit value widths; fix/clarify some API docs

* ICU-13530 add javadoc

* ICU-13530 document UCPTrie binary data format

* ICU-13530 update .nrm formatVersion 3->4, document change in surrogate handling with new trie

* ICU-13530 re-hardcode NFC data

* move trie swapper code into new file; add new files to Windows project files; turn off trie debugging

* ICU-13530 minor cleanup

* ICU-13530 test more range starts; fix a C test leak

* ICU-13530 regenerate Java data from scratch

* ICU-13530 review feedback changes: API docs typos, more @internal, C++11 field initializers, fix potential leak in MutableCodePointTrie::fromUCPTrie()

* ICU-13530 rename interface FilterValue to ValueFilter
2018-09-27 14:27:38 -07:00
Yoshito Umaoka
8a52f44951
ICU-13413 Merge pull request #43 from yumaoka/icu4j-java7
ICU-13413 Update the minimum Java runtime version to 7
2018-09-27 14:27:38 -07:00
Yoshito Umaoka
a4fc56f382
ICU-13413 More clean-ups for code redundant/unnecessary for Java 7 and up
Simplified redundant code and removed unnecessary code path for Java 7 and later runtime in various places. Also cleaned up stale comments.

There is one test code change in com.ibm.icu.dev.test.localespi.NumberFormatTest - number keyword in test case was changed to Arab to arab. This test case was skipped with Java 6 runtime. It looks this code was note tested on Java 7 and later. @number=Arab does not work because it's case sensitive and must be all lower case letters.
2018-09-27 14:27:38 -07:00
Yoshito Umaoka
18789823b2
ICU-13413 Updated eclipse plugin build/project files to use Java 7 as bootstrap java version 2018-09-27 14:27:38 -07:00
Yoshito Umaoka
4159176aa7
ICU-13413 Updated ULocale class to directly calls Java 7 Locale methods
Previously, ULocale had to work on Java 6, therefore, Java 7 specific Locale methods needed to be invoked by reflection. For the next release, we can directly call Java 7 Locale APIs, so rather complicated reflection implementation were replaced with simple API calls.
2018-09-27 14:27:38 -07:00
Yoshito Umaoka
135bb1e380
ICU-13413 Migrating some Utility class methods with Java 7 Objects class method
Previously, some developers accidentally introduced Java 7 Objects class utility methods. At that time, we once added Java 6 compatible implementation in com.ibm.icu.impl.Utility class. Now, we use Java 7 as the minimum supported Java version, so we can use Java 7 Objects class and methods.

There are some extra changes dropping generics type from constructor. For example from `ArrayList<String> output = new ArrayList<String>();` to `ArrayList<String> output = new ArrayList<>();`. The updated syntax is allowed since Java 7. We have eclipse project configured to normalize such expression, therefore, files touched by this commit were automatically updated. These changes are not directly related to Java 7 Objects method replacement.
2018-09-27 14:27:38 -07:00
Andy Heninger
fa5ae3dc45
ICU-7270 Line Break rule LB8 updated for UAX-14 conformance. (#41)
Includes all line break tailorings.
Corresponding updates to monkey test rules.
State table builder, fix missed table optimization, uncovered by new rule.
2018-09-27 14:27:38 -07:00
Yoshito Umaoka
d1c761762a
ICU-13413 Java 7 migration in build scripts and eclipse project files
- Updated javac/javadoc source and target Java version to 1.7.
- Eclipse projects to use execution environment JavaSE-1.7.
- Replaced java6.bootclasspath with java7.bootclasspath.
- Removed pre-Java 7 conditional build targets.
- Deleted stylesheet for JDK6 build.
2018-09-27 14:27:38 -07:00
Victor Chang
b69407ffab
ICU-20007 Remove incorrect ICU4J doc of DecimalFormat.setGroupingUsed
DecimalFormat.setGroupingUsed(true) no longer sets the grouping size to 3.
The java doc should be removed.
2018-09-27 14:27:38 -07:00
Fredrik Roubert
ed5201f293
ICU-20016 Skip testing pseudolocale region codes for display names.
At the moment, CLDR pseudolocale region codes don't have display names.
As long as that is so, ICU must not require them to exist in order to
not cause bogus test failures when building with pseudolocales.
2018-09-27 14:27:37 -07:00
Fredrik Roubert
b24880b19f
ICU-20018 Remove hardcoded expectations on available locales.
Both the number of installed locales and the name of the last locale
change with updates to and customizations of CLDR/ICU data so test
expectations on these values will cause bogus test failures.
2018-09-27 14:27:37 -07:00
Shane Carr
bd1d915c2d
ICU-20056 Adding test for minFrac with maxSig 2018-09-27 14:27:37 -07:00
Norbert Runge
95f7cad6b3
ICU-13829 Removes duplicate import statement. 2018-09-27 14:27:37 -07:00
Norbert Runge
f97507dab2
ICU-13829 Revert order of imports to conform with (almost all of) the other files in this directory. 2018-09-27 14:27:37 -07:00
Norbert Runge
298462e5fe
ICU-13829 Languagematcher: Fixes straightforward code issues. 2018-09-27 14:27:37 -07:00
Markus Scherer
99354ec034 ICU-13851 case mapping data: when fetching delta make sure to read from start of exceptional values
X-SVN-Rev: 41550
2018-06-24 21:12:22 +00:00
Fredrik Roubert
f1d29ce62e ICU-13766 Make UnicodeMapTest#TestUnicodeMapGeneralCategory smaller.
There are 0x10FFFF code points and this test allocates memory for every
code point tested. Testing every single one of them is therefore memory
intensive and slow, without adding any real value to the test.

Instead an equivalent test coverage can be achieved by testing a much
smaller set of ranges of code points that cover all categories.

X-SVN-Rev: 41547
2018-06-21 20:17:22 +00:00
Shane Carr
3f7a4aa202 ICU-13840 Fixing memory overflow issue in number_decimalquantity.cpp and adding tests.
X-SVN-Rev: 41541
2018-06-19 21:52:52 +00:00
Peter Edberg
7c28a58b1f ICU-13823 Integrate data from cldr/tags/release-33-1-d03 to ICU trunk for ICU62 final
X-SVN-Rev: 41525
2018-06-14 22:33:37 +00:00
Yoshito Umaoka
08eed14678 ICU-13822 Updated ICU4J data - removing VES.
X-SVN-Rev: 41512
2018-06-06 23:02:35 +00:00
Shane Carr
724e3b4f10 ICU-13804 Fixing test failure from r41497.
X-SVN-Rev: 41498
2018-06-01 00:48:18 +00:00
Shane Carr
dd7667d330 ICU-13804 Making number parsing code more robust when given empty symbol strings.
X-SVN-Rev: 41497
2018-06-01 00:31:54 +00:00
Shane Carr
29937704cd ICU-8610 Responding to number skeleton code review feedback.
X-SVN-Rev: 41483
2018-05-30 03:34:41 +00:00
Peter Edberg
8d17fb0e1b ICU-13667 ICU 62rc BRS, update measurement unit constants
X-SVN-Rev: 41481
2018-05-30 01:22:52 +00:00
Peter Edberg
4e6358240c ICU-13792 Integrate data from cldr/tags/release-33-1-d02 to ICU trunk for ICU62rc
X-SVN-Rev: 41479
2018-05-29 22:33:10 +00:00
Yoshito Umaoka
97011814e0 ICU-13667 BRS62 - Added ICU 62 serialization test data, removed ICU 57.
X-SVN-Rev: 41478
2018-05-29 21:26:24 +00:00
Yoshito Umaoka
eaa25479e1 ICU-13667 BRS62 - added ICU4J 62 API signature file (and deleted ICU4J 52 API signature file).
X-SVN-Rev: 41477
2018-05-29 21:12:48 +00:00
Yoshito Umaoka
a12265d749 ICU-13667 Updated API change report, fixed API report tool to handle 5 letter sub-package (number) properly.
X-SVN-Rev: 41476
2018-05-29 21:07:31 +00:00
Yoshito Umaoka
2076cee7d8 ICU-13667 BRS62 Clean up import statements.
X-SVN-Rev: 41475
2018-05-29 19:34:43 +00:00
Yoshito Umaoka
f4c892c6c4 ICU-13667 Fixing compiler warnings and API doc issues.
X-SVN-Rev: 41473
2018-05-29 17:06:21 +00:00
Jeff Genovy
70e173a4a2 ICU-13667 Update ICU4J version number for 62.1 release candidate, and readme file.
X-SVN-Rev: 41467
2018-05-25 23:39:14 +00:00