Commit Graph

35 Commits

Author SHA1 Message Date
Markus Scherer
fe3eb3ed5c
ICU-13530 add UCPTrie/CodePointTrie, switch normalization to use it (#48)
* ICU-13530 copy C/C++ files UTrie2 -> UTrie3

X-SVN-Rev: 40754

* ICU-13530 UTrie3 new files copied from UTrie2: rename types/functions/macros

X-SVN-Rev: 40755

* ICU-13530 debug-print building each UTrie2

X-SVN-Rev: 40756

* ICU-13530 remove two-byte-UTF-8 errorValue block; move highValue from end of data array into header; add errorValue to header

X-SVN-Rev: 40762

* ICU-13530 UTrie3 U16_NEXT/PREV: errorValue for unpaired surrogates

X-SVN-Rev: 40763

* ICU-13530 no more separate values for lead surrogate code units

X-SVN-Rev: 40764

* ICU-13530 change from 11:5 trie bits to 10:6 for simpler UTF-8 code

X-SVN-Rev: 40766

* ICU-13530 UTrie2 build UTrie3 as well, print sizes

X-SVN-Rev: 40767

* ICU-13530 debug-print countSame, sumOverlaps, countInitial

X-SVN-Rev: 40768

* ICU-13530 debug-print whether trie is for CanonIterData

X-SVN-Rev: 40769

* ICU-13530 no index-shift for BMP data, no separate index-2 for 2-byte UTF-8; builder changes incomplete

X-SVN-Rev: 40777

* ICU-13530 remove errorValue and highStart from UNewTrie3

X-SVN-Rev: 40778

* ICU-13530 rewrite UTrie3 builder code

X-SVN-Rev: 40783

* ICU-13530 UTrie3 bug fixes

X-SVN-Rev: 40788

* ICU-13530 fully re-inline _UTRIE3_U8_NEXT()

X-SVN-Rev: 40790

* ICU-13530 find most common all-same data block for dataNullBlock and initialValue

X-SVN-Rev: 40792

* ICU-13530 UTrie3 iterator functions take start and return the end of a range, rather than callback call for each range

X-SVN-Rev: 40800

* ICU-13530 mask off unused data value bits before building a UTrie3 with values less than 32 bits wide

X-SVN-Rev: 40803

* ICU-13530 split utrie3builder.h out of utrie3.h

X-SVN-Rev: 40804

* ICU-13530 separate types UTrie3 vs. UTrie3Builder, implement builder as wrapper over C++ class Trie3Builder in .cpp

X-SVN-Rev: 40809

* ICU-13530 function to make a UTrie3Builder from a UTrie3

X-SVN-Rev: 40810

* ICU-13530 debug-print some data; some cleanup

X-SVN-Rev: 40865

* ICU-13530 BMP 10:6 but supplementary 10:6:4

X-SVN-Rev: 40984

* ICU-13530 move errorValue & highValue to the end of the data table, minimal padding to 4 bytes

X-SVN-Rev: 41011

* ICU-13530 index-1 table gap of index-2 null blocks

X-SVN-Rev: 41018

* ICU-13530 test with more than 128k compacted data

X-SVN-Rev: 41034

* ICU-13530 supplementary bits 11:5:4 saves a little space

X-SVN-Rev: 41039

* ICU-13530 supplementary bits 6:5:5:4 instead of gap: about same size but simpler

X-SVN-Rev: 41050

* ICU-13530 remove unnecessary utrie3_clone(built trie)

X-SVN-Rev: 41058

* ICU-13530 remove unnecessary UTrie3StringIterator

X-SVN-Rev: 41059

* ICU-13530 back to UTRIE3_GET...() macros *returning* data values

X-SVN-Rev: 41060

* ICU-13530 fast vs. small

X-SVN-Rev: 41066

* ICU-13530 always load NFC data, add simple normalization performance test

X-SVN-Rev: 41110

* ICU-13530 change normalization main trie to UTrie3 with special values for lead surrogates; forbid non-inert surrogate code *points* because unable to store values different from code *units*; runtime code work around that for code point lookup and iteration; adjust UTS 46 for normalization no longer mapping unpaired surrogates to U+FFFD

X-SVN-Rev: 41122

* ICU-13530 simplenormperf bug fix and NFC base line

X-SVN-Rev: 41126

* ICU-13530 move normalization getRange skipping lead surrogates to API getRangeSkipLead()

X-SVN-Rev: 41182

* ICU-13530 switch CanonIterData and gennorm2 Norms to UTrie3

X-SVN-Rev: 41183

* ICU-13530 remove unused overwrite parameter from setRange()

X-SVN-Rev: 41184

* ICU-13530 getRange skip lead -> fixed surrogates

X-SVN-Rev: 41219

* ICU-13530 minor cleanup

X-SVN-Rev: 41221

* ICU-13530 UTS 46 code map unpaired surrogates to U+FFFD before normalization

X-SVN-Rev: 41224

* ICU-13530 minor internal-docs cleanup

X-SVN-Rev: 41225

* ICU-13530 rename UTrie3 to UCPTrie, and other name changes

X-SVN-Rev: 41226

* ICU-13530 add 8-bit data option; add type-any & valueBits-any for fromBinary(); macros consistently source type then data width

X-SVN-Rev: 41234

* ICU-13530 scrub the API docs for the proposal

X-SVN-Rev: 41319

* ICU-13530 tag internal definitions as such, or move them to an internal header

X-SVN-Rev: 41320

* ICU-13530 Java API skeleton

X-SVN-Rev: 41326

* ICU-13530 API feedback: ValueWidth, MutableCodePointTrie, base CodePointMap, ...

X-SVN-Rev: 41382

* ICU-13530 add UCPTrie valueWidth field and padding, and combine data pointers into a union

X-SVN-Rev: 41408

* ICU-13530 switch some macros to using dataAccess parameter: separate index vs. data lookups, no macro variant for each value width

X-SVN-Rev: 41409

* ICU-13530 StringIterator is no longer a java.util.Iterator (bad fit)

X-SVN-Rev: 41455

* ICU-13530 CodePointTrie.java code complete

X-SVN-Rev: 41518

* ICU-13530 finish Java port incl test; keep C++ parallel

* ICU-13530 adjust API for feedback: rename HandleValue to FilterValue, change getRange+getRangeFixedSurr(bool allSurr) to enum RangeOption+getRange(enum option); change remaining C macros to use dataAccess for 16/32/8-bit value widths; fix/clarify some API docs

* ICU-13530 add javadoc

* ICU-13530 document UCPTrie binary data format

* ICU-13530 update .nrm formatVersion 3->4, document change in surrogate handling with new trie

* ICU-13530 re-hardcode NFC data

* move trie swapper code into new file; add new files to Windows project files; turn off trie debugging

* ICU-13530 minor cleanup

* ICU-13530 test more range starts; fix a C test leak

* ICU-13530 regenerate Java data from scratch

* ICU-13530 review feedback changes: API docs typos, more @internal, C++11 field initializers, fix potential leak in MutableCodePointTrie::fromUCPTrie()

* ICU-13530 rename interface FilterValue to ValueFilter
2018-09-27 14:27:38 -07:00
Jeff Genovy
16598e5468 ICU-13436 Fix various typos and spelling mistakes.
X-SVN-Rev: 40645
2017-10-26 21:33:43 +00:00
Andy Heninger
7a72a8f032 ICU-13380 Fixes for memory leaks & other problems found by Address Sanitizer.
X-SVN-Rev: 40529
2017-10-03 04:11:26 +00:00
Andy Heninger
242e02c388 ICU-12764 icu4c utf-8 source files, update Copyright notices.
X-SVN-Rev: 39583
2017-01-20 00:20:31 +00:00
Michael Ow
61607c2773 ICU-12564 Update copyright notice in trunk
X-SVN-Rev: 38848
2016-06-15 18:58:17 +00:00
Yoshito Umaoka
00ca13e126 ICU-12564 Reverted r38761 and r38762, because we want to prepend the Unicode copyright for existing source files, instead of replacing copyright comments.
X-SVN-Rev: 38776
2016-05-31 21:45:07 +00:00
Michael Ow
c9f199a30f ICU-12564 Update copyright notice in ICU4C
X-SVN-Rev: 38761
2016-05-26 22:32:17 +00:00
Markus Scherer
5d7b60d863 ICU-11761 include assert.h
X-SVN-Rev: 37653
2015-07-08 18:17:53 +00:00
Markus Scherer
f87b2c9b3c ICU-11761 share equal strings even if they need explicit length; more generic root resource for flexibility (experiment with more work on pool bundles)
X-SVN-Rev: 37649
2015-07-07 18:50:21 +00:00
Markus Scherer
a1348758c1 ICU-11761 simpler memory management with UnicodeString & LocalArray
X-SVN-Rev: 37644
2015-07-03 02:12:17 +00:00
Markus Scherer
9312cb2d1a ICU-11761 SResource virtual dispatch for preflighting & writing resources; C++ify some of SRBRoot
X-SVN-Rev: 37643
2015-07-02 23:51:09 +00:00
Markus Scherer
62e756505c ICU-11751 polymorphic TableResource, ArrayResource, IntResource, IntVectorResource, BinaryResource
X-SVN-Rev: 37619
2015-06-24 20:15:08 +00:00
Markus Scherer
43c47b5685 ICU-11751 change genrb.c to .cpp, polymorphic StringResource & AliasResource with UnicodeString
X-SVN-Rev: 37618
2015-06-24 09:11:42 +00:00
Markus Scherer
344357d3f6 ICU-11720 better memory management in genrb parseBinary() using auto pointers
X-SVN-Rev: 37577
2015-06-16 13:14:14 +00:00
Markus Scherer
5b21336927 ICU-11719 better memory management in genrb parseImport() using string class & auto pointers
X-SVN-Rev: 37576
2015-06-16 12:34:36 +00:00
Markus Scherer
f99546419f ICU-11711 better memory management in GenrbImporter::getRules() using string class & auto pointers
X-SVN-Rev: 37575
2015-06-16 11:23:04 +00:00
Markus Scherer
a9d7c3e4bd ICU-11449 reorder single scripts not groups, scripts/groups can start on top-16-bit boundaries, data formatVersion 5 for new scripts data and optional reorderRanges appended to reorderCodes
X-SVN-Rev: 36924
2015-01-07 03:37:11 +00:00
Markus Scherer
ec55298e96 ICU-11042 Han radical-stroke order data in the root collator instead of CJK tailorings, adjusted code & tests (merged from branches/markus/rootunihan)
X-SVN-Rev: 36162
2014-08-14 18:47:27 +00:00
Markus Scherer
435623bc05 ICU-11058 support nested collation rule imports; make the importer stateless
X-SVN-Rev: 36157
2014-08-13 22:58:14 +00:00
Markus Scherer
2e958e2bab ICU-8082 private collation type: do not build binary and do not list as an available keyword value
X-SVN-Rev: 36011
2014-07-08 17:44:53 +00:00
Markus Scherer
9e4365c9e6 ICU-10810 genrb: preflight strings on final parse tree, not while building the tree
X-SVN-Rev: 35544
2014-03-28 22:03:03 +00:00
Michael Ow
b8358fd167 ICU-10766 Rename ResourceType enum values to avoid collision on MinGW build
X-SVN-Rev: 35376
2014-03-07 00:46:13 +00:00
Markus Scherer
e977c057a9 ICU-9101 merge branches/markus/collv2@35225 into the trunk
X-SVN-Rev: 35227
2014-02-25 21:21:49 +00:00
Markus Scherer
8067293d35 ICU-10043 ignore the genrb --omitCollationRules flag while importing rules
X-SVN-Rev: 34874
2014-01-11 00:28:53 +00:00
Markus Scherer
ed3bb51647 ICU-10499 genrb use explicit UnicodeString constructors
X-SVN-Rev: 34867
2014-01-10 21:41:03 +00:00
Markus Scherer
b20a9793f9 ICU-10282 deprecate bad API RuleBasedCollator::cloneRuleData(), call cloneBinary()
X-SVN-Rev: 34314
2013-09-13 19:25:01 +00:00
Markus Scherer
31ca61dc7a ICU-9679 remove another unused variable
X-SVN-Rev: 32784
2012-11-09 22:50:25 +00:00
Markus Scherer
2f6df5d520 ICU-9434 use static_cast<pointer type>(void *) not reinterpret_cast
X-SVN-Rev: 32119
2012-08-08 00:50:16 +00:00
Michael Ow
671a7be44b ICU-9322 Fix some memory check errors in ICU tools
X-SVN-Rev: 31829
2012-05-17 19:58:22 +00:00
Michael Ow
0ca13b73b0 ICU-9292 Merge BEAM warning fixes from branch into trunk
X-SVN-Rev: 31792
2012-05-03 05:50:26 +00:00
Andy Heninger
5b3a708915 ICU-8992 Compiler Warning cleanup for gcc 4.6.1
X-SVN-Rev: 31329
2012-02-06 19:57:08 +00:00
Markus Scherer
43bb4f2cf4 ICU-8827 Unicode 6.1 update: svn merge branches/markus/uni61 at r31062 into the trunk
X-SVN-Rev: 31071
2011-12-08 22:37:24 +00:00
Umesh Nair
4a6f817e00 ICU-8108 Allowing imports from the same file by suppressing building binaries for imported rules
X-SVN-Rev: 31036
2011-12-06 00:23:46 +00:00
Markus Scherer
c2367c67f9 ICU-8601 fix extern-variable C/C++ mismatch in genrb (broke Windows)
X-SVN-Rev: 30155
2011-05-27 18:56:14 +00:00
Markus Scherer
41c8654d92 ICU-8601 replace C U_INLINE hack with C++ inline, and also rename a few other .c files to .cpp for future UMTX_CHECK replacement
X-SVN-Rev: 30148
2011-05-26 20:05:39 +00:00