Commit Graph

128 Commits

Author SHA1 Message Date
Markus Scherer
410c2b4580 ICU-8972 finish preparseucd.py parsing uchar.h, prepare data in order of old genpname/preparse.pl
X-SVN-Rev: 31158
2011-12-19 19:53:57 +00:00
Markus Scherer
5597c3ef96 ICU-8972 progress towards preparseucd.py also parsing uchar.h & uscript.h and writing pnames_data.h; add remaining ICU-specific properties that used to be in SyntheticPropertyAliases.txt
X-SVN-Rev: 31157
2011-12-19 05:21:15 +00:00
Markus Scherer
7c797527dc ICU-8972 PropsBuilder makes more sense than PropsWriter
X-SVN-Rev: 31156
2011-12-19 01:49:34 +00:00
Markus Scherer
0b9ba091db ICU-8972 move genpname/genpname.cpp to genprops/pnameswriter.cpp; use on-the-fly property names data for ppucd parsing: avoids having to rebuild ICU & tools before generating properties data
X-SVN-Rev: 31153
2011-12-18 21:33:57 +00:00
Markus Scherer
10243eee46 ICU-8972 preparseucd.py: add ISO 15924-only script codes, change argv[2] to path/to/ICU/src/root, prepend UTF-8 BOM to some test files
X-SVN-Rev: 31151
2011-12-18 16:01:49 +00:00
Markus Scherer
b1e48658b3 ICU-8972 genprops: merge props2writer.cpp (which used to be props2.cpp) into corepropswriter.cpp (which used to be store.c); avoids clumsy back-and-forth
X-SVN-Rev: 31150
2011-12-18 08:17:16 +00:00
Markus Scherer
cb626ab6d7 ICU-8972 genprops: minor fix, guard against nv=-100
X-SVN-Rev: 31149
2011-12-18 07:23:10 +00:00
Markus Scherer
f0dbf9f00b ICU-8972 genprops: use ppucd.txt for all uprops.icu properties including Script_Extensions; cleanup
X-SVN-Rev: 31147
2011-12-18 06:53:54 +00:00
Markus Scherer
7270f8f618 ICU-8972 genprops: use ppucd.txt for gc, nt, nv
X-SVN-Rev: 31145
2011-12-18 03:21:54 +00:00
Markus Scherer
764caf4347 ICU-8972 genprops: remove --csource option (always write both C & binary), use argv[1]=path/to/ICU/src/root instead of --destdir; separate finalizing vs. writing data
X-SVN-Rev: 31144
2011-12-18 00:37:18 +00:00
Markus Scherer
2cac672e6e ICU-8972 genprops: use ppucd.txt for enumerated properties
X-SVN-Rev: 31143
2011-12-17 06:31:25 +00:00
Markus Scherer
008e89c02f ICU-8972 genprops: use ppucd.txt for binary properties
X-SVN-Rev: 31142
2011-12-17 05:45:58 +00:00
Markus Scherer
21b5ebc902 ICU-8972 genprops: use ppucd.txt for the age property; some cleanup
X-SVN-Rev: 31141
2011-12-17 05:03:26 +00:00
Markus Scherer
03f9f2307e ICU-8972 genprops: start to read ppucd.txt, use that for Decomposition_Type
X-SVN-Rev: 31137
2011-12-16 06:51:58 +00:00
Markus Scherer
85db2273c8 ICU-8972 rename genprops data structure builder files in preparation for merging Unicode tools
X-SVN-Rev: 31133
2011-12-16 00:41:08 +00:00
Markus Scherer
fc3909efc8 ICU-8972 script to preparse the UCD (multiple .txt files) into a single ppucd.txt with a simple syntax
X-SVN-Rev: 31130
2011-12-15 23:58:14 +00:00
Markus Scherer
b0702b4677 ICU-9013 exclude Unicode_1_Name data from unames.icu
X-SVN-Rev: 31124
2011-12-15 21:43:23 +00:00
Markus Scherer
1856fafdc3 ICU-8827 genuca bug fix: UCA 6.1 has a max contraction length of 4 not 3, parameterize code & raise max length
X-SVN-Rev: 31051
2011-12-07 06:14:56 +00:00
Markus Scherer
46555dac60 ICU-8827 more genuca white space parsing fixes
X-SVN-Rev: 31046
2011-12-07 00:34:11 +00:00
Markus Scherer
a7394689d8 ICU-8827 ignore Script_Extensions @missing value <script>
X-SVN-Rev: 31030
2011-12-05 20:05:26 +00:00
Markus Scherer
8087532167 ICU-8827 adjust for new @missing values; for Name_Alias only use type=corrections for now
X-SVN-Rev: 31014
2011-12-03 00:20:31 +00:00
Markus Scherer
0ce7b55545 ICU-8827 PropertyAliases-6.1.0d11.txt formally adds Script_Extensions
X-SVN-Rev: 31010
2011-12-02 21:03:34 +00:00
Markus Scherer
d7a6e85957 ICU-8827 property names data from PropertyValueAliases-6.1.0d14.txt
X-SVN-Rev: 31004
2011-12-02 01:09:11 +00:00
Markus Scherer
b13255af27 ICU-8827 improve parsing of reorder codes
X-SVN-Rev: 30914
2011-11-03 23:09:27 +00:00
Markus Scherer
99937057c0 ICU-8827 gencase report parsing errors, and cope with @missing lines in SpecialCasing.txt and CaseFolding.txt
X-SVN-Rev: 30887
2011-10-31 23:49:12 +00:00
Markus Scherer
c8921a5a1d ICU-8827 genuca verify UCA == UCD versions
X-SVN-Rev: 30883
2011-10-30 00:20:58 +00:00
Markus Scherer
ef5aeb05a3 ICU-8827 Unicode 6.1 tools: ignore UTS #46 NV8, update version numbers
X-SVN-Rev: 30880
2011-10-29 00:02:09 +00:00
Markus Scherer
e2ccb065cd ICU-8827 Unicode 6.1 tools update
X-SVN-Rev: 30870
2011-10-26 04:41:57 +00:00
Markus Scherer
7b666417fd ICU-7752 tools: rename generated *_props_data.c files to .h and add #ifndef guard
X-SVN-Rev: 30816
2011-10-12 22:57:58 +00:00
Markus Scherer
018383ded0 ICU-8555 add nine new ISO 15924 script codes from 2010-12-21
X-SVN-Rev: 30084
2011-05-10 20:54:38 +00:00
Markus Scherer
dbcb1154c8 ICU-8167 bytestriebuilder.h is now public
X-SVN-Rev: 29533
2011-03-03 22:02:58 +00:00
Markus Scherer
83562e872b ICU-8167 adjust to BytesTrieBuilder API changes
X-SVN-Rev: 29368
2011-01-27 21:42:08 +00:00
Markus Scherer
88a77e8130 ICU-8167 rename ByteTrie to BytesTrie, DictTrie to StringTrie, etc.
X-SVN-Rev: 29266
2011-01-05 21:06:55 +00:00
Markus Scherer
951e2b9f40 ICU-8105 PropNameData constants need to be defined in the icu namespace
X-SVN-Rev: 29254
2010-12-31 23:53:55 +00:00
Markus Scherer
3e29cb9f1f ICU-8105 genpname write propname_data.h and pnames.icu formatVersion 2 with ByteTries; merge branches/markus/tries -r 29042:29248
X-SVN-Rev: 29251
2010-12-31 18:06:45 +00:00
Stuart Gill
814cc0151f ICU-3984 update genuca to match changes in ICU trunk
X-SVN-Rev: 29001
2010-11-04 20:41:35 +00:00
Stuart Gill
7e1bef5e94 ICU-3984 tool changes for collation reordering
X-SVN-Rev: 28941
2010-10-28 19:05:02 +00:00
Markus Scherer
9d45127908 ICU-7264 do not add U+FFFE, new merge-sort character with 02 weights, into invuca so that tailoring primary-after an ignorable works
X-SVN-Rev: 28820
2010-10-13 22:20:26 +00:00
Markus Scherer
ad7ef5dc33 ICU-7264 permit space between [variable top] bytes; check number of READHEX bytes; some cleanup of variable top code
X-SVN-Rev: 28814
2010-10-13 17:40:32 +00:00
Markus Scherer
023cf21cdf ICU-7264 permit 02 bytes for U+FFFE, new merge-sort character
X-SVN-Rev: 28802
2010-10-12 23:54:40 +00:00
Markus Scherer
a788c4e44c ICU-7264 check for supported prefix length
X-SVN-Rev: 28787
2010-10-11 21:01:31 +00:00
Markus Scherer
efa2e954e1 ICU-7264 support supplementary code points in FractionalUCA.txt
X-SVN-Rev: 28773
2010-10-07 19:46:41 +00:00
Markus Scherer
41e92c0c68 ICU-7264 minor bug fix
X-SVN-Rev: 28650
2010-09-20 04:12:24 +00:00
Markus Scherer
e4f1a48919 ICU-7264 split UCD files into unidata & testdata for easier ICU4C update
X-SVN-Rev: 28649
2010-09-20 04:11:49 +00:00
Markus Scherer
b0b91dd032 ICU-7264 handle uchar.h UJoiningGroup aliases because final Unicode 6 changes old U_JG_HAMZA_ON_HEH_GOAL into alias for U_JG_TEH_MARBUTA_GOAL
X-SVN-Rev: 28648
2010-09-19 14:59:17 +00:00
Markus Scherer
02a21226d9 ICU-7264 adjust IdnaMappingTable reformatting to handle new STD3 values
X-SVN-Rev: 28644
2010-09-17 22:56:53 +00:00
Markus Scherer
00a996d08f ICU-7264 stderr output for STD3 status changes
X-SVN-Rev: 28563
2010-08-31 16:01:21 +00:00
Markus Scherer
4e91247beb ICU-7264 mark characters as disallowed_STD3_valid if their decompositions contain STD3-disallowed characters
X-SVN-Rev: 28562
2010-08-31 15:16:52 +00:00
Markus Scherer
9c5573b1f9 ICU-7264 mark characters as disallowed_STD3_valid if their decompositions contain STD3-disallowed characters
X-SVN-Rev: 28561
2010-08-31 15:10:37 +00:00
Markus Scherer
ab9fc77dfb ICU-7264 generate UTS #46 data with disallowed_STD3_valid and disallowed_STD3_mapped
X-SVN-Rev: 28560
2010-08-31 05:48:38 +00:00
Markus Scherer
4473e0a758 ICU-7264 switch ubidi.icu and ucase.icu from UTrie to UTrie2
X-SVN-Rev: 28396
2010-07-31 19:16:48 +00:00
Markus Scherer
3183eeaac0 ICU-7264 switch uprops.icu from UTrie to UTrie2
X-SVN-Rev: 28385
2010-07-29 05:00:55 +00:00
Markus Scherer
3bf87d9766 ICU-7264 parse ScriptExtensions.txt, write uprops.icu formatVersion 7
X-SVN-Rev: 28377
2010-07-27 23:32:04 +00:00
Markus Scherer
4824f9c355 ICU-7264 genuca with new nfc.nrm, document sequencing a bit
X-SVN-Rev: 28368
2010-07-26 21:04:16 +00:00
Markus Scherer
58d21ee915 ICU-7264 scripts for data file generation
X-SVN-Rev: 28364
2010-07-23 23:51:14 +00:00
Markus Scherer
fa05e3a3d3 ICU-7264 delete temporary comment in genpname/preparse.pl
X-SVN-Rev: 28360
2010-07-22 23:34:36 +00:00
Markus Scherer
e72d90de1a ICU-7264 add ScriptExtensions.txt, new scripts, new blocks, fix genpname/preparse.pl
X-SVN-Rev: 28359
2010-07-22 23:30:47 +00:00
Stuart Gill
5aaa8be80e ICU-7611 Generalize library references
X-SVN-Rev: 28337
2010-07-21 00:39:45 +00:00
Markus Scherer
5d2aaca2a9 ICU-7167 fix rudimentary CE bytes check in genuca
X-SVN-Rev: 28194
2010-06-15 06:06:24 +00:00
Markus Scherer
55167fbd50 ICU-7611 add readme.txt
X-SVN-Rev: 28136
2010-06-04 19:07:33 +00:00
Markus Scherer
7d8d37e7b9 ICU-7611 reduce gennorm so it only writes .txt files for gennorm2, without code for building unorm.icu
X-SVN-Rev: 28133
2010-06-04 18:01:08 +00:00
Markus Scherer
311eb8cb78 ICU-7611 CMake files for Unicode build tools
X-SVN-Rev: 28129
2010-06-03 22:47:48 +00:00
Markus Scherer
e996a945d9 ICU-7611 move the gennorm tool to tools/c/unicode/ where its siblings are
X-SVN-Rev: 28128
2010-06-03 22:32:58 +00:00
Markus Scherer
ecd660579b ICU-7144 handle deviation characters and disallowed ASCII characters in code
X-SVN-Rev: 27999
2010-04-29 00:33:58 +00:00
Markus Scherer
9bbee2c4ba ICU-7144 adjust to IdnaMappingTable.txt separating reserved and assigned ranges
X-SVN-Rev: 27949
2010-04-19 21:05:15 +00:00
Markus Scherer
33aac993a5 ICU-7611 move genuca from icu tree to tools tree
X-SVN-Rev: 27927
2010-04-10 04:15:40 +00:00
Markus Scherer
baed720ac1 ICU-7611 move genprops from icu tree to tools tree
X-SVN-Rev: 27923
2010-04-09 23:36:31 +00:00
Markus Scherer
0b205c3cb2 ICU-7611 move genpname from icu tree to tools tree
X-SVN-Rev: 27922
2010-04-09 23:36:05 +00:00
Markus Scherer
d14ce517fd ICU-7611 move gennames from icu tree to tools tree
X-SVN-Rev: 27921
2010-04-09 23:35:40 +00:00
Markus Scherer
ae1317acc3 ICU-7611 move gencase from icu tree to tools tree
X-SVN-Rev: 27920
2010-04-09 23:35:13 +00:00
Markus Scherer
205add1f63 ICU-7611 move genbidi from icu tree to tools tree
X-SVN-Rev: 27919
2010-04-09 23:34:04 +00:00
Markus Scherer
5f1d46ae5b ICU-7144 remove some debug code
X-SVN-Rev: 27908
2010-04-08 22:55:04 +00:00
Markus Scherer
0c7723b943 ICU-7144 remove label separators from the base exclusion set
X-SVN-Rev: 27886
2010-03-30 23:33:50 +00:00
Markus Scherer
d0a3f751a8 ICU-7144 make all unassigned code points disallowed (changes Default_Ignorable_Code_Point from ignored)
X-SVN-Rev: 27885
2010-03-30 22:23:59 +00:00
Markus Scherer
04f3398d30 ICU-7144 derive the base exclusion set by comparing IDNA2003 with NFKC_Casefold (follow change in UTS #46)
X-SVN-Rev: 27880
2010-03-29 22:29:36 +00:00
Markus Scherer
97c3e0ae45 ICU-7144 fix 2260, 226E, 226F mappings
X-SVN-Rev: 27793
2010-03-05 22:24:48 +00:00
Markus Scherer
46ec4b3cc7 ICU-7144 script to turn Unicode's IdnaMappingTable.txt into ICU gennorm2 source file format
X-SVN-Rev: 27791
2010-03-05 22:12:37 +00:00
Markus Scherer
cb3d8ade6d ICU-7144 quick & dirty tool to recreate the UTS #46 data table according to the spec
X-SVN-Rev: 27751
2010-03-02 22:59:05 +00:00