Commit Graph

22750 Commits

Author SHA1 Message Date
Hugo van der Merwe
cf46b4136e ICU-20568 unitsdata.cpp/.h and associated tests
Read unitsTest.txt, prepare to run unit tests.
PR: https://github.com/sffc/icu/pull/20
Commit: 9f65f41f183ded57efdf4f7c0cdf56c02e441024

No-op/cosmetic: reformat with clang-format ident:4 line-length:105.
PR: https://github.com/sffc/icu/pull/25
Commit: 4be0a2bc55fc3567f790a53cc988abd4c3508419

Parse unitPreferencesTest.txt preparing to run data-driven tests
PR: https://github.com/sffc/icu/pull/26
Commit: 9e021f9b6fbee06283ef4c45dbfe5c7b3abd2dbf

Improve test output: no more fprintfs.
PR: https://github.com/sffc/icu/pull/34
Commit: 12a5289238d68e3aa184727d7d7af55d386613c7

Update to latest CLDR test files: b7a23f3f41 CLDR-13587.
PR: https://github.com/sffc/icu/pull/33
Commit: 44e9afb32187529a4599548c42f70c7deb3e46bf

Add unitsdata.cpp, getConversionRatesInfo, and unit tests.
PR: https://github.com/sffc/icu/pull/31
Commit: ca34233e0804c1175f210e5557bb984da2eab244

Update unitsTest.txt and unitPreferencesTest.txt from current CLDR master
PR: https://github.com/sffc/icu/pull/41
Commit: be7f69941ddf4373e9d66149e95f5b8345d7de5d
2020-09-10 22:39:18 +02:00
younies
adcc646e51 ICU-20568 Skeleton for Testing Units
PR: https://github.com/sffc/icu/pull/6
Commit: 9b3ca66937c40044055bf9a8e13db57bcdc75fec
2020-09-10 22:39:18 +02:00
Jeff Genovy
9a220a7381 ICU-21272 Add Windows ARM64 Debug build to CI build configuration. 2020-09-10 12:22:19 -07:00
Markus Scherer
18c4a69f80 ICU-9961 replace U_DRAFT/U_STABLE/U_INTERNAL with U_CAPI 2020-09-10 11:23:44 -07:00
shaobero
e25390f469 ICU-21272 Making changes in makedata.mak to fix Debug cross-build for ARM & ARM64 2020-09-10 10:10:08 -07:00
Frank Tang
35dfdf1d02 ICU-21043 Fix calendar problems
This PR fixes
ICU-21043 Erroneous date display in indian calendar of all dates prior to 0001-01-01.
ICU-21044 Hebrew Calendar calculation is incorrect when the year < 1
ICU-21045 Erroneous date display in islamic and islamic-rgsa calendars of all dates prior to 0622-07-18.
ICU-21046 Erroneous date display in islamic-umalqura calendar of all dates prior to  -195366-07-23.

The problem in the indiancal.cpp is
* ICU-21043 the gregorian/julain convesion is wrong. Swith to use
i18n/gregoimp.h fix the problem.

The problem in the hebrwcal.cpp is
* ICU-21044 Use ClockMath::floorDivid to replace x / y for x maybe <  0
also need to fix incorrect value in test file

The problem in the islamcal.cpp:
* ICU-21045: The math of % negative number for year and month is wrong.
* ICU-21046: Not use int64_t methods and caused int32_t overflow while the year is a
huge negative number. Cast to int64_t to force using the int64_t version
for the math fix the problem.

Also add tests to exhaust test 8000 years for all calendar. In quick
mode, only test 2.5 years.

Switch to 0-based month. fix TZ for test.
2020-09-10 00:21:18 -07:00
gnrunge
9e85ba5690 ICU-21268 Adds NULL string checks to assert statements to prevent
segmentation faults when unit tests run without ICU data.

ICU-21268 Fix value type: char* is not ompatible with u'..' value.
2020-09-09 17:50:45 -07:00
Markus Scherer
c3fe7e09d8 ICU-21267 stop defining & using FALSE & TRUE macros in public headers
See #1282
2020-09-09 15:13:56 -07:00
Peter Edberg
ad043349ed ICU-21106 avoid buffer overflow in u_fopen_u, fix WIN32_API fclose(NULL) 2020-09-09 00:21:15 -07:00
gnrunge
b6ff1393d5 ICU-21268 Add error checks to avoid segmentation faults if tests run with stubdata only.
ICU-21268 Use 'nullptr' instead of NULL when checking for pointer value.

ICU-21268: Fixes error message.

ICU-21268 Style fixes (space before '{').
2020-09-08 18:22:29 -07:00
Jeff Genovy
2361a3258b ICU-21264 Re-enable optimizations for various functions on MSVC for VS2019 and above. 2020-09-08 18:05:02 -07:00
Jeff Genovy
941fddc19d ICU-21108 Add build using VS2015 toolset, fix job names. 2020-09-08 13:58:38 -07:00
Jeff Genovy
ef4913316f ICU-21271 Fix valgrind failure in dtptngen.cpp 2020-09-08 13:52:53 -07:00
Hugo van der Merwe
8e04bf016d ICU-21174 Drop MaybeStackVector::appendAll because it requires copy constructor 2020-09-05 23:29:33 +02:00
Peter Edberg
b066f65a50 ICU-21249 integrate CLDR release-38-alpha1 to ICU trunk 2020-09-04 15:05:22 -07:00
Rich Gillam
ff2272d7e1 ICU-21164 Fixed bug in OlsonTimeZone constructor that could leave the object in a not-fully-initialized state. 2020-09-04 10:09:39 -07:00
Markus Scherer
9971c663ff ICU-21257 remove #License fragment from license URLs 2020-09-04 10:02:17 -07:00
Shane F. Carr
4f18ef2ef8 ICU-21258 Refactor code and tests for compact data known issue 2020-09-03 16:17:40 -07:00
Rich Gillam
edaebfa64e ICU-21155 Add support for all hour and day-period skeleton characters to DateIntervalFormat 2020-09-03 16:14:08 -07:00
Shane F. Carr
caadb345e5 ICU-20826 Updating C++ ListFormatter impl to be more like Java 2020-09-03 17:52:03 -05:00
Fredrik Roubert
a827ab72c9 ICU-21035 Delete meaningless calls to uloc_getKeywordValue().
These two calls to uloc_getKeywordValue() write to the buffer "id" which
then immediately after is overwritten by calls to idForLocale() so they
can simply be removed without any loss of functionality.
2020-09-03 20:11:21 +02:00
Fredrik Roubert
936f53a1f1 ICU-21035 Update locale implementation to use ulocimp_getKeywordValue(). 2020-09-03 19:02:47 +02:00
Elango Cheran
cb7f197821 ICU-21254 Add plural rule parsing for exponent operand in C++ 2020-09-03 09:17:15 -07:00
Shane F. Carr
82545ecc2c ICU-21249 Update instructions in StaticUnicodeSetsTest 2020-09-03 12:29:44 +02:00
Shane F. Carr
28707b44b4 ICU-21249 Patch for failing exhastive unit tests 2020-09-03 12:29:44 +02:00
Fredrik Roubert
f9ada4b3cb ICU-21035 Pass ByteSink from Locale::getKeywordValue() to uloc_getKeywordValue().
This eliminates the need for a scratch buffer in Locale::getKeywordValue()
and also the need for counting bytes required in uloc_getKeywordValue(),
something that ByteSink will now handle correctly.
2020-09-03 00:48:43 +02:00
Shane F. Carr
fcc3bcb43e ICU-21249 Update numberpermutationtest.txt 2020-09-03 00:46:59 +02:00
Frank Tang
08aafba664 ICU-21236 Refactor UniqueCharStrings / CharStringMap
Move them UniqueCharStrings and CharStringMap from
loclikelysubtags.{h,cpp} to separate header files
so so we can reuse them to implement
https://github.com/unicode-org/icu/pull/1254
2020-09-02 13:45:10 -07:00
Jeff Genovy
341608f7b6 ICU-21108 Update to use/support VS2019, and add extra CI builds for VS2017 and VS2019.
Change to use the Windows 10 SDK for Win32 (Win7) builds on the VS2019 ADO images.
The Windows 10 SDK is backwards compatible to Windows 7, if WINVER and
_WIN32_WINNT are set before compiling.

Clean up the vcxproj files from VC7.1
2020-09-02 13:01:27 -07:00
gnrunge
9a54efc5c7 ICU-21261 Adds 'units' data to pseudolocale data generation. Also corrects the documentation in a few places. 2020-09-02 10:56:56 -07:00
Peter Edberg
e618a1cc2d ICU-21249 integrate CLDR release 38 alpha0 to ICU trunk 2020-09-02 10:23:14 -07:00
younies
15747825bc ICU-20963 Add .clang-format file to icu4c/source
See #1143
2020-09-01 21:19:06 +02:00
Frank Tang
bc864aeac6 ICU-21051 Add copyright Remove License from sh
See #1281
2020-09-01 13:26:41 -05:00
Shane F. Carr
196d5e1182 ICU-20923 Fix compact notation with percent. 2020-09-01 04:36:04 -05:00
Markus Scherer
2f39d33498 ICU-13727 limit length of input for Punycode encode/decode
See #1241
2020-08-31 16:53:07 -07:00
Erik Torres
ba3b6acd5c ICU-21249 BRS68RC Version update and regenerate configure for v68
BRS68RC Updating files for release
2020-08-31 19:35:14 -04:00
Victor Chang
7a57d299df ICU-21247 Move #include "unicode/localpointer.h" into U_SHOW_CPLUSPLUS_API block 2020-08-31 15:19:29 -07:00
Markus Scherer
650bd402fe ICU-13727 add U_INPUT_TOO_LONG_ERROR & ICUInputTooLongException 2020-08-31 10:32:11 -07:00
Fredrik Roubert
ffd450c637 ICU-21035 Replace backward compatibility _getVariant*() functions.
The one single caller of _getVariantEx() is _getVariant() and this in
turn only has 3 callers, one of which is commented out (so it can be
deleted), one of which doesn't actually do anything (so it too can be
deleted) and one which can be replaced by inlining the use of
CheckedArrayByteSink.

This also allows _getVariantEx() to be renamed to _getVariant() as it's
the only such function left now.
2020-08-31 13:33:15 +02:00
Markus Scherer
f62693aa02 ICU-13416 change Armenian (hy) uppercase/titlecase of և ligature ech-yiwn 2020-08-30 18:19:10 -07:00
Frank Tang
cad3903cc2 ICU-21051 Add Locale::createCanonical performance test 2020-08-29 02:29:58 -07:00
Shane F. Carr
0101e2632c ICU-20164 Make NoUnit a zero-cost abstraction over MeasureUnit.
See #1230
2020-08-29 00:01:49 -05:00
Hugo van der Merwe
35eae09a7c ICU-21246 Handle kilogram SI prefix correctly
Fix kilogram parsing: ignore 'kilogram' as a stem, we have 'gram'.

Failures in the added unit test before the fix: withSIPrefix resulted
in 'microkilogram', and kilogram's prefix was considered to be
"ONE" (i.e. 10^0).
2020-08-29 00:07:08 +02:00
Shane F. Carr
0dbe1ae013 ICU-20927 Adding French narrow duration unit test. 2020-08-28 15:31:38 -05:00
Craig Cornelius
408cd128fc ICU-21242 rephrase documentation using term master
See #1255
2020-08-28 12:42:20 -07:00
David Beaumont
9bcd35aee0 ICU-21240 Removing unused makefile files and updating docs. 2020-08-28 20:41:03 +02:00
Hugo van der Merwe
66b2458a26 ICU-21066 Copy includelist of CLDR testData with tools/cldr/build.xml 2020-08-28 01:12:12 +02:00
Shane F. Carr
59cdbe1f53 ICU-21232 Improve performance of DecimalFormat#parse on long numbers 2020-08-27 14:33:05 -05:00
Fredrik Roubert
05d49fd373 ICU-21035 Replace backward compatibility locale_getKeywords() function.
By updating the last 3 callers to do dynamic memory allocation instead,
the fixed memory buffer function becomes obsolete.
2020-08-27 13:46:07 +02:00
Fredrik Roubert
d070fdddd1 ICU-21035 Remove obsolete use of CharString::getAppendBuffer().
The complicated buffer allocation code is inherited from times past but
no longer serves any purpose, it's now possible to instead simply call
the CharString copy constructor.
2020-08-26 13:49:26 +02:00
Fredrik Roubert
e8f3d5c657 ICU-21035 Replace backward compatibility ulocimp_getCountry() overload.
By always calling the dynamic memory allocation implementation directly
instead, the fixed memory buffer boundary gets pushed one step further
towards the edges.
2020-08-26 13:48:54 +02:00
Fredrik Roubert
db3278a71e ICU-21035 Replace backward compatibility ulocimp_getScript() overload.
By always calling the dynamic memory allocation implementation directly
instead, the fixed memory buffer boundary gets pushed one step further
towards the edges.
2020-08-25 00:30:54 +02:00
Frank Tang
3b188e46a4 ICU-21157 Fix test code
See #1256
2020-08-24 14:37:09 -07:00
Hugo van der Merwe
0387d1b988 ICU-21230 Add status to MaybeStackArray(int32_t newCapacity), plumb it through. 2020-08-24 22:55:26 +02:00
Markus Scherer
c8aa800735 ICU-21035 add & use CharString::extract(dest buffer)
See #1253
2020-08-21 16:29:25 -07:00
Markus Scherer
39da689d30 ICU-21184 rephrase docs/comments using the term grandfathered 2020-08-21 14:13:03 -07:00
Frank Tang
cde54fc5ba ICU-21157 Fix incorrect null termination.
See #1236
2020-08-20 16:22:34 -07:00
Fredrik Roubert
3a8e0a2fe9 ICU-21035 Replace backward compatibility ulocimp_getLanguage() overload.
By always calling the dynamic memory allocation implementation directly
instead, the fixed memory buffer boundary gets pushed one step further
towards the edges.
2020-08-20 23:38:30 +02:00
Erik Torres
a7a8cc3809 ICU-20808 Add test for the C locale to default to en_US_POSIX 2020-08-20 14:12:09 -07:00
Jeff Genovy
26775d7d13 ICU-21159 Fix memory leaks in cintltst TestUsingDefaultWarning. 2020-08-19 16:50:55 -07:00
David Beaumont
4cf2379d3c ICU-21066 Improvements in Ant build files
See #1224
2020-08-19 12:44:40 +02:00
Shane F. Carr
d2789a035b ICU-20774 Add option to icupkg to allow missing deps between resource files. 2020-08-17 20:09:08 -05:00
Frank Tang
e395e8abcd ICU-21159 Document U_USING_DEFAULT_WARNING in .h
Document the fact
uloc_getDisplay(Language|Script|Country|Variant|Keyword|KeywordValue)
would fallback with the code, case canonicalied in same cases, and
set the status to U_USING_DEFAULT_WARNING.

No change to the implementation behavior. Only complete the missing
comments and tweak line wrap, remove double spaces and add test to
validate this pre-existing behavior that I added the documents now.
2020-08-17 15:18:01 -07:00
Markus Scherer
9279e49d2f ICU-21030 validate ACE label edge cases 2020-08-14 14:32:47 -07:00
Markus Scherer
e19d12997b ICU-21212 full range check for Punycode digits 2020-08-14 11:37:40 -07:00
Paul Smith
e3f2c0dd70 ICU-21217 Windows: Fix link command for extra/scrptrun 2020-08-14 00:43:24 -07:00
Shane F. Carr
2ea56b531a ICU-21182 Adding UNumberRangeFormatter (C API).
See #1193
2020-08-12 21:33:38 -05:00
Shane F. Carr
88f092a9fb ICU-21191 Improve configurability of testtagsguards.sh 2020-08-11 21:22:31 -05:00
Shane F. Carr
bf2c2c5ca7 ICU-21176 Add aliases for terms "whitelist" and "blacklist" in data filter
See #1189
2020-08-11 21:21:59 -05:00
younies
7997955f2d ICU-21174 Add error-checking methods to MaybeStackVector & MemoryPool.
- MaybeStackVector::emplaceBackAndCheckErrorCode()
- MemoryPool::createAndCheckErrorCode()

Started with cherry-picks of:
3b505977c0e82659896125698389b59eabe50b14
63b93bde5c4c9fe030d490d5d448087aa0d4e5fd
2020-08-11 19:52:09 +02:00
Frank Tang
0eed48038b ICU-20725 Fix stack overflow of u_unescapeAt
See #1207
2020-08-10 14:59:38 -07:00
Paul Smith
ed268e77e3 ICU-21216 Windows: Ensure directory exists before running rc
When building in parallel it's possible that the rc command is invoked
before other rules get around to creating the output directory.
2020-08-10 12:16:47 -07:00
Hugo van der Merwe
4276d83e8a ICU-21174 Debug msg nitpick: s/MaybeStacArray/MaybeStackArray/ 2020-08-07 19:11:00 +02:00
Hugo van der Merwe
66d8df5338 ICU-21174 Delete not-to-be-used methods. Add a constructor warning. 2020-08-07 19:11:00 +02:00
Hugo van der Merwe
abdf0e667d ICU-21174 Add a memory allocation check in MeasureUnit. 2020-08-06 13:17:34 +02:00
Hugo van der Merwe
0b815fb8c3 ICU-21059 Load simple unit IDs from convertUnits.
PR: https://github.com/icu-units/icu/pull/41
Commit: 7877f0409019827b2d8d43b0843656322181972b
2020-08-05 10:57:19 +02:00
Hugo van der Merwe
5d97c9f13e ICU-21174 Increase safety of Measure and writeAffixes.
Measure: initialize unit to nullptr, and don't dereference it if it is
nullptr.

NumberFormatterImpl::writeAffixes: U_ASSERT not-null, instead of
segfaulting for coding mistakes.
2020-08-05 01:52:40 +02:00
Shane F. Carr
ee2d8b0103 ICU-21175 Add cnvalias as a dependency of misc_res 2020-08-04 18:50:43 -05:00
Hugo van der Merwe
6b595d1c01 ICU-21076 Delete unneeded MeasureUnit data & code 2020-08-05 01:48:32 +02:00
Frank Tang
863582c2a4 ICU-20465 Calendar/DateFormat listen to tz extension
See #1176
2020-08-04 13:33:03 -07:00
Frank Tang
8ca80c4b6d ICU-21158 Fix doc of UDISPCTX_NO_SUBSTITUTE
See #1200
2020-07-31 18:39:46 -07:00
Frank Tang
7ddc231195 ICU-20734 Improve fuzzer_driver
See #1204
2020-07-31 15:30:03 -07:00
Frank Tang
41d1d57af0 ICU-21122 Fix flaky TestAdoptCalendarLeak 2020-07-29 20:39:55 -07:00
Frank Tang
d7ec310436 ICU-20684 Fix uninitialized in isMatchAtCPBoundary
Downstream bug https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=15505
Fix Fuzzer-detected Use-of-uninitialized-value in isMatchAtCPBoundary

To test to show the bug in the new test case, configure and build with
CFLAGS="-fsanitize=memory" CXXFLAGS="-fsanitize=memory" ./runConfigureICU \
  --enable-debug --disable-release  Linux  --disable-layoutex

Test with
cintltst /tsutil/custrtst
2020-07-29 14:21:53 -07:00
Andy Heninger
895aff3bff ICU-21178 Add check for corrupt rbbitst.txt data.
In the test data from rbbitst.txt, two or more adjacent boundary markers with
no intervening test data were accepted, with no indication of a problem.

This situation occurred, as described in bug ICU-21178, with a bad import of
some test cases from CLDR. PR #1194 corrected the problem with the test data
in ICU4C. This PR adds code to flag this situation in the test data, and
also propagates the data fix to ICU4J's copy of rbbitst.txt.
2020-07-24 15:16:12 -07:00
Frank Tang
0d4b1c1cb9 ICU-21160 Fix the length return by preflight
See #1178
2020-07-21 18:05:20 -07:00
Andy Heninger
003b431540 ICU-13590 RBBI, improve handling of concurrent look-ahead rules.
Change the mapping from rule number to boundary position to use a simple array
instead of a linear search lookup map.

Look-ahead rules have a preceding context, a boundary position, and following context.
In the implementation, when the preceding context matches, the potential boundary
position is saved. Then, if the following context proves to match, the saved boundary is
returned as an actual boundary.

Look-ahead rules are numbered, and the implementation maintains a map from
rule number to the tentative saved boundary position.

In an earlier improvement to the rule builder, the rule numbering was changed to be a
contiguous sequence, from the original sparse numbering. In anticipation of
changing the mapping from number to position to use a simple array.
2020-07-21 14:39:15 -07:00
Ramon
2de2585f1b ICU-13339 Do not parse decimal point for integers 2020-07-20 23:52:59 -05:00
Hugo van der Merwe
e734111ee5 ICU-21192 MeasureUnit Identifier spec compliance: s/p/pow/
Specification:
https://www.unicode.org/reports/tr35/tr35-general.html#Unit_Identifiers
2020-07-16 01:58:32 +02:00
David Beaumont
dfc8b8b746 ICU-20697 Delete now unused files and documentation for the old ICU LDML tooling. 2020-07-14 20:27:28 +02:00
Michael Block
f917c43cf1 ICU-21178 Adding the trailing space back into two RBBI test cases. 2020-07-07 16:05:05 -07:00
Makoto Kato
c9037ca8d3 ICU-11992 uprv_tzname doesn't return valid time zone on Android 2020-07-06 10:11:20 -07:00
John Wilcock
6fe86f3934 ICU-21173 Add support for more currency variants. ICU4C equivalent of…
See #1184
2020-07-03 04:51:15 +02:00
Hugo van der Merwe
3fca290880 ICU-21174 Add a const version of MaybeStackVector::getAlias().
(Also makes a tiny tweak to appendAll() documentation.)
2020-07-02 01:56:08 +02:00
Markus Scherer
4d428cb8f3 ICU-21176 spoof checker: remove whitelist/blacklist metaphors from API docs 2020-07-01 15:21:05 -07:00
John Wilcock
9219c6ae03 ICU-13733 Added test for mismatching currency format for strict-mode parsing
See #1169
2020-06-30 02:22:57 +02:00
Diego Barrios Romero
de0306daaa ICU-21170 Fix function prototypes 2020-06-25 15:31:11 -07:00
Łukasz Wojniłowicz
ed56301abd ICU-20545 Ensure that path ends with detected file separator
CharString, when asked, appends U_FILE_SEP_CHAR at the end of the string
it holds, if it won't find U_FILE_SEP_CHAR or U_FILE_ALT_SEP_CHAR there.
The problem starts if the dir variable uses
U_FILE_ALT_SEP_CHAR which is not equal to U_FILE_SEP_CHAR. Then the
resulting path could look like this
../data\
instead of this
../data/

This patch uses U_FILE_SEP_CHAR unless it detects that the dir variable
doesn't use it, and uses U_FILE_ALT_SEP_CHAR instead.
2020-06-24 11:38:41 -07:00
Markus Scherer
ef12882fdb ICU-21144 LocaleMatcher setMaxDistance(), isMatch() 2020-06-23 13:56:49 -07:00
Andy Heninger
99dc49a0c0 ICU-20869 Fix compiler warning in FixedDecimal::getFractionalDigits().
Fix a clang compiler warning and a potential undefined behavior arising
from casting an out-of-range double to an int. See the Jira ticket for a
more detailed description of the problem.

This PR is to fix the immediate problem. Longer term, the function
may be replaced entirely - see issue ICU-21147.
2020-06-19 11:31:42 -07:00