This change adds support for building the ICU4C UWP projects for ARM64.
Additionally it adds CI builds for ARM32 and ARM64.
Co-Authored-By: Daniel Ju <daju@microsoft.com>
The Azure DevOps VM images that are used in the CI builds recently
updated the version of MSYS that is installed, which is causing the
MSYS builds to fail. It seems that this new version has issues with
using the shell cmd method in the makefiles to obtain the current path.
However, we can avoid this entirely by using the cygpath utility, which
also avoids the overhead of launching a new cmd instance just to get
the path. Unfortunately, it seems that older versions of MSYS don't
have this utility program included by default, though versions of
MSYS 2 and up do have it included by default.
We can continue using the old approach for the older versions and use
the cygpath approach for new versions.
RulebasedCollator().
ICU-20652 Adds test/fuzzer/Makefile (auto-generated upon ICU4C configuration)
to .gitignore.
ICU-20652 In response to PR#693 review, corrects allocation size of char16_t
buffer.
While at it, adds generated files to .gitignore.
targets to test/fuzzer/ directory. This will enable compilation and
smoke test of fuzzer targets as part of the ICU continuous build.
ICU-20652 Fixed exit-on-error behaviour of fuzzer targets execution.
Minor clean-ups and improvements
ICU-20652 Modifies fuzzer/Makefile.in to fix Windows build issue.
ICU-20627 Adds explicit enablement of fuzzer targets build to ICU4C
configuration and Makefile.in. File 'configure' was created from
'configure.ac' by executing 'autoreconf'.
autoreconf added some new entries into 'configure' about runstatedir. Not sure
why it did this, they are not related to fuzzer.
ICU now uses namespaces. When trying to use the scrptrun.h header, the
header cannot compile stand-alone as it is inheriting from UObject which
is namespaced now. Add namespace macros to properly inherit.
ICU-20566 Add scrptrun to the extra high level Makefile
The scrptrun project was not being added to the extra portion of the ICU
build. Add it as a sub project so it can always be built.
ICU-20566 Properly namespace the ScriptRun test
The ScriptRun class is now namespaced. Update the test reference.
ICU-20566 Use int instead of void to compile
Clang will complain about having a void main without an int main to
invoke. Change void to int and return 0.
- Use STATIC_NEW for mutex creation, to avoid order-of-destruction problems
by avoiding destruction altogether, while avoiding memory leak reports.
- Remove UConditionVar, replace with direct use of std::condition_variable
Regression was in 1afef30549
PR #418 [ICU-20187]
- We dropped the mapping from "C" in uloc_canonicalize,
but then putil did not handle cases where a codepage was
set (such as C.UTF-8).
- Add an additional check in uprv_getDefaultLocaleID() for
locales that end up as "C" or "POSIX" after removing codepage
suffix.
- Also fix regression where aa@bb would become aa__BB__BB
(incorrectly doubled __BB)
Remove explicit extern "C++" scope.
Remove C++ macros in C mode.
Fix issues detected by make -C test/hdrtst.
Run test/hdrtest during CI to detect future breakage.
Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
This fixes a regression introduced by commit
b12a927c93 for issue ICU-13778.
The above commit improved the error checking in the
DateTimePatternGenerator class, adding checks for errors/failures
where there previously was none at all. This was done in order to
catch catastrophic errors like out-of-memory (OOM), and properly
report them to the caller, rather than ignoring/hiding these errors.
However, in doing so it exposed a case where the code was depending
on ignoring errors in order to fall-back to the Gregorian calendar
when the default ICU locale is set to root.
This restores the previous behavior, by allowing the error of
U_MISSING_RESOURCE_ERROR to fall-though and continue without
reporting back an error to the caller.
Note: This regression was technically introduced in ICU 63, and
also effects ICU 64 as well.
- in the rpath case, we do NOT want a trailing space in LD_SONAME
regression was introduced in ICU-20526 with
83a0542b5b (master)
b76cb6517e (in maint-64)
(cherry picked from commit 1c553b9cf2)
- in the rpath case, we do NOT want a trailing space in LD_SONAME
regression was introduced in ICU-20526 with
83a0542b5b (master)
b76cb6517e (in maint-64)
- StringSegment, ICU4C:
* Moved to top icu namespace
* Compilation unit renamed to string_segment.
- NumberStringBuilder, C and J:
* Moved to main icu namespace
* Compilation unit renamed to formatted_string_builder
* Renamed class to FormattedStringBuilder
- Moves nextPosition logic of NumberStringBuilder to helper class
- added PKGDATA_TRAILING_SPACE to all of the pkgdataMakefile.in file.
- NOTE: Users who create their own pkgdata.inc / icupkg.inc files may need
to recreate this PKGDATA_TRAILING_SPACE behavior.
- used the above variable, normally undefined, in mh-* files that need a trailing space
- Also, fixed use of system() in pkgdata.cpp per ICU-20538
This was causing pkgdata to return a zero status even on clang
failure, masking this issue.
(cherry picked from commit 83a0542b5b)
- added PKGDATA_TRAILING_SPACE to all of the pkgdataMakefile.in file.
- NOTE: Users who create their own pkgdata.inc / icupkg.inc files may need
to recreate this PKGDATA_TRAILING_SPACE behavior.
- used the above variable, normally undefined, in mh-* files that need a trailing space
- Also, fixed use of system() in pkgdata.cpp per ICU-20538
This was causing pkgdata to return a zero status even on clang
failure, masking this issue.
- remove the old LocaleMatcher implementation code
- move the XLocaleMatcher code into LocaleMatcher, same for test
- remove unused internal methods
- stop comparing old vs. new performance
- generate langInfo.txt resource bundle file with precomputed likely-subtags and matcher data
- make genrb handle multi-line binary values
- load likely-subtags & distance data from new langInfo.res bundle
- test that built data == loaded data
- move data builders to tools, no more runtime dependency on builder code
Builds res_index.txt based on directory glob minus aliases read from deprecates XML file.
In ICU 64, please use the ICU Data Build Tool instead of reslocal.mk for locale filtering.
This change was introduced by the following commit e9946ec98e.
However, there's really no reason why the Windows UWP version should require the icudtl.dat file
to be in the same directory as the icuuc.dll file.
Remove the dependencies from the ICU library code on static constructors
that were introduced by using std::mutex and condition variables. The
mutexes are lazily initialized by embedding them as local static variables
in getter functions, and relying on the C++ compiler/runtime to do thread
safe initialization of them.
- see also ICU-20062
- add a `-B` option to the two python invocations on Windows
- set PYTHONDONTWRITEBYTECODE in configure.ac and icudefs.mk.in
Co-authored-by: Fredrik Roubert <roubert@google.com>
- If icu/source/data/locales/root.txt missing, skip
python rules.mk generation.
- Also, create build directories properly as needed
- Also includes noise changes to configure
(configure was probably generated using unreleased
autoconf 2.70 or 2.69 + patches)
- eac8f4b31a did not
regen configure properly, so BUILDTOOL_OPTS is now
ICU_DATA_BUILDTOOL_OPTS
- If icu/source/data/locales/root.txt missing, skip
python rules.mk generation.
- Also, create build directories properly as needed
- Also includes noise changes to configure
(configure was probably generated using unreleased
autoconf 2.70 or 2.69 + patches)
- eac8f4b31a did not
regen configure properly, so BUILDTOOL_OPTS is now
ICU_DATA_BUILDTOOL_OPTS
- Fixes filterrb.cpp to check for wildcard when at a leaf.
- Adds additional verbose logging to genrb.
- Fixes filtration to add deps to dep_targets instead of dep_files.
- Separates dep_files to common_dep_files and specific_dep_files.
- use git instead of svn for versioning
- update generated file names to match official downloads
Example filenames:
- icu4c-63_1-c5749aaf94-src.tgz (includes git hash or tag)
- icu4c-63_1-src.tgz (hard link to above)
- icu4c-src.tgz (symlink)
- icu4c-63_1-c5749aaf94-src.zip (includes git hash or tag)
- icu4c-63_1-src.zip (hard link to above)
- icu4c-src.zip (symlink)
- icu4c-63_1-c5749aaf94-data.zip (includes git hash or tag)
- icu4c-63_1-data.zip (hard link to above)
- icu4c-data.zip (symlink)
- icu4c-63_1-c5749aaf94-docs.zip (includes git hash or tag)
- icu4c-63_1-docs.zip (hard link to above)
- icu4c-docs.zip (symlink)
Methods implementead as 'inline' but not declared 'inline' cause clang++
to throw compilation warnings on Windows. This adds 'inline' to the
relevant method declarations.
This eliminates the need for scratch buffers in any code path that ends
with these functions and also eliminates the need for counting bytes,
something that ByteSink will now handle correctly when needed.
Existing calls to uloc_addLikelySubtags() and uloc_minimizeSubtags()
throughout ICU4C implementation code are also updated to instead use
either the Locale or ulocimp_* functions with the new API.
None of this should have any externally visible effect, it's all about
cleaning up implementation internals.
from assumed UTF-8 resulted in an extremely large percentage of Unicode
replacement characters in the data passed to the API under test.
ICU-20217 Uses fuzzer generated bytes to make random selection of locales, converters,
etc., replacing the random number generator. This way the fuzzer can control
the selections.
ICU-20217 Minor follow-ups from code review.
Removes fuzzer target break_iterator_utf32_fuzzer which does not perform
anything useful what the regular break iterator fuzzer target already performs.
ICU-20217 Fixes for-loop body.
ICU-20217 Uses am allocated buffer to pass head-truncated fuzzer data to the
API under test. The fuzzer may otherwise not detect buffer underflow.
by
ICU-20217 Typing fix.
ICU-20217 Fixing typing.
ICU-20217 Improve fuzzer targets, move truncated fuzzer data into a
new buffer to prevent that buffer underflow goes undetected.
ICU-20217 Fixes buffer management of fuzzer-provided data.
ICU-20217 Factor in PR review comments.
Cherry-pick cldrbug 11492: Bad symbols for NaN in sv, ksh, kl, se locales.
Cherry-pick cldrbug 11491: sd, month name for July uses character not in exemplars.
Updated the various ICU4J *.jar files as well.
- Changes Java DecimalFormat boolean get* methods to is*.
- Makes the new draft methods non-virtual.
- Removes obsolete template class in header file.
- Adds proper U_HIDE tags in unum.h and decimfmt.h
If a file with an input line larger than INT32_MAX (i.e. 2 GB) contains
an UTF8 character after that limit, escapesrc crashes on 64 bit systems
or does not remove incomplete files on 32 bit systems.
The issue is that an unchecked cast from size_t to int32_t can turn
negative, which results in negative offsets during array access.
This will eventually lead to an out of boundary read, which most likely
crashes the tool.
This patch sets a fixed limit on 1 GB to make sure that no side effects
occur if the line is exactly INT32_MAX or a few bytes less. It should
still be way more than anyone would really need.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If gencnval encounters an empty input file the function resolveAlias
triggers an out of boundary write due to uniqueAliasArr pointing to
a 0 byte reserved memory address.
This patch protects the code in question with a check for
knownAliasesCount being not 0.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
- Adds first "span" field category
- Re-implements DateIntervalFormat#fallbackFormat to use FieldPositionHandler
- New temporary wiring in SimpleFormatter
- Adds additional logic to NumberStringBuilder.
- Extends logic of number::impl::Field type.
- Adds tests for RBNF support.
- Adds tests from ftang's original PR.
- CreateFileMappingW is marked for both desktop and UWP apps, so we can call that in both code paths.
- We can use the W version of the CreateFileMapping API instead of A version since we pass a NULL for the name anyways.
- We can call the same API CreateFile[A|W] from both the UWP and Win32 versions of the code, reducing one of the UWP forks.
- Add a work-around for older versions of the Windows 10 SDK UWP headers.
- Remove the code that was creating a custom security descriptor (but setting everything to NULL) and pass null to the API directly. This way we will get the default security descriptor instead of the NULL dacl.
- Change to use nullptr instead of NULL in C++ code.
