Fixed syntax error in the collation rules of Lower Sorbian language.
Collation test added in order to test the bugs like this early.
Reported-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
[BZ #23208]
* localedata/Makefile (test-input): Add dsb_DE.UTF-8.
(LOCALES): Likewise.
* localedata/dsb_DE.UTF-8.in: New file.
* localedata/locales/dsb_DE (LC_COLLATE): Fix syntax error.
In some places there was still the old Unicode version 10.0.0 in the files.
* localedata/charmaps/UTF-8: Use correct Unicode version 11.0.0 in comment.
* localedata/locales/i18n_ctype: Use correct Unicode version in comments
and headers.
* localedata/unicode-gen/utf8_gen.py: Add option to specify Unicode version
* localedata/unicode-gen/Makefile: Use option to specify Unicode version
for utf8_gen.py
There is a glibc optimization which allows for locale categories
to be removed during static compilation. There have been various
bugs for this support over the years, with bug 16915 being the
most recent. The solution there was to emit a reference to all the
categories to avoid any being removed. This fix, although it's in
the generic __nl_langinfo_l function, doesn't appear to be enough
to fix the case for a statically linked program that uses newlocale
and nl_langinfo_l. This commit doesn't fix the problem, but it does
add a XFAIL'd test case such that a fix can be applied against this
and the XFAIL removed. It's not entirely clear that the problem is
the same as that which was seen in bug 16915.
Unicode 11.0.0 Support: Character encoding, character type info, and
transliteration tables are all updated to Unicode 11.0.0, using
the generator scripts contributed by Mike FABIAN (Red Hat).
Some info about the number of characters added:
Total added characters in newly generated CHARMAP: 684
Total added characters in newly generated WIDTH: 119
alpha: Added 380 characters in new ctype which were not in old ctype
combining: Added 56 characters in new ctype which were not in old ctype
combining_level3: Added 37 characters in new ctype which were not in old ctype
graph: Added 684 characters in new ctype which were not in old ctype
lower: Added 82 characters in new ctype which were not in old ctype
print: Added 684 characters in new ctype which were not in old ctype
punct: Added 304 characters in new ctype which were not in old ctype
tolower: Added 79 characters in new ctype which were not in old ctype
totitle: Added 33 characters in new ctype which were not in old ctype
toupper: Added 79 characters in new ctype which were not in old ctype
upper: Added 79 characters in new ctype which were not in old ctype
No characters were removed.
[BZ #23308]
* unicode-gen/Makefile (UNICODE_VERSION): Set to 11.0.0.
* localedata/unicode-gen/DerivedCoreProperties.txt: Update to Unicode 11.0.0.
* localedata/unicode-gen/EastAsianWidth.txt: likewise.
* localedata/unicode-gen/PropList.txt: likewise.
* localedata/unicode-gen/UnicodeData.txt: likewise.
* localedata/charmaps/UTF-8: Regenerate.
* localedata/locales/i18n_ctype: likewise.
* localedata/locales/tr_TR: likewise.
* localedata/locales/translit_circle: likewise.
* localedata/locales/translit_cjk_compat: likewise.
* localedata/locales/translit_combining: likewise.
* localedata/locales/translit_compat: likewise.
* localedata/locales/translit_font: likewise.
* localedata/locales/translit_fraction: likewise.
This locale already contained correct data in mon array. Updated from
CLDR to start the month names with the lowercase letters.
alt_mon is a new import from CLDR. The change has been consulted
off-list with a native speaker.
[BZ #23140]
* localedata/locales/hy_AM (mon): Synchronize with CLDR (lowercase,
genitive case).
(alt_mon): New entry, import from CLDR (nominative case).
Kashubian language is not supported by CLDR, data copied from Wikipedia
and documents released by RJK (official Kashubian Language Council),
also consulted with a native speaker.
Note that this language also needs ab_alt_mon feature due to the month
May: nominative "môj", genitive "maja"; abbreviated nominative "môj",
abbreviated genitive "maj".
[BZ #23140]
* localedata/locales/csb_PL (mon): Rename to...
(alt_mon): This.
(abmon): Rename to...
(ab_alt_mon): This.
(mon): Add with proper genitive forms, copy from Wikipedia.
(abmon): Likewise.
Thank you Michal Ostrowski for the feedback.
[BZ #19485]
* localedata/locales/csb_PL (mon): Fix typos:
"łżëkwiôt" -> "łżëkwiat" (April); "lëpinc" -> "lëpińc" (July).
(yesstr): Add, value is "jo".
(nostr): Add, value is "nié".
As a followup of fixing bug 10871, these three languages now support two
grammatical cases of the month names.
This commit does not resolve the bug because there are more languages
to be committed.
[BZ #23140]
* localedata/locales/gd_GB (mon): Rename to...
(alt_mon): This.
(mon): Import from CLDR (genitive case).
* localedata/locales/hsb_DE (mon): Rename to...
(alt_mon): This.
(mon): Import from CLDR (genitive case).
* localedata/locales/wa_BE (mon): Rename to...
(alt_mon): This.
(mon): Add, fill with the proper genitive forms, but CLDR data
is incomplete; completed according to the comments in this file.
(d_t_fmt): Do not use "di" before the month name, no longer needed.
* localedata/locales/wa_BE (country_name): Reword
"Beljike" -> "Beldjike".
[BZ #23152]
* localedata/locales/gd_GB (abmon): Fix typo in May:
"Mhàrt" -> "Cèit". Adjust the comment according to the change.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
As spotted by GNOME translation team, Greek language has the actually
visible difference between the abbreviated nominative and the abbreviated
genitive case for some month names. Examples:
May:
abbreviated nominative: "Μάι" -> abbreviated genitive: "Μαΐ"
July:
abbreviated nominative: "Ιούν" -> abbreviated genitive: "Ιουλ"
and more month names with similar differences.
Original discussion: https://bugzilla.gnome.org/show_bug.cgi?id=793645#c21
[BZ #22937]
* localedata/locales/el_CY (abmon): Rename to...
(ab_alt_mon): This.
(abmon): Import from CLDR (abbreviated genitive case).
* localedata/locales/el_GR (abmon): Rename to...
(ab_alt_mon): This.
(abmon): Import from CLDR (abbreviated genitive case).
A GNOME translator asked to use the same abbreviated month names
as provided by CLDR. This sounds reasonable. See the discussion:
https://bugzilla.gnome.org/show_bug.cgi?id=793645#c27
[BZ #22932]
* localedata/locales/lt_LT (abmon): Synchronize with CLDR.
See this bug https://sourceware.org/bugzilla/show_bug.cgi?id=22898
These lines don’t yet work because of a glibc bug, not because of
problems in the locale data. No matter what sorting rules one uses,
these characters cannot be sorted at all at the moment.
As soon as that bug is fixed, these lines should be added back to the
test file.
* localedata/cmn_TW.UTF-8.in: Remove the lines which cannot
be sorted correctly at the moment because of a bug.
With out this, adding collation test files like localedata/gez_ER.UTF-8@abegede.in
does not work for locales which contain @ modifiers.
* gen-locales.mk: Make test files which contain @ modifiers in their
name work.
* localedata/gen-locale.sh: Likewise.
* localedata/da_DK.ISO-8859-1.in: In the new iso14651_t1_common file
downloaded from ISO, the collation order of @-. and space has changed.
Therefore, this test file needed to be adapted.
* localedata/fr_CA.UTF-8.in: Likewise.
* localedata/fr_FR.UTF-8.in: Likewise.
* localedata/uk_UA.UTF-8.in: Likewise.
* localedata/cs_CZ.UTF-8.in: adapt this test file to the collation
order of ȥ in the new iso14651_t1_common file.
* localedata/pl_PL.UTF-8.in: Likewise.
Entries for characters which have “IGNORE” on all 4 levels like:
<U0001> IGNORE;IGNORE;IGNORE;IGNORE % START OF HEADING (in ISO 6429)
are changed into:
<U0001> IGNORE;IGNORE;IGNORE;<U0001> % START OF HEADING (in ISO 6429)
i.e. putting the code point of the character into the fourth level
instead of “IGNORE”. Without that change, all such characters
would compare equal which would make a wcscoll test case fail.
It is better to have a clearly defined sort order even for characters
like this so it is good to use the code point as a tie-break.
* localedata/locales/iso14651_t1_common: Use the code point of a
character in the fourth collation level instead of IGNORE for all
entries which have IGNORE on all 4 levels.
* localedata/locales/iso14651_t1_common: Add some convenient collation
symbols like <AFTER-A>, <BEFORE-A> to make tailoring easier using
rules similar to those in CLDR.
* localedata/locales/iso14651_t1_common: The new version of this
file downloaded from ISO contained several syntax errors which
are fixed by this patch.
[BZ #14095] - Review / update collation data from Unicode / ISO 14651
File downloaded from:
http://standards.iso.org/iso-iec/14651/ed-4/ISO14651_2016_TABLE1_en.txt
Updating this file alone is not enough, there are problems in the new
file which need to be fixed and the collation rules for many locales
need to be adapted. This is done by the following patches.
This update also fixes the problem that many characters are treated as
identical when sorting because they were not yet in the old
iso14651_t1_common file, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1336308
- Infinite (∞) and empty set (∅) are treated as if they were the same character by sort and uniq
[BZ #14095]
* localedata/locales/iso14651_t1_common: Update file to
latest version from ISO (ISO14651_2016_TABLE1_en.txt).
LC_TIME in these 4 locales is identical, using “copy "es_BO"” makes
that more obvious.
[BZ #22646]
* localedata/locales/es_CL (LC_TIME): copy "es_BO".
* localedata/locales/es_CU (LC_TIME): copy "es_BO".
* localedata/locales/es_EC (LC_TIME): copy "es_BO".
[BZ #10871]
* localedata/locales/ru_RU (mon): Rename to...
(alt_mon): This.
(abmon): Rename to...
(ab_alt_mon): This.
(mon): Import from CLDR (genitive case).
(abmon): Copy from the old content except the 5th month which is
now in the genitive case, even when abbreviated.
* localedata/locales/ru_UA: Likewise.
* time/tst-strptime.c (day_tests): Add an actual example of
a difference between %b and %Ob in Russian.
Primary month names are in a genitive case now, alternative month names
are in a nominative case.
The alternative digits hack is no longer needed and has been removed.
[BZ #10871]
* localedata/locales/uk_UA (mon): Renamed to...
(alt_mon): This.
(alt_digits): "0" removed and then renamed to...
(mon): This.
(date_fmt): Definition changed not to use the alternative
digits hack.
[BZ #10871]
* localedata/locales/pl_PL: Alternative month names added,
primary month names are genitive now.
* time/tst-strptime.c (day_tests): Actually use a genitive case
of a month name in Polish language.
Some languages (Slavic, Baltic, etc.) require a genitive case of the
month name when formatting a full date (with the day number) while
they require a nominative case when referring to the month standalone.
This requirement cannot be fulfilled without providing two forms for
each month name. From now it is specified that nl_langinfo(MON_1)
series (up to MON_12) and strftime("%B") generate the month names in
the grammatical form used when the month is a part of a complete date.
If the grammatical form used when the month is named by itself is needed,
the new values nl_langinfo(ALTMON_1) (up to ALTMON_12) and
strftime("%OB") are supported. This new feature is optional so the
languages which do not need it or do not yet provide the updated
locales simply do not use it and their behaviour is unchanged.
[BZ #10871]
* locale/C-time.c (_nl_C_LC_TIME): Add alternative month names,
define them as the same as primary full month names explicitly.
* locale/categories.def (LC_TIME): Add alt_mon and wide-alt_mon.
* locale/langinfo.h (__ALTMON_1, __ALTMON_2, __ALTMON_3, __ALTMON_4,
__ALTMON_5, __ALTMON_6, __ALTMON_7, __ALTMON_8, __ALTMON_9, __ALTMON_10,
__ALTMON_11, __ALTMON_12, _NL_WALTMON_1, _NL_WALTMON_2, _NL_WALTMON_3,
_NL_WALTMON_4, _NL_WALTMON_5, _NL_WALTMON_6, _NL_WALTMON_7,
_NL_WALTMON_8, _NL_WALTMON_9, _NL_WALTMON_10, _NL_WALTMON_11,
_NL_WALTMON_12): New enum constants.
[__USE_GNU] (ALTMON_1, ALTMON_2, ALTMON_3, ALTMON_4, ALTMON_5, ALTMON_6,
ALTMON_7, ALTMON_8, ALTMON_9, ALTMON_10, ALTMON_11, ALTMON_12): New
macros.
* locale/programs/ld-time.c (struct locale_time_t): Add alt_mon,
walt_mon, and alt_mon_defined members.
(time_output): Output alt_mon and walt_mon members.
(time_read): Read them, initialize them as copies of mon and wmon
respectively if they are missing, initialize alt_mon_defined.
* locale/programs/locfile-kw.gperf (alt_mon): Define.
* locale/programs/locfile-kw.h: Regenerate.
* locale/programs/locfile-token.h (tok_alt_mon): New enum constant.
* localedata/tst-langinfo.c (map): Add tests for the new constants
ALTMON_1 .. ALTMON_12.
* time/Makefile [$(run-built-tests) = yes] (LOCALES): Add fr_FR.UTF-8
and pl_PL.UTF-8.
* time/strftime_l.c (f_altmonth): New macro.
(__strftime_internal): Handle %OB format.
* time/strptime_l.c [_LIBC] (alt_month_name): New macro.
(__strptime_internal): Handle %OB format.
* time/tst-strptime.c (day_tests): Add tests to parse different forms
of month names including the new %OB format specifier.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reported-by: Robert Pluim <rpluim@gmail.com>
* localedata/locales/gu_IN (LC_IDENTIFICATION): Fix an obvious typo
in date: "2004-14-09" should be "2004-09-14".
* localedata/locales/lo_LA: Fix an obvious typo in date in the header:
"2003-15-09" should be "2003-09-15".
* localedata/locales/bho_NP (LC_IDENTIFICATION): Fix an obvious typo
in date: "2017-24-07" should be "2017-07-24".
* localedata/locales/mai_IN: Likewise.
* localedata/locales/mai_NP: Likewise.
The current date format prefixes one-digit days with a space, resulting
in ugly two spaces:
$ LC_ALL=hu_HU.UTF-8 date
2018. jan. 1., hétfő, 21:25:35 CET
^^
The official orthography rules doesn't contain an explicit rule about
this (which already gives no sane reason for double space), and an
implicit example of "1848. március 9." under bullet point 296 at
http://helyesiras.mta.hu/helyesiras/default/akh12 contains a single
space only. It's sure not convincing on an HTML page, but I confirm
that the official book edition (e.g.
https://www.libri.hu/en/konyv/a-magyar-helyesiras-szabalyai-32.html)
also contains a single space there.
[BZ #22657]
* localedata/locales/hu_HU (d_t_fmt): Avoid a leading space
before the day number which may produce a double space.
(date_fmt): Likewise.
[BZ #22524]
* localedata/Makefile: Add lt_LT.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/lt_LT.UTF-8.in: New file for testing the collation.
* localedata/locales/lt_LT (LC_COLLATE): Use “copy "iso14651_t1"”
and build the collation rules upon that.
[BZ #22515]
* localedata/Makefile: Add hsb_DE.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/hsb_DE.UTF-8.in: New file for testing the collation.
* localedata/locales/hsb_DE (LC_COLLATE): Use “copy "iso14651_t1"”
and build the collation rules upon that.
[BZ #22517]
* localedata/Makefile: Add et_EE.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/et_EE.UTF-8.in: New file for testing the collation.
* localedata/locales/et_EE (LC_COLLATE): Use “copy "iso14651_t1"”
and build the collation rules upon that.
[BZ #22527]
* localedata/locales/tr_TR (LC_COLLATE): Base collation rules
on iso14651_t1. A test file localedata/tr_TR.UTF-8.in is already
available, this rewrite of the collation rules does reproduce
the test file in the same order.
[BZ #10580]
* localedata/locales/hr_HR (LC_TIME): Use two letters for the
digraphs in the month and day names. Using single code points for
digraphs is deprecated. While there are dedicated Unicode
codepoints, for the digraphs, these are included for backwards
compatibility and modern texts use a sequence of Basic Latin
characters. See: https://www.unicode.org/faq/ligature_digraph.html
This makes the month and day names agree exactly with CLDR now,
CLDR does not use the single code points for the digraphs either.
According to CLDR, collation rules for Serbian and Bosnian
should be the same as for Croatian.
[BZ #22534]
* localedata/Makefile: Add sr_RS.UTF-8 and bs_BA.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/bs_BA.UTF-8.in: New file (same as hr_HR.UTF-8.in).
* localedata/sr_RS.UTF-8.in: New file (same as hr_HR.UTF-8.in).
* localedata/locales/bs_BA (LC_COLLATE): Use “copy "hr_HR"”.
* localedata/locales/sr_RS (LC_COLLATE): Use “copy "hr_HR"”.
[BZ #10580]
* localedata/locales/hr_HR (LC_COLLATE): Base collation rules on
iso14651_t1.
* localedata/locales/hr_HR (LC_TIME): Sync month and day names with
CLDR (except use ligatures for the digraphs, CLDR does not use
the ligatures), add first_workday, some fixes in the date and time
formats.
* localedata/locales/hr_HR (LC_CTYPE): Add transliteration rules
for Đ and đ.
* localedata/locales/hr_HR (LC_MONETARY): Change currency_symbol to
lower case. p_cs_precedes and n_cs_precedes should be 0 instead of 1.
Add int_p_cs_precedes and int_n_cs_precedes.
* localedata/locales/hr_HR (LC_NUMERIC): Change thousands_sep to
"<U202F>" (NARROW NO-BREAK SPACE) and grouping to 3;3 (Agrees with
LC_MONETARY now).
* localedata/locales/hr_HR (LC_TELEPHONE): Add tel_dom_fmt.
* localedata/locales/hr_HR (LC_NAME): Add name_mr, name_mrs, and
name_miss.
* localedata/locales/hr_HR (LC_ADDRESS): Add country_post, country_isbn,
and lang_lib. Change postal_fmt.
change
[BZ #17750]
* Makefile: add fr_CA.UTF-8 to test-input and LOCALES.
* localedata/fr_CA.UTF-8.in: New file with test data for backward
accents sorting.
* localedata/fr_FR.UTF-8.in: Fix test data for forward accents
sorting.
* localedata/locales/cs_CZ (LC_COLLATE): Remove “define DIACRIT_FORWARD”
* localedata/locales/de_DE (LC_COLLATE): Likewise.
* localedata/locales/hu_HU (LC_COLLATE): Likewise.
* localedata/locales/lb_LU (LC_COLLATE): Likewise.
* localedata/locales/yuw_PG (LC_COLLATE): Likewise.
* localedata/locales/fr_CA (LC_COLLATE): Add “define DIACRIT_BACKWARD”
* localedata/locales/iso14651_t1_common: Use “ifdef DIACRIT_FORWARD”
instead of “ifdef DIACRIT_BACKWARD”.
The only locale which currently needs backward accents sorting is fr_CA.
Therefore, forward accents sorting should be the default.
Before this patch, backwards accent sorting was the default and all
locales except fr_CA had to use
define DIACRIT_FORWARD
before
copy "iso14651_t1"
Most locales didn’t do that and thus got the inappropriate backwards accents sorting
by accident. Now only the fr_CA locale needs to use
define DIACRIT_BACKWARD
before
copy "iso14651_t1"
Original patch slightly modified by: Mike FABIAN <mfabian@redhat.com>
The LOCALES variable in the localedata had two instances of cs_CZ
which generated the following warning:
../gen-locales.mk:11: target '/opt/build/localedata/cs_CZ.UTF-8/LC_CTYPE' given more than once in the same rule
Dropped the duplicate entry.
[BZ #22336]
* localedata/locales/cs_CZ (LC_COLLATE): Use “copy "iso14651_t1"”
and implement the collation rules for cs from CLDR on top of that.
* Makefile: Add cs_CZ.UTF-8 to test-input and to the list
of locales to be built for testing.
* cs_CZ.UTF-8.in: New file with test data to test the Czech sorting.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
[BZ #22469]
* localedata/locales/pl_PL (LC_COLLATE): Use “copy "iso14651_t1"”
and implement the collation rules for pl from CLDR on top of that.
* Makefile: Add pl_PL.UTF-8 to test-input and to the list
of locales to be built for testing.
* pl_PL.UTF-8.in: New file with test data to test the Polish sorting.
[BZ #15537]
* localedata/locales/lv_LV (LC_COLLATE): Fix collation by
using “copy "iso14651_t1"” and then implementing the
collation rules for lv from CLDR on top of that.
* Makefile: Add lv_LV.UTF-8 to test-input and to the list
of locales to be built for testing.
* lv_LV.UTF-8.in: New file with test data to test the Latvian
sorting.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Update all sourceware links to https. The website redirects
everything to https anyway so let the web server do a bit less work.
The only reference that remains unchanged is the one in the old
ChangeLog, since it didn't seem worth changing it.
* NEWS: Update sourceware link to https.
* configure.ac: Likewise.
* crypt/md5test-giant.c: Likewise.
* dlfcn/bug-atexit1.c: Likewise.
* dlfcn/bug-atexit2.c: Likewise.
* localedata/README: Likewise.
* malloc/tst-mallocfork.c: Likewise.
* manual/install.texi: Likewise.
* nptl/tst-pthread-getattr.c: Likewise.
* stdio-common/tst-fgets.c: Likewise.
* stdio-common/tst-fwrite.c: Likewise.
* sunrpc/Makefile: Likewise.
* sysdeps/arm/armv7/multiarch/memcpy_impl.S: Likewise.
* wcsmbs/tst-mbrtowc2.c: Likewise.
* configure: Regenerate.
* INSTALL: Regenerate.
Following the previous work by Carlos O'Donell the category of LC_CTYPE
is correctly set to "i18n:2012" rather than "unicode:2014" and the
i18n_ctype file is once again regenerated from scratch to make sure it
does not contain any manual additions except the copyright message.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* localedata/unicode-gen/gen_unicode_ctype.py (output_head):
category of LC_CTYPE set to "i18n:2012".
* localedata/locales/i18n_ctype: Regenerate.
[BZ #19485]
* localedata/locales/csb_PL (LC_TIME): Fix “abmon” for March
and use a better translation for March in “mon”.
* localedata/locales/csb_PL: Use more ASCII to improve the
readability of the source.
[BZ #13953]
* localedata/locales/km_KH: Use ASCII as much
as possible for better readability of the source and
remove useless comments.
* localedata/locales/km_KH (LC_TIME): Remove era stuff, it
was commented out and apparently wrong anyway because it was
using Lao characters. If Buddhist era should be used
for km_KH, a native speaker should write the correct formaat
for Khmer.
* localedata/locales/km_KH (LC_TIME): Add first_weekday 1
(According to CLDR, the first weekday for Cambodia is Sunday).
* localedata/locales/km_KH (LC_NAME): Remove name_mr and name_mrs
(These were using Lao characters which must be wrong. If we get
the correct data from a native speaker, we could add it back, until
then it is better not to have name_mr and name_mrs at all than
having it wrong).
[BZ #15260]
* localedata/locales/doi_IN (LC_MESSAGES): Match only for the
first letters of yesstr and nostr in yesexpr and noexpr,
not for the full words.
* localedata/locales/hne_IN (LC_MESSAGES): Likewise.
* localedata/locales/kok_IN (LC_MESSAGES): Likewise.
* localedata/locales/mr_IN (LC_MESSAGES): Likewise.
* localedata/locales/sat_IN (LC_MESSAGES): Likewise.
* localedata/locales/km_KH (LC_MESSAGES): Match also for the
first letters of yesstr and nostr in yesexpr and noexpr,
until now only English was matched in yesexpr and noexpr.
* localedata/locales/tl_PH (LC_MESSAGES): Use “copy "fil_PH"”
instead of “copy "en_US"”. CLDR has yesstr and nostr data for
fil but not for tl. As tl and fil are very similar, using fil
is probably better than using English.
Pablo was l10n/i18n coordinator back in the old days but MandrakeSoft is
dead now
* localedata/locales/br_FR (LC_IDENTIFICATON): Add
Thierry Vignaud <thierry.vignaud@gmail.com> as the contact
for the br_FR locale.
"Ket" is the the most used negative answer, as it's the negative answer
to a positively phrased question
It's used as it or with the verb ("Ne ran ket", ...)
As such, "Ket" is used in most translations.
"Nann" is less used as it's the negative answer to a negatively phrased
question
See https://en.wikipedia.org/wiki/Yes_and_no for explanations about
languages with 3 or 4 form systems.
We still keep "Nn" for short answers as:
- new learners are used to "Non" in french
- and they often misuses "Nann"
- for compatibility with english
[BZ #21706]
* localedata/locales/br_FR (LC_MESSAGES): Fix nostr.
From localedef --help:
Output control:
...
--no-warnings=<warnings> Comma-separated list of warnings to disable;
supported warnings are: ascii, intcurrsym
...
--warnings=<warnings> Comma-separated list of warnings to enable;
supported warnings are: ascii, intcurrsym
Locales using SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII
compatible. In order to build locales using these character maps, and
have localedef exit with a status of 0, we add new option to localedef
to disable or enable specific warnings. The options are --no-warnings
and --warnings, to disable and enable specific warnings respectively.
The options take a comma-separated list of warning names. The warning
names are taken directly from the generated warning. When a warning
that can be disabled is issued it will print something like this: foo is
not defined [--no-warnings=foo]
For the initial implementation we add two controllable warnings; first
'ascii' which is used by the localedata installation makefile target to
install SHIFT_JIS and SHIFT_JISX0213-using locales without error; second
'intcurrsym' which allows a program to use a non-standard international
currency symbol without triggering a warning. The 'intcurrsym' is
useful in the future if country codes are added that are not in our
current ISO 4217 list, and the user wants to avoid the warning. Having
at least two warnings to control gives an example for how the changes
can be extended to more warnings if required in the future.
These changes allow ja_JP.SHIFT_JIS and ja_JP.SHIFT_JISX0213 to be
compiled without warnings using --no-warnings=ascii. The
localedata/Makefile $(INSTALL-SUPPORTED-LOCALES) target is adjusted to
automatically add `--no-warnings=ascii` for such charmaps, and likewise
localedata/gen-locale.sh is adjusted with similar logic.
v2: Bring verbose, be_quiet, and all warning control booleans into
record-status.c, and compile this object file to be used by locale,
iconv, and localedef. Any users include record-status.h.
v3: Fix an instance of boolean coercion in set_warning().
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
The localedata collation test data is encoded in a particular
character set. We rename the test data to match the full locale
name with encoding, and adjust the Makefile and sort-test.sh
script. This allows us to have a future C.UTF-8 test that is
disambiguated from the built-in C locale.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
After the transition to generating a distinct file for Unicode ctype
information e.g. i18n_ctype, the check target was left with the wrong
target name. This patch fixes the check target and regenerates the
files with more information than previously used, filling in the the
LC_IDENTIFICATION data.
Tested on x86_64 by regenerating from Unicode source files, and
running checks. Tested by subsequently rebuilding all locales.
No regressions in testsuite.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
Reported-by: Rafal Luzynski <digitalfreak@lingonborough.com>
* localedata/locales/hi_IN (LC_MESSAGES): In yesexpr and noexpr,
also check for the first characters of yesstr and nostr.
* localedata/locales/kn_IN (LC_MESSAGES): Likewise.
* localedata/locales/ks_IN@devanagari (LC_MESSAGES): Likewise.
* localedata/locales/chr_US (LC_MESSAGES): In yesexpr and noexpr,
match also for the contents of yesstr and nostr. As the first letter
of yesstr and nostr is equal, checking only for the first letter
is not enough.
* localedata/locales/ug_CN (LC_MESSAGES): Fix noexpr and yesexpr
by including the first letters of nostr and yesexpr in the regexp.
Also make it more readable by using ASCII where possible.
* localedata/locales/te_IN (LC_MESSAGES): Fix noexpr by including
the first letter of nostr in the regexp. It agrees with CLDR now.
Also make it more readable by using ASCII where possible.
* localedata/locales/km_KH (LC_MESSAGES): Fix yestr and nostr.
The yesstr and nostr apparently came from CLDR. And CLDR has a bug there:
these strings contain a U+17D6 (which somewhat looks like a colon)
instead of a real colon to separate the full words for “yes”
and “no” from the single letter responses.
* localedata/locales/ka_GE (LC_MESSAGES): Fix yesexp to make
it agree with CLDR (include the first letter of yesstr).
Also make it more readable by using ASCII where possible.
* localedata/locales/mr_IN (LC_MESSAGES): Fix yesstr and nostr
and improve yesexpr and noexpr. The yesstr and nostr apparently
came from CLDR. And CLDR has a bug there: these strings contain
a U+0903 (which looks like a colon) instead of a real colon
to separate the full words for “yes” and “no” from the single
letter responses.
Using all characters of the full words for yes and no in yesexpr and noexpr
makes no sense here, especially not because the words for yes and no
share one character.
* localedata/locales/bn_BD (LC_MESSAGES): Use only the first
letters of the full yesstr and nostr in yesexpr and noexpr.
* localedata/locales/an_ES (LC_MESSAGES): Add yesstr and nostr.
* localedata/locales/an_ES (LC_ADDRESS): Add lang_term and lang_lib.
* localedata/locales/an_ES: Make source more readable by using ASCII
where possible.
* localedata/locales/tpi_PG (LC_MESSAGES): Fix yesexpr and noexpr
by adding the generic +1 and -0 as in all other locales.
* localedata/locales/tpi_PG (LC_TIME): Fix some typos in the month and
day names and make it more readable by using ASCII where possible.
[BZ #16777]
* localedata/locales/pl_PL (LC_MONETARY): Use U+202F as mon_thousands_sep
and improve readability by using more ASCII.
* localedata/locales/pl_PL (LC_NUMERIC): Use U+202F as thousands_sep
and improve readability by using more ASCII.
The Valencian (meridional Catalan) locale is basically a copy of the
Catalan locale. The point of having a separate locale is only for PO
translations. This locale is already provided by several distributions
and is already supported by various projects like LibreOffice, Mozilla,
Gnome, KDE.
Aurelien Jarno <aurelien@aurel32.net>
[BZ #2522]
* localedata/locales/ca_ES@valencia: New file.
* localedata/SUPPORTED: Add ca_ES@valencia/UTF-8.
CLDR uses this pattern as well.
[BZ #22019]
* localedata/locales/el_GR: Set n_cs_precedes to 0.
* localedata/locales/el_CY: copy "el_GR" because it is identical.
* stdlib/tst-strfmon_l.c: adapt test case.
The error and warning handling in localedef, locale, and iconv
is a bit of a mess.
We use ugly constructs like this:
WITH_CUR_LOCALE (error (1, errno, gettext ("\
cannot read character map directory `%s'"), directory));
to issue errors, and read error_message_count directly from the
error API to detect errors. The problem with that is that the
code also uses error to print warnings, and informative messages.
All of this leads to problems where just having warnings will
produce an exit status as-if errors had been seen.
To fix this situation I have adopted the following high-level
changes:
* All errors are counted distinctly.
* All warnings are counted distinctly.
* All informative messages are not counted.
* Increasing verbosity cannot generate *more* errors, and
it previously did for errors conditional on verbose,
this is now fixed.
* Increasing verbosity *can* generate *more* warnings.
* Making the output quiet cannot generate *fewer* errors,
and it previously did for errors conditional on be_quiet,
this is now fixed.
* Each of error, warning, and informative message has it's
own function to call defined in record-status.h, and they
are: record_error, record_warning, and record_verbose.
* The record_error function always records an error, but
conditional on be_quiet may not print it.
* The record_warning function always records a warning,
but conditional on be_quiet may not print it.
* The record_verbose function only prints the verbose
message if verbose is true and be_quiet is false.
This has allowed the following fix:
* Previously any warnings were being treated as errors
because they incremented error_message_count, but now
we properly return an exit status of 1 if there are
warnings but output was generated.
All of this allows localedef to correctly decide if errors,
or warnings were present, and produce the correct exit code.
The locale and iconv programs now also use record-status.h
and we have removed the WITH_CUR_LOCALE hack, and instead
have internal push_locale/pop_locale functions centralized
in the record routines.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
The commit does the following things:
* Move non-transliteration Unicode generated data to i18n_ctype.
* Copy the i18n_ctype data into i18n and add transliteration.
In the future, any locale which needs Unicode LC_CTYPE data can
also just use `copy i18n_ctype` and get the base character classes
and maps without transliteration.
Tested by compiling all the locales and my prototype C.UTF-8 which
uses it.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
This code page is identical to code page 850 except that X'D5'
has been changed from LI61 (dotless i) to SC20 (euro symbol).
The code points from /x01 to /x1f in the /localedata/charmaps/IBM858
file have the same mapping as those in localedata/charmaps/ANSI_X3.4-1968.
That means they disagree with with
ftp://ftp.software.ibm.com/software/globalization/gcoc/attachments/CP00858.txt
in that range.
For example, localedata/charmaps/IBM858 and localedata/charmaps/ANSI_X3.4-1968 have:
“<U0001> /x01 START OF HEADING (SOH)”
whereas CP00858.txt has:
“01 SS000000 Smiling Face”
That means that CP00858.txt is not really ASCII-compatible and to make
it ASCII-compatible we deviate fro CP00858.txt in the code points from /x01
to /x1f.
[BZ #21084]
* benchtests/strcoll-inputs/filelist#en_US.UTF-8: Add IBM858 and ibm858.c.
* iconvdata/Makefile: Add IBM858.
* iconvdata/gconv-modules: Add IBM858.
* iconvdata/ibm858.c: New file.
* iconvdata/tst-tables.sh: Add IBM858
* localedata/charmaps/IBM858: New file.
“Bengali” still remained in some comments in the bn_BD locale file,
in iso-639.def and in a test input file. Change it there as well.
“Bangla” is now used as the English name for this language in CLDR.
[BZ #14925]
* libio/tst-widetext.input: Change “Bengali” to “Bangla”.
* locale/iso-639.def: Change “Bengali” to “Bangla”.
* localedata/locales/bn_BD: “Bengali” was still used in some
comments. Change it to “Bangla”.
[BZ #22070]
* localedata/unicode-gen/utf8_gen.py: Set the width for
characters with Prepended_Concatenation_Mark property to 1
* localedata/charmaps/UTF-8: Updated using the improved script.
Writing ranges of neighbouring characters with the same with like this
<U000E0100>...<U000E01EF> 0
in charmaps/UTF-8 is more efficient than writing many single character lines
like:
<U000E0100> 0
<U000E0101> 0
...
[BZ #21750]
* unicode-gen/utf8_gen.py: Write all ranges of neighbouring characters
with the same width using the range notation in charmaps/UTF-8.
[BZ #15332]
* locales/es_CU (LC_MONETARY): use “,” for mon_decimal_point
and “.” for mon_thousands_sep (to agree with CLDR)
* locales/es_CU (LC_NUMERIC): Likewise.
[BZ #22038]
* locales/so_DJ (LC_TIME): Fix abday, abmon and
make t_fmt in the comment agree with the value of t_fmt.
* locales/so_ET (LC_TIME): Fix abday (From Axa to Axd)
* locales/so_KE (LC_TIME): Fix abday (From Axa to Axd)
* locales/so_SO (LC_TIME): Fix abday (From Axa to Axd)
[BZ #13805]
* locales/ru_RU (LC_MONETARY): Use “,” for mon_decimal_point
(to agree with CLDR).
* locales/ru_RU (LC_NUMERIC): Write mon_decimal_point in ASCII
for readability.
* locales/os_RU (LC_MONETARY): Copy from ru_RU,
makes it agree with CLDR.
Add locale for “Morisyen” which is also called “Mauritian Creole”
and is spoken in Mauritius.
[BZ #21971]
* localedata/SUPPORTED: Add mfe_MU/UTF-8.
* localedata/locales/mfe_MU: New File.
[BZ #21971]
* locale/iso-639.def: add Morisyen.
[BZ #21750]
* unicode-gen/utf8_gen.py (U+00AD): Set width to 1.
* unicode-gen/utf8_gen.py (U+1160..U+11FF): Set width to 0.
* unicode-gen/utf8_gen.py (U+3248..U+324F): Set width to 2.
* unicode-gen/utf8_gen.py (U+4DC0..U+4DFF): Likewise.
[BZ #19852]
[BZ #21750]
* unicode-gen/utf8_gen.py: Process EastAsianWidth lines before
UnicodeData lines so the latter have precedence; remove hack
to group output by EastAsianWidth ranges.
[BZ #14925]
* locales/bn_BD (LC_IDENTIFICATION): Change language name in
“title” and “language” from Bengali to Bangla.
* locales/bn_IN (LC_IDENTIFICATION): Likewise.
The custom stuff which was in LC_CTYPE of the km_KH locale seems
to be a very incomplete subset of what one gets by using
“copy "i18n"”. I cannot find anything special there which is not
in “copy "i18n"”, only lots of stuff which is missing.
[BZ #20008]
* locales/km_KH (LC_CTYPE): Use “copy "i18n"”.
[BZ #20482]
* locales/de_AT (LC_TIME): Use 2 letter abbreviations in abday.
* locales/de_BE (LC_TIME): Use 2 letter abbreviations in abday.
* locales/de_CH (LC_TIME): Use 2 letter abbreviations in abday.
* locales/de_DE (LC_TIME): Use readable ASCII in abday.
* locales/de_IT (LC_TIME): Use readable ASCII in abday.
* locales/de_LU (LC_TIME): Use 2 letter abbreviations in abday.
See also [BZ #20756].
U+202F NARROW NO-BREAK SPACE: a narrow form of a no-break space,
typically the width of a thin space or a mid space.
U+2009 THIN SPACE.
Many languages use small gap as thousands separator.
Thousands separator should not be a plain space, but a narrow space.
And additionally, it is not allowed to wrap line in the middle of the
number.
Locale data were created in a deep age of 8-bit encodings, so most of
them use space (incorrect: it allows wrapping the line in the middle
of the number), or NBSP (better, but typographically incorrect: space
between groups is too wide).
Now UNICODE is widely supported, so we should leave legacy characters
in favor of correct UNICODE character.
UNICODE has a dedicated character for this purpose:
NNBSP
U+202F NARROW NO-BREAK SPACE: a narrow form of a no-break space,
typically the width of a thin space or a mid space
The NNBSP exists since Unicode 3.0.
Use of NNBSP will prevent line wrapping in the midle of number and
improve readability of numbers.
[BZ #20756]
* locales/aa_DJ (LC_MONETARY): Replace space by NNBSP as thousands separator.
* locales/az_AZ (LC_MONETARY): Likewise.
* locales/be_BY (LC_MONETARY): Likewise.
* locales/be_BY@latin (LC_MONETARY): Likewise.
* locales/bg_BG (LC_MONETARY): Likewise.
* locales/bs_BA (LC_MONETARY): Likewise.
* locales/ce_RU (LC_MONETARY): Likewise.
* locales/crh_UA (LC_MONETARY): Likewise.
* locales/cs_CZ (LC_MONETARY): Likewise.
* locales/cs_CZ (LC_NUMERIC): Likewise.
* locales/cv_RU (LC_MONETARY): Likewise.
* locales/de_AT (LC_MONETARY): Likewise.
* locales/eo (LC_MONETARY): Likewise.
* locales/es_CR (LC_MONETARY): Likewise.
* locales/es_CR (LC_NUMERIC): Likewise.
* locales/es_CU (LC_MONETARY): Likewise.
* locales/et_EE (LC_MONETARY): Likewise.
* locales/et_EE (LC_NUMERIC): Likewise.
* locales/fi_FI (LC_MONETARY): Likewise.
* locales/fi_FI (LC_NUMERIC): Likewise.
* locales/fr_CA (LC_MONETARY): Likewise.
* locales/fr_FR (LC_MONETARY): Likewise.
* locales/fr_FR (LC_NUMERIC): Likewise.
* locales/fr_LU (LC_MONETARY): Likewise.
* locales/fr_LU (LC_NUMERIC): Likewise.
* locales/hr_HR (LC_MONETARY): Likewise.
* locales/ht_HT (LC_NUMERIC): Likewise.
* locales/kk_KZ (LC_MONETARY): Likewise.
* locales/kk_KZ (LC_NUMERIC): Likewise.
* locales/ky_KG (LC_MONETARY): Likewise.
* locales/ky_KG (LC_NUMERIC): Likewise.
* locales/lv_LV (LC_MONETARY): Likewise.
* locales/lv_LV (LC_NUMERIC): Likewise.
* locales/mg_MG (LC_MONETARY): Likewise.
* locales/mhr_RU (LC_MONETARY): Likewise.
* locales/mk_MK (LC_MONETARY): Likewise.
* locales/mk_MK (LC_NUMERIC): Likewise.
* locales/mn_MN (LC_MONETARY): Likewise.
* locales/nb_NO (LC_MONETARY): Likewise.
* locales/nb_NO (LC_NUMERIC): Likewise.
* locales/nl_AW (LC_MONETARY): Likewise.
* locales/nl_NL (LC_MONETARY): Likewise.
* locales/nn_NO (LC_MONETARY): Likewise.
* locales/os_RU (LC_MONETARY): Likewise.
* locales/pap_AW (LC_MONETARY): Likewise.
* locales/pap_CW (LC_MONETARY): Likewise.
* locales/ru_RU (LC_MONETARY): Likewise.
* locales/ru_RU (LC_NUMERIC): Likewise.
* locales/ru_UA (LC_MONETARY): Likewise.
* locales/sk_SK (LC_MONETARY): Likewise.
* locales/sk_SK (LC_NUMERIC): Likewise.
* locales/sl_SI (LC_MONETARY): Likewise.
* locales/sl_SI (LC_NUMERIC): Likewise.
* locales/sq_MK (LC_MONETARY): Likewise.
* locales/sv_SE (LC_MONETARY): Likewise.
* locales/sv_SE (LC_NUMERIC): Likewise.
* locales/tg_TJ (LC_MONETARY): Likewise.
* locales/tt_RU (LC_MONETARY): Likewise.
* locales/tt_RU@iqtelif (LC_MONETARY): Likewise.
* locales/uk_UA (LC_MONETARY): Likewise.
* locales/uk_UA (LC_NUMERIC): Likewise.
* locales/unm_US (LC_MONETARY): Likewise.
* locales/unm_US (LC_NUMERIC): Likewise.
* locales/wo_SN (LC_MONETARY): Likewise.
[BZ #17563]
[BZ #16905]
* locales/cmn_TW (LC_COLLATE): Use cns11643_stroke file for sorting.
* locales/cmn_TW (LC_TIME): Improve time and date formats.
* locales/cmn_TW (LC_MESSAGES): Add yesstr and nostr.
* locales/cns11643_stroke: New file, stroke count collation for
traditional Chinese.
