Fixed syntax error in the collation rules of Lower Sorbian language.
Collation test added in order to test the bugs like this early.
Reported-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
[BZ #23208]
* localedata/Makefile (test-input): Add dsb_DE.UTF-8.
(LOCALES): Likewise.
* localedata/dsb_DE.UTF-8.in: New file.
* localedata/locales/dsb_DE (LC_COLLATE): Fix syntax error.
In some places there was still the old Unicode version 10.0.0 in the files.
* localedata/charmaps/UTF-8: Use correct Unicode version 11.0.0 in comment.
* localedata/locales/i18n_ctype: Use correct Unicode version in comments
and headers.
* localedata/unicode-gen/utf8_gen.py: Add option to specify Unicode version
* localedata/unicode-gen/Makefile: Use option to specify Unicode version
for utf8_gen.py
Unicode 11.0.0 Support: Character encoding, character type info, and
transliteration tables are all updated to Unicode 11.0.0, using
the generator scripts contributed by Mike FABIAN (Red Hat).
Some info about the number of characters added:
Total added characters in newly generated CHARMAP: 684
Total added characters in newly generated WIDTH: 119
alpha: Added 380 characters in new ctype which were not in old ctype
combining: Added 56 characters in new ctype which were not in old ctype
combining_level3: Added 37 characters in new ctype which were not in old ctype
graph: Added 684 characters in new ctype which were not in old ctype
lower: Added 82 characters in new ctype which were not in old ctype
print: Added 684 characters in new ctype which were not in old ctype
punct: Added 304 characters in new ctype which were not in old ctype
tolower: Added 79 characters in new ctype which were not in old ctype
totitle: Added 33 characters in new ctype which were not in old ctype
toupper: Added 79 characters in new ctype which were not in old ctype
upper: Added 79 characters in new ctype which were not in old ctype
No characters were removed.
[BZ #23308]
* unicode-gen/Makefile (UNICODE_VERSION): Set to 11.0.0.
* localedata/unicode-gen/DerivedCoreProperties.txt: Update to Unicode 11.0.0.
* localedata/unicode-gen/EastAsianWidth.txt: likewise.
* localedata/unicode-gen/PropList.txt: likewise.
* localedata/unicode-gen/UnicodeData.txt: likewise.
* localedata/charmaps/UTF-8: Regenerate.
* localedata/locales/i18n_ctype: likewise.
* localedata/locales/tr_TR: likewise.
* localedata/locales/translit_circle: likewise.
* localedata/locales/translit_cjk_compat: likewise.
* localedata/locales/translit_combining: likewise.
* localedata/locales/translit_compat: likewise.
* localedata/locales/translit_font: likewise.
* localedata/locales/translit_fraction: likewise.
This locale already contained correct data in mon array. Updated from
CLDR to start the month names with the lowercase letters.
alt_mon is a new import from CLDR. The change has been consulted
off-list with a native speaker.
[BZ #23140]
* localedata/locales/hy_AM (mon): Synchronize with CLDR (lowercase,
genitive case).
(alt_mon): New entry, import from CLDR (nominative case).
Kashubian language is not supported by CLDR, data copied from Wikipedia
and documents released by RJK (official Kashubian Language Council),
also consulted with a native speaker.
Note that this language also needs ab_alt_mon feature due to the month
May: nominative "môj", genitive "maja"; abbreviated nominative "môj",
abbreviated genitive "maj".
[BZ #23140]
* localedata/locales/csb_PL (mon): Rename to...
(alt_mon): This.
(abmon): Rename to...
(ab_alt_mon): This.
(mon): Add with proper genitive forms, copy from Wikipedia.
(abmon): Likewise.
Thank you Michal Ostrowski for the feedback.
[BZ #19485]
* localedata/locales/csb_PL (mon): Fix typos:
"łżëkwiôt" -> "łżëkwiat" (April); "lëpinc" -> "lëpińc" (July).
(yesstr): Add, value is "jo".
(nostr): Add, value is "nié".
As a followup of fixing bug 10871, these three languages now support two
grammatical cases of the month names.
This commit does not resolve the bug because there are more languages
to be committed.
[BZ #23140]
* localedata/locales/gd_GB (mon): Rename to...
(alt_mon): This.
(mon): Import from CLDR (genitive case).
* localedata/locales/hsb_DE (mon): Rename to...
(alt_mon): This.
(mon): Import from CLDR (genitive case).
* localedata/locales/wa_BE (mon): Rename to...
(alt_mon): This.
(mon): Add, fill with the proper genitive forms, but CLDR data
is incomplete; completed according to the comments in this file.
(d_t_fmt): Do not use "di" before the month name, no longer needed.
* localedata/locales/wa_BE (country_name): Reword
"Beljike" -> "Beldjike".
[BZ #23152]
* localedata/locales/gd_GB (abmon): Fix typo in May:
"Mhàrt" -> "Cèit". Adjust the comment according to the change.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
As spotted by GNOME translation team, Greek language has the actually
visible difference between the abbreviated nominative and the abbreviated
genitive case for some month names. Examples:
May:
abbreviated nominative: "Μάι" -> abbreviated genitive: "Μαΐ"
July:
abbreviated nominative: "Ιούν" -> abbreviated genitive: "Ιουλ"
and more month names with similar differences.
Original discussion: https://bugzilla.gnome.org/show_bug.cgi?id=793645#c21
[BZ #22937]
* localedata/locales/el_CY (abmon): Rename to...
(ab_alt_mon): This.
(abmon): Import from CLDR (abbreviated genitive case).
* localedata/locales/el_GR (abmon): Rename to...
(ab_alt_mon): This.
(abmon): Import from CLDR (abbreviated genitive case).
A GNOME translator asked to use the same abbreviated month names
as provided by CLDR. This sounds reasonable. See the discussion:
https://bugzilla.gnome.org/show_bug.cgi?id=793645#c27
[BZ #22932]
* localedata/locales/lt_LT (abmon): Synchronize with CLDR.
Entries for characters which have “IGNORE” on all 4 levels like:
<U0001> IGNORE;IGNORE;IGNORE;IGNORE % START OF HEADING (in ISO 6429)
are changed into:
<U0001> IGNORE;IGNORE;IGNORE;<U0001> % START OF HEADING (in ISO 6429)
i.e. putting the code point of the character into the fourth level
instead of “IGNORE”. Without that change, all such characters
would compare equal which would make a wcscoll test case fail.
It is better to have a clearly defined sort order even for characters
like this so it is good to use the code point as a tie-break.
* localedata/locales/iso14651_t1_common: Use the code point of a
character in the fourth collation level instead of IGNORE for all
entries which have IGNORE on all 4 levels.
* localedata/locales/iso14651_t1_common: Add some convenient collation
symbols like <AFTER-A>, <BEFORE-A> to make tailoring easier using
rules similar to those in CLDR.
* localedata/locales/iso14651_t1_common: The new version of this
file downloaded from ISO contained several syntax errors which
are fixed by this patch.
[BZ #14095] - Review / update collation data from Unicode / ISO 14651
File downloaded from:
http://standards.iso.org/iso-iec/14651/ed-4/ISO14651_2016_TABLE1_en.txt
Updating this file alone is not enough, there are problems in the new
file which need to be fixed and the collation rules for many locales
need to be adapted. This is done by the following patches.
This update also fixes the problem that many characters are treated as
identical when sorting because they were not yet in the old
iso14651_t1_common file, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1336308
- Infinite (∞) and empty set (∅) are treated as if they were the same character by sort and uniq
[BZ #14095]
* localedata/locales/iso14651_t1_common: Update file to
latest version from ISO (ISO14651_2016_TABLE1_en.txt).
LC_TIME in these 4 locales is identical, using “copy "es_BO"” makes
that more obvious.
[BZ #22646]
* localedata/locales/es_CL (LC_TIME): copy "es_BO".
* localedata/locales/es_CU (LC_TIME): copy "es_BO".
* localedata/locales/es_EC (LC_TIME): copy "es_BO".
[BZ #10871]
* localedata/locales/ru_RU (mon): Rename to...
(alt_mon): This.
(abmon): Rename to...
(ab_alt_mon): This.
(mon): Import from CLDR (genitive case).
(abmon): Copy from the old content except the 5th month which is
now in the genitive case, even when abbreviated.
* localedata/locales/ru_UA: Likewise.
* time/tst-strptime.c (day_tests): Add an actual example of
a difference between %b and %Ob in Russian.
Primary month names are in a genitive case now, alternative month names
are in a nominative case.
The alternative digits hack is no longer needed and has been removed.
[BZ #10871]
* localedata/locales/uk_UA (mon): Renamed to...
(alt_mon): This.
(alt_digits): "0" removed and then renamed to...
(mon): This.
(date_fmt): Definition changed not to use the alternative
digits hack.
[BZ #10871]
* localedata/locales/pl_PL: Alternative month names added,
primary month names are genitive now.
* time/tst-strptime.c (day_tests): Actually use a genitive case
of a month name in Polish language.
Reported-by: Robert Pluim <rpluim@gmail.com>
* localedata/locales/gu_IN (LC_IDENTIFICATION): Fix an obvious typo
in date: "2004-14-09" should be "2004-09-14".
* localedata/locales/lo_LA: Fix an obvious typo in date in the header:
"2003-15-09" should be "2003-09-15".
* localedata/locales/bho_NP (LC_IDENTIFICATION): Fix an obvious typo
in date: "2017-24-07" should be "2017-07-24".
* localedata/locales/mai_IN: Likewise.
* localedata/locales/mai_NP: Likewise.
The current date format prefixes one-digit days with a space, resulting
in ugly two spaces:
$ LC_ALL=hu_HU.UTF-8 date
2018. jan. 1., hétfő, 21:25:35 CET
^^
The official orthography rules doesn't contain an explicit rule about
this (which already gives no sane reason for double space), and an
implicit example of "1848. március 9." under bullet point 296 at
http://helyesiras.mta.hu/helyesiras/default/akh12 contains a single
space only. It's sure not convincing on an HTML page, but I confirm
that the official book edition (e.g.
https://www.libri.hu/en/konyv/a-magyar-helyesiras-szabalyai-32.html)
also contains a single space there.
[BZ #22657]
* localedata/locales/hu_HU (d_t_fmt): Avoid a leading space
before the day number which may produce a double space.
(date_fmt): Likewise.
[BZ #22524]
* localedata/Makefile: Add lt_LT.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/lt_LT.UTF-8.in: New file for testing the collation.
* localedata/locales/lt_LT (LC_COLLATE): Use “copy "iso14651_t1"”
and build the collation rules upon that.
[BZ #22515]
* localedata/Makefile: Add hsb_DE.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/hsb_DE.UTF-8.in: New file for testing the collation.
* localedata/locales/hsb_DE (LC_COLLATE): Use “copy "iso14651_t1"”
and build the collation rules upon that.
[BZ #22517]
* localedata/Makefile: Add et_EE.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/et_EE.UTF-8.in: New file for testing the collation.
* localedata/locales/et_EE (LC_COLLATE): Use “copy "iso14651_t1"”
and build the collation rules upon that.
[BZ #22527]
* localedata/locales/tr_TR (LC_COLLATE): Base collation rules
on iso14651_t1. A test file localedata/tr_TR.UTF-8.in is already
available, this rewrite of the collation rules does reproduce
the test file in the same order.
[BZ #10580]
* localedata/locales/hr_HR (LC_TIME): Use two letters for the
digraphs in the month and day names. Using single code points for
digraphs is deprecated. While there are dedicated Unicode
codepoints, for the digraphs, these are included for backwards
compatibility and modern texts use a sequence of Basic Latin
characters. See: https://www.unicode.org/faq/ligature_digraph.html
This makes the month and day names agree exactly with CLDR now,
CLDR does not use the single code points for the digraphs either.
According to CLDR, collation rules for Serbian and Bosnian
should be the same as for Croatian.
[BZ #22534]
* localedata/Makefile: Add sr_RS.UTF-8 and bs_BA.UTF-8 to test-input
and to the list of locales to be built for testing.
* localedata/bs_BA.UTF-8.in: New file (same as hr_HR.UTF-8.in).
* localedata/sr_RS.UTF-8.in: New file (same as hr_HR.UTF-8.in).
* localedata/locales/bs_BA (LC_COLLATE): Use “copy "hr_HR"”.
* localedata/locales/sr_RS (LC_COLLATE): Use “copy "hr_HR"”.
[BZ #10580]
* localedata/locales/hr_HR (LC_COLLATE): Base collation rules on
iso14651_t1.
* localedata/locales/hr_HR (LC_TIME): Sync month and day names with
CLDR (except use ligatures for the digraphs, CLDR does not use
the ligatures), add first_workday, some fixes in the date and time
formats.
* localedata/locales/hr_HR (LC_CTYPE): Add transliteration rules
for Đ and đ.
* localedata/locales/hr_HR (LC_MONETARY): Change currency_symbol to
lower case. p_cs_precedes and n_cs_precedes should be 0 instead of 1.
Add int_p_cs_precedes and int_n_cs_precedes.
* localedata/locales/hr_HR (LC_NUMERIC): Change thousands_sep to
"<U202F>" (NARROW NO-BREAK SPACE) and grouping to 3;3 (Agrees with
LC_MONETARY now).
* localedata/locales/hr_HR (LC_TELEPHONE): Add tel_dom_fmt.
* localedata/locales/hr_HR (LC_NAME): Add name_mr, name_mrs, and
name_miss.
* localedata/locales/hr_HR (LC_ADDRESS): Add country_post, country_isbn,
and lang_lib. Change postal_fmt.
change
[BZ #17750]
* Makefile: add fr_CA.UTF-8 to test-input and LOCALES.
* localedata/fr_CA.UTF-8.in: New file with test data for backward
accents sorting.
* localedata/fr_FR.UTF-8.in: Fix test data for forward accents
sorting.
* localedata/locales/cs_CZ (LC_COLLATE): Remove “define DIACRIT_FORWARD”
* localedata/locales/de_DE (LC_COLLATE): Likewise.
* localedata/locales/hu_HU (LC_COLLATE): Likewise.
* localedata/locales/lb_LU (LC_COLLATE): Likewise.
* localedata/locales/yuw_PG (LC_COLLATE): Likewise.
* localedata/locales/fr_CA (LC_COLLATE): Add “define DIACRIT_BACKWARD”
* localedata/locales/iso14651_t1_common: Use “ifdef DIACRIT_FORWARD”
instead of “ifdef DIACRIT_BACKWARD”.
The only locale which currently needs backward accents sorting is fr_CA.
Therefore, forward accents sorting should be the default.
Before this patch, backwards accent sorting was the default and all
locales except fr_CA had to use
define DIACRIT_FORWARD
before
copy "iso14651_t1"
Most locales didn’t do that and thus got the inappropriate backwards accents sorting
by accident. Now only the fr_CA locale needs to use
define DIACRIT_BACKWARD
before
copy "iso14651_t1"
Original patch slightly modified by: Mike FABIAN <mfabian@redhat.com>
[BZ #22336]
* localedata/locales/cs_CZ (LC_COLLATE): Use “copy "iso14651_t1"”
and implement the collation rules for cs from CLDR on top of that.
* Makefile: Add cs_CZ.UTF-8 to test-input and to the list
of locales to be built for testing.
* cs_CZ.UTF-8.in: New file with test data to test the Czech sorting.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
