Commit Graph

1748 Commits

Author SHA1 Message Date
Florian Weimer
e7c14e542d support: Use macros for *stat wrappers
Macros will automatically use the correct types, without
having to fiddle with internal glibc macros.  It's also
impossible to get the types wrong due to aliasing because
support_check_stat_fd and support_check_stat_path do not
depend on the struct stat* types.

The changes reveal some inconsistencies in tests.

Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2024-08-16 16:05:20 +02:00
Maciej W. Rozycki
1b97a9f23b support: Add FAIL test failure helper
Add a FAIL test failure helper analogous to FAIL_RET, that does not
cause the current function to return, providing a standardized way to
report a test failure with a message supplied while permitting the
caller to continue executing, for further reporting, cleaning up, etc.

Update existing test cases that provide a conflicting definition of FAIL
by removing the local FAIL definition and then as follows:

- tst-fortify-syslog: provide a meaningful message in addition to the
  file name already added by <support/check.h>; 'support_record_failure'
  is already called by 'support_print_failure_impl' invoked by the new
  FAIL test failure helper.

- tst-ctype: no update to FAIL calls required, with the name of the file
  and the line number within of the failure site additionally included
  by the new FAIL test failure helper, and error counting plus count
  reporting upon test program termination also already provided by
  'support_record_failure' and 'support_report_failure' respectively,
  called by 'support_print_failure_impl' and 'adjust_exit_status' also
  respectively.  However in a number of places 'printf' is called and
  the error count adjusted by hand, so update these places to make use
  of FAIL instead.  And last but not least adjust the final summary just
  to report completion, with any error count following as reported by
  the test driver.

- test-tgmath2: no update to FAIL calls required, with the name of the
  file of the failure site additionally included by the new FAIL test
  failure helper.  Also there is no need to track the return status by
  hand as any call to FAIL will eventually cause the test case to return
  an unsuccesful exit status regardless of the return status from the
  test function, via a call to 'adjust_exit_status' made by the test
  driver.

Reviewed-by: DJ Delorie <dj@redhat.com>
2024-07-26 13:21:34 +01:00
Stefan Liebler
c0f21e073d tests-mbwc: Silence gcc 14 -Werror=format-overflow=
With gcc 14, I get this warning/werror when building the localedata tests:
tests-mbwc/tsp_common.c: In function ‘result.constprop.isra’:
tests-mbwc/tsp_common.c:55:43: error: ‘%s’ directive writing up to 92 bytes into a region of size between 0 and 114 [-Werror=format-overflow=]
   55 |   sprintf (result_rec, "%s:%s:%d:%d:%d:%c:%s\n", func, loc, rec_no, seq_no,
      |                                           ^~
In file included from ../include/bits/stdio2.h:1,
                 from ../libio/stdio.h:980,
                 from ../include/stdio.h:14,
                 from tests-mbwc/tsp_common.c:10:
In function ‘sprintf’,
    inlined from ‘result.constprop.isra’ at tests-mbwc/tsp_common.c:55:3:
../libio/bits/stdio2.h:30:10: note: ‘__builtin___sprintf_chk’ output between 20 and 234 bytes into a destination of size 132
   30 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   31 |                                   __glibc_objsize (__s), __fmt,
      |                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   32 |                                   __va_arg_pack ());
      |                                   ~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

This patch now gets rid of using sprintf and the result_rec buffer and just
prints to fp directly.
2024-06-27 16:49:26 +02:00
Mike FABIAN
3ea79f5085 Define ISO 639-3 "ltg" (Latgalian) and add ltg_LV locale
Resolves: BZ # 31411

References:
https://iso639-3.sil.org/code/ltg
https://en.wikipedia.org/wiki/Latgalian_language
https://github.com/unicode-org/cldr/blob/main/common/main/ltg.xml
2024-06-17 10:53:16 +02:00
Mike FABIAN
10733d6a72 localedata: Lowercase day and abday in cs_CZ
Resolves: BZ # 25119

Also to sync with CLDR
2024-06-11 10:33:54 +02:00
David Paleino
eb37015879 localedata: add new locales scn_IT
Signed-off-by: David Paleino <dapal@debian.org>
2024-06-07 15:45:18 +02:00
Mike FABIAN
28bf4783d9 localedata: cv_RU: update translation
Resolves: BZ # 21271
2024-05-23 14:39:35 +02:00
Jules Bertholet
25c9c3789e localedata: Fix several issues with the set of characters considered 0-width [BZ #31370]
= `Default_Ignorable_Code_Point`s should have width 0 =

Unicode specifies (https://www.unicode.org/faq/unsup_char.html#3) that characters
with the `Default_Ignorable_Code_Point` property

> should be rendered as completely invisible (and non advancing, i.e. “zero width”),
if not explicitly supported in rendering.

Hence, `wcwidth()` should give them all a width of 0, with two exceptions:

- the soft hyphen (U+00AD SOFT HYPHEN) is assigned width 1 by longstanding precedent
- U+115F HANGUL CHOSEONG FILLER needs a carveout
  due to the unique behavior of the conjoining Korean jamo characters.
  One composed Hangul "syllable block" like 퓛
  is made up of two to three individual component characters, or "jamo".
  These are all assigned an `East_Asian_Width` of `Wide`
  by Unicode, which would normally mean they would all be assigned
  width 2 by glibc; a combination of (leading choseong jamo) +
  (medial jungseong jamo) + (trailing jongseong jamo) would then have width 2 + 2 + 2 = 6.
  However, glibc (and other wcwidth implementations) special-cases jungseong and jongseong,
  assigning them all width 0,
  to ensure that the complete block has width 2 + 0 + 0 = 2 as it should.
  U+115F is meant for use in syllable blocks
  that are intentionally missing a leading jamo;
  it must be assigned a width of 2 even though it has no visible display
  to ensure that the complete block has width 2.

However, `wcwidth()` currently (before this patch)
incorrectly assigns non-zero width to
U+3164 HANGUL FILLER and U+FFA0 HALFWIDTH HANGUL FILLER;
this commit fixes that.

Unicode spec references:
- Hangul:  §3.12 https://www.unicode.org/versions/Unicode15.0.0/ch03.pdf#G24646 and
  §18.6 https://www.unicode.org/versions/Unicode15.0.0/ch18.pdf#G31028
- `Default_Ignorable_Code_Point`: §5.21 https://www.unicode.org/versions/Unicode15.0.0/ch05.pdf#G40095.

= Non-`Default_Ignorable_Code_Point` format controls should be visible =

The Unicode Standard, §5.21 - Characters Ignored for Display
(https://www.unicode.org/versions/Unicode15.0.0/ch05.pdf#G40095)
says the following:

> A small number of format characters (General_Category = Cf )
> are also not given the Default_Ignorable_Code_Point property.
> This may surprise implementers, who often assume
> that all format characters are generally ignored in fallback display.
> The exact list of these exceptional format characters
> can be found in the Unicode Character Database.
> There are, however, three important sets of such format characters to note:
>
> - prepended concatenation marks
> - interlinear annotation characters
> - Egyptian hieroglyph format controls
>
> The prepended concatenation marks always have a visible display.
> See “Prepended Concatenation Marks” in [*Section 23.2, Layout Controls*](https://www.unicode.org/versions/Unicode15.1.0/ch23.pdf#M9.35858.HeadingBreak.132.Layout.Controls)
> for more discussion of the use and display of these signs.
>
> The other two notable sets of format characters that exceptionally are not ignored
> in fallback display consist of the interlinear annotation characters,
> U+FFF9 INTERLINEAR ANNOTATION ANCHOR through
> U+FFFB INTERLINEAR ANNOTATION TERMINATOR,
> and the Egyptian hieroglyph format controls,
> U+13430 EGYPTIAN HIEROGLYPH VERTICAL JOINER through
> U+1343F EGYPTIAN HIEROGLYPH END WALLED ENCLOSURE.
> These characters should have a visible glyph display for fallback rendering,
> because if they are not displayed,
> it is too easy to misread the resulting displayed text.
> See “Annotation Characters” in [*Section 23.8, Specials*](https://www.unicode.org/versions/Unicode15.1.0/ch23.pdf#M9.21335.Heading.133.Specials),
> as well as [*Section 11.4, Egyptian Hieroglyphs*](https://www.unicode.org/versions/Unicode15.1.0/ch11.pdf#M9.73291.Heading.1418.Egyptian.Hieroglyphs)
> for more discussion of the use and display of these characters.

glibc currently correctly assigns non-zero width to the prepended concatenation marks,
but it incorrectly gives zero width to the interlinear annotation characters
(which a generic terminal cannot interpret)
and the Egyptian hieroglyph format controls
(which are not widely supported in rendering implementations at present).
This commit fixes both these issues as well.

= Derive Hangul syllable type from Unicode data =

Previosuly, the jungseong and jongseong jamo ranges
were hard-coded into the script. With this commit, they are instead parsed
from the HangulSyllableType.txt data file published by Unicode.
This does not affect the end result.

Signed-off-by: Jules Bertholet <julesbertholet@quoi.xyz>
2024-05-15 14:31:06 +02:00
Mike FABIAN
88dca8d5f8 localedata: fix weekdays in mdf_RU locale
From Кирилл Изместьев <izmestevks@basealt.ru>,
see: https://sourceware.org/bugzilla/show_bug.cgi?id=31530#c6
and the following comments.
2024-05-08 14:27:40 +02:00
Mike FABIAN
79fe4a0fa0 localedata: add mdf_RU locale
Resolves: BZ # 31530
2024-05-08 14:27:40 +02:00
Carlos O'Donell
684fbab755 locale: Handle loading a missing locale twice (Bug 14247)
Delay setting file->decided until the data has been successfully loaded
by _nl_load_locale().  If the function fails to load the data then we
must return and error and leave decided untouched to allow the caller to
attempt to load the data again at a later time.  We should not set
decided to 1 early in the function since doing so may prevent attempting
to load it again. We want to try loading it again because that allows an
open to fail and set errno correctly.

On the other side of this problem is that if we are called again with
the same inputs we will fetch the cached version of the object and carry
out no open syscalls and that fails to set errno so we must set errno to
ENOENT in that case.  There is a second code path that has to be handled
where the name of the locale matches but the codeset doesn't match.

These changes ensure that errno is correctly set on failure in all the
return paths in _nl_find_locale().

Adds tst-locale-loadlocale to cover the bug.

No regressions on x86_64.

Co-authored-by: Jeff Law <law@redhat.com>
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2024-04-22 16:03:00 -04:00
Carlos O'Donell
db50990ddb Update SHARED-FILES and license for Unicode 15.1.0.
In 2018 the license changed to use Unicode-3.0 license.
The Unicode License is a permissive MIT type of license.
Automation is updated to fetch the correct license file to
keep it in sync with the data files.

The new license is OSI approved and has an SPDX identifer:
https://opensource.org/license/unicode-license-v3
https://spdx.org/licenses/Unicode-3.0.html

The FSF and the GNU Project have been contacted to update
the license list for this license:
https://www.gnu.org/licenses/license-list.html
2024-02-27 08:27:14 -05:00
Mike FABIAN
07fd072caf localedata: ssy_ER: Fix syntax error 2024-02-08 08:13:37 +01:00
Dragan Stanojević (Nevidljivi)
559010e471 localedata: hr_HR: change currency to EUR/€
Resolves: BZ # 29845
2024-02-08 08:13:37 +01:00
Mike FABIAN
30a61b1dd9 Change lv_LV collation to agree with the recent change in CLDR
Resolves: https://sourceware.org/bugzilla/show_bug.cgi?id=23774

See this change in CLDR committed on 2024-01-29:
635e2d3d05
2024-02-08 08:13:37 +01:00
Mike FABIAN
5176a830e7 localedata: Use consistent values for grouping and mon_grouping
Resolves: BZ # 31205

Adapt test cases in test-grouping_iterator.c
2024-01-25 11:41:02 +01:00
Mike FABIAN
8393f4f72b localedata: renamed: aa_ER@saaho -> ssy_ER
Resolves: BZ # 19956
2024-01-18 11:44:38 +01:00
Mike FABIAN
8e474d5e40 localedata: add crh_RU, Crimean Tartar language in the Cyrillic script as used in Russia.
Resolves: BZ # 24386
2024-01-18 09:18:57 +01:00
Mike FABIAN
ce787f36e6 localedata: tr_TR, ku_TR: Sync with CLDR: “Turkey” -> “Türkiye”
Resolves: BZ # 31257
2024-01-18 08:30:34 +01:00
Mike FABIAN
70e26de105 localedata: miq_NI: Shorten month names in abmon
Resolves: BZ # 23172
2024-01-18 07:56:24 +01:00
Mike FABIAN
ce77e6919f localedata: add gbm_IN locale
Resolves: BZ # 19479
2024-01-17 17:50:33 +01:00
Mike FABIAN
9d2703c109 localedata: anp_IN: Fix abbreviated month names
Resolves: BZ # 31239

The correct abbreviated month names were apparently given in the comment above `abmon`.
But the value of `abmon` was apparently just copied from the value of `mon` and this
mistake was hard to see because code point notation <Uxxxx> was used. After converting
to UTF-8 it was obvious that there was apparently a copy and paste mistake.
2024-01-15 23:12:48 +01:00
Mike FABIAN
064c708c78 localedata/unicode-gen/utf8_gen.py: fix Hangul syllable name
Resolves: BZ # 29506
2024-01-14 11:42:28 +01:00
Mike FABIAN
fe6c8bab3a localedata: Remove redundant comments 2024-01-13 00:54:40 +01:00
Mike FABIAN
c0c259c3bd localedata: revert all the remaining locale sources to UTF-8 2024-01-11 15:04:25 +01:00
Mike FABIAN
e71c27b7ec localedata: am_ET ber_DZ en_GB en_PH en_US fil_PH kab_DZ om_ET om_KE ti_ET tl_PH: convert to UTF-8 2024-01-11 13:36:08 +01:00
Mike FABIAN
cb8e8b2e21 localedata: resolve cyclic dependencies
Resolves: BZ # 24006
2024-01-11 13:36:08 +01:00
Mike FABIAN
449aa2698c localedata: kv_RU: convert to UTF-8 2024-01-11 13:36:08 +01:00
Mike FABIAN
dff5023a87 localedata: add new locale kv_RU
Resolves: BZ # 30605
2024-01-11 13:36:08 +01:00
Carlos O'Donell
a09b2aacd9 localedata: Sort Makefile variables.
Sort Makefile variables using scrips/sort-makefile-lines.py.

No regressions on x86_64.
2024-01-10 14:08:26 -05:00
Mike FABIAN
46e713be57 localedata: su_ID: make lang_name agree with CLDR 2024-01-09 17:11:58 +01:00
Mike FABIAN
4cf0bd8431 localedata: add new locale su_ID
Resolves: BZ # 27312
2024-01-09 17:11:58 +01:00
Mike FABIAN
03f2265a37 localedata: add new locale zgh_MA
Resolves: BZ # 12908

https://iso639-3.sil.org/code/zgh
2024-01-09 17:11:58 +01:00
Mike FABIAN
4514f82929 localedata: add tok/UTF-8 to SUPPORTED 2024-01-09 12:08:14 +01:00
Mike FABIAN
ed97da8c7a localedata: tok: add yY and nN to yesexpr and noexpr
See: https://sourceware.org/bugzilla/show_bug.cgi?id=31221#c2
2024-01-09 12:08:14 +01:00
Mike FABIAN
2ddf2f8db1 localedata: tok: convert to UTF-8 2024-01-09 12:08:14 +01:00
Janet Blackquill
d3a2aecc1c localedata: add data for tok (Toki Pona)
Resolves: BZ # 31221

glibc can recognise its code, but does not have its data.
This patch remedies that.

Signed-off-by: Janet Blackquill <uhhadd@gmail.com>
2024-01-09 12:07:48 +01:00
Mike FABIAN
e171ad7d59 localedata: dz_BT, bo_CN: convert to UTF-8 2024-01-08 17:02:09 +01:00
Valery Ushakov
4c2b356be5 localedata: dz_BT, bo_CN: Fix spelling of "phur bu" in both Tibetan and Dzongkha
Resolves: BZ # 31086
2024-01-08 16:44:28 +01:00
Valery Ushakov
6b8419ba5f localedata: bo_CN: Fix spelling errors in Tibetan data
Resolves: BZ # 31086
2024-01-08 16:39:31 +01:00
Valery Ushakov
c4f648ed4d localedata: bo_CN: Fix incomplete edit in Tibetan yesexpr
Resolves: BZ # 31086
2024-01-08 16:08:07 +01:00
Valery Ushakov
460f26e51b localedata: dz_BT: Fix spelling errors in Dzongha data
Resolves: BZ # 31086
2024-01-08 16:04:59 +01:00
Mike FABIAN
d333a2e0fb localedata: unicode-gen: Remove redundant \s* from regexp, fix comments 2024-01-08 10:06:42 +01:00
Mike FABIAN
6f87f46bf4 localedata: convert the remaining *_RU locales to UTF-8 2024-01-08 10:06:42 +01:00
Mike FABIAN
e9f5dc7e4a localedata: ru_RU, ru_UA: convert to UTF-8 2024-01-04 16:32:44 +01:00
Mike FABIAN
d61a2bd782 localedata: es_??: convert to UTF-8 2024-01-04 16:03:08 +01:00
Mike FABIAN
734abeda98 localedata: miq_NI: convert to UTF-8 2024-01-04 16:03:08 +01:00
Mike FABIAN
b31a01909c localedata: fy_DE: make this "Western Frisian" to agree with the language code "fy"
Resolves: BZ # 14522
2024-01-03 20:55:44 +01:00
Mike FABIAN
3c173c1f63 localedata: fy_DE, fy_NL: convert to UTF-8 2024-01-03 20:07:21 +01:00
Mike FABIAN
bec492c1da localedata: ast_ES: convert to UTF-8 2024-01-03 17:44:52 +01:00