= `Default_Ignorable_Code_Point`s should have width 0 =
Unicode specifies (https://www.unicode.org/faq/unsup_char.html#3) that characters
with the `Default_Ignorable_Code_Point` property
> should be rendered as completely invisible (and non advancing, i.e. “zero width”),
if not explicitly supported in rendering.
Hence, `wcwidth()` should give them all a width of 0, with two exceptions:
- the soft hyphen (U+00AD SOFT HYPHEN) is assigned width 1 by longstanding precedent
- U+115F HANGUL CHOSEONG FILLER needs a carveout
due to the unique behavior of the conjoining Korean jamo characters.
One composed Hangul "syllable block" like 퓛
is made up of two to three individual component characters, or "jamo".
These are all assigned an `East_Asian_Width` of `Wide`
by Unicode, which would normally mean they would all be assigned
width 2 by glibc; a combination of (leading choseong jamo) +
(medial jungseong jamo) + (trailing jongseong jamo) would then have width 2 + 2 + 2 = 6.
However, glibc (and other wcwidth implementations) special-cases jungseong and jongseong,
assigning them all width 0,
to ensure that the complete block has width 2 + 0 + 0 = 2 as it should.
U+115F is meant for use in syllable blocks
that are intentionally missing a leading jamo;
it must be assigned a width of 2 even though it has no visible display
to ensure that the complete block has width 2.
However, `wcwidth()` currently (before this patch)
incorrectly assigns non-zero width to
U+3164 HANGUL FILLER and U+FFA0 HALFWIDTH HANGUL FILLER;
this commit fixes that.
Unicode spec references:
- Hangul: §3.12 https://www.unicode.org/versions/Unicode15.0.0/ch03.pdf#G24646 and
§18.6 https://www.unicode.org/versions/Unicode15.0.0/ch18.pdf#G31028
- `Default_Ignorable_Code_Point`: §5.21 https://www.unicode.org/versions/Unicode15.0.0/ch05.pdf#G40095.
= Non-`Default_Ignorable_Code_Point` format controls should be visible =
The Unicode Standard, §5.21 - Characters Ignored for Display
(https://www.unicode.org/versions/Unicode15.0.0/ch05.pdf#G40095)
says the following:
> A small number of format characters (General_Category = Cf )
> are also not given the Default_Ignorable_Code_Point property.
> This may surprise implementers, who often assume
> that all format characters are generally ignored in fallback display.
> The exact list of these exceptional format characters
> can be found in the Unicode Character Database.
> There are, however, three important sets of such format characters to note:
>
> - prepended concatenation marks
> - interlinear annotation characters
> - Egyptian hieroglyph format controls
>
> The prepended concatenation marks always have a visible display.
> See “Prepended Concatenation Marks” in [*Section 23.2, Layout Controls*](https://www.unicode.org/versions/Unicode15.1.0/ch23.pdf#M9.35858.HeadingBreak.132.Layout.Controls)
> for more discussion of the use and display of these signs.
>
> The other two notable sets of format characters that exceptionally are not ignored
> in fallback display consist of the interlinear annotation characters,
> U+FFF9 INTERLINEAR ANNOTATION ANCHOR through
> U+FFFB INTERLINEAR ANNOTATION TERMINATOR,
> and the Egyptian hieroglyph format controls,
> U+13430 EGYPTIAN HIEROGLYPH VERTICAL JOINER through
> U+1343F EGYPTIAN HIEROGLYPH END WALLED ENCLOSURE.
> These characters should have a visible glyph display for fallback rendering,
> because if they are not displayed,
> it is too easy to misread the resulting displayed text.
> See “Annotation Characters” in [*Section 23.8, Specials*](https://www.unicode.org/versions/Unicode15.1.0/ch23.pdf#M9.21335.Heading.133.Specials),
> as well as [*Section 11.4, Egyptian Hieroglyphs*](https://www.unicode.org/versions/Unicode15.1.0/ch11.pdf#M9.73291.Heading.1418.Egyptian.Hieroglyphs)
> for more discussion of the use and display of these characters.
glibc currently correctly assigns non-zero width to the prepended concatenation marks,
but it incorrectly gives zero width to the interlinear annotation characters
(which a generic terminal cannot interpret)
and the Egyptian hieroglyph format controls
(which are not widely supported in rendering implementations at present).
This commit fixes both these issues as well.
= Derive Hangul syllable type from Unicode data =
Previosuly, the jungseong and jongseong jamo ranges
were hard-coded into the script. With this commit, they are instead parsed
from the HangulSyllableType.txt data file published by Unicode.
This does not affect the end result.
Signed-off-by: Jules Bertholet <julesbertholet@quoi.xyz>
The gnulib version contains an important change (9ce573cde), which
fixes some problems with multithreading, entropy loss, and ASLR leak
nfo. It also fixes an issue where getrandom is not being used
on some new files generation (only for __GT_NOCREATE on first try).
The 044bf893ac removed __path_search, which is now moved to another
gnulib shared files (stdio-common/tmpdir.{c,h}). Tthis patch
also fixes direxists to use __stat64_time64 instead of __xstat64,
and move the include of pathmax.h for !_LIBC (since it is not used
by glibc). The license is also changed from GPL 3.0 to 2.1, with
permission from the authors (Bruno Haible and Paul Eggert).
The sync also removed the clock fallback, since clock_gettime
with CLOCK_REALTIME is expected to always succeed.
It syncs with gnulib commit 323834962817af7b115187e8c9a833437f8d20ec.
Checked on x86_64-linux-gnu.
Co-authored-by: Bruno Haible <bruno@clisp.org>
Co-authored-by: Paul Eggert <eggert@cs.ucla.edu>
Reviewed-by: Bruno Haible <bruno@clisp.org>
Sync tzselect, zdump, zic to TZDB 2024a.
This patch incorporates the following TZDB source code changes,
listed roughly in descending order of importance.
zic now supports links to links, needed for future tzdata
zic now defaults to '-b slim'
zic now updates output files atomically
zic has new options -R, -l -, -p -
zic -r now uses -00 for unspecified timestamps
zdump now uses [lo,hi) for both -c and -t
Fix several integer overflow bugs
zic now checks input bytes more carefully
Simplify and fix new TZDIR setup
Default time_t to 64 bits on glibc 2.34+ 32-bit
zic now generates TZ strings that conform to POSIX when all-year DST
zic -v now shows extreme-int tm_year transitions
Fix zic bug in last time type of Asia/Gaza etc.
Fix zic bug with Palestine after 2075
Fix bug uncovered by recent change to Iran history
Fix 'zic -b fat' bug with Port Moresby 32-bit data
Fix zic bug with -r @X where X is deduced from TZ
Fix bug with zic -r cutoff before 1st transition
Fix leap second expiry and truncation
Fix zic bug on Linux 2.6.16 and 2.6.17
Fix bug with 'zic -d /a/b/c' if /a is unwriteable
Don't mistruncate TZif files at leap seconds
Fix zdump undefined behavior if !USE_LTZ
zdump -v reports localtime+gmtime failures better
Fix zdump diagnostic for missing timezone
Don't assume nonempty argv
Port better to C23
Do not assume negative >> behavior
I18nize zdump a bit better
Port zdump to right_only installations
New tzselect menu option 'now'
tzselect can now use current time to help choose
Improve tzselect behavior for Turkey etc.
tzselect: do not create temporary files
tzselect: work around mawk bug with {2,}
tzselect: Port to POSIX awk, which prohibits -v newlines
Do not use empty RE in tzselect
Don't set TZ in tzselect
Avoid sed, expr in tzselect
tzselect: Fix problems with spaces in TZDIR
Improve tzselect diagnostics
Remove zic workaround for Qt bug 53071
Remove zic support for "min" in Rule lines
Remove zic support for zic -y, Rule TYPEs, pacificnew
Remove tzselect workaround for Bash 1.14.7 bug
* SHARED-FILES: Update to match current sync.
* config.h.in (HAVE_STRERROR): Remove; no longer needed.
* timezone/Makefile ($(objpfx)zic.o): Depend on tzdir.h.
($(objpfx)tzdir.h): New rule to build a placeholder.
* timezone/private.h, timezone/tzfile.h, timezone/version:
* timezone/zdump.c, timezone/zic.c: Copy verbatim from TZDB 2024a.
All the crypt related functions, cryptographic algorithms, and
make requirements are removed, with only the exception of md5
implementation which is moved to locale folder since it is
required by localedef for integrity protection (libc's
locale-reading code does not check these, but localedef does
generate them).
Besides thec code itself, both internal documentation and the
manual is also adjusted. This allows to remove both --enable-crypt
and --enable-nss-crypt configure options.
Checked with a build for all affected ABIs.
Co-authored-by: Zack Weinberg <zack@owlfolio.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Since the shared code now has special status with respect to
copyrights, port them into a more structured format in the source tree
and add a python function that parses and returns a dictionary with
the information.
I need this to exclude these files from the Contributed-by changes and
I reckon it would be useful to know these files for future tooling.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
