When the base is 0 or 2 and the first two characters are '0' and 'b',
but the rest are no binary digits. In this case this is no error,
and strtol must return 0 and ENDPTR points to the 'x' or 'b'.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
C2x adds binary integer constants starting with 0b or 0B, and supports
those constants in strtol-family functions when the base passed is 0
or 2. Implement that strtol support for glibc.
As discussed at
<https://sourceware.org/pipermail/libc-alpha/2020-December/120414.html>,
this is incompatible with previous C standard versions, in that such
an input string starting with 0b or 0B was previously required to be
parsed as 0 (with the rest of the string unprocessed). Thus, as
proposed there, this patch adds 20 new __isoc23_* functions with
appropriate header redirection support. This patch does *not* do
anything about scanf %i (which will need 12 new functions per long
double variant, so 12, 24 or 36 depending on the glibc configuration),
instead leaving that for a future patch. The function names would
remain as __isoc23_* even if C2x ends up published in 2024 rather than
2023.
Making this change leads to the question of what should happen to
internal uses of these functions in glibc and its tests. The header
redirection (which applies for _GNU_SOURCE or any other feature test
macros enabling C2x features) has the effect of redirecting internal
uses but without those uses then ending up at a hidden alias (see the
comment in include/stdio.h about interaction with libc_hidden_proto).
It seems desirable for the default for internal uses to be the same
versions used by normal code using _GNU_SOURCE, so rather than doing
anything to disable that redirection, similar macro definitions to
those in include/stdio.h are added to the include/ headers for the new
functions.
Given that the default for uses in glibc is for the redirections to
apply, the next question is whether the C2x semantics are correct for
all those uses. Uses with the base fixed to 10, 16 or any other value
other than 0 or 2 can be ignored. I think this leaves the following
internal uses to consider (an important consideration for review of
this patch will be both whether this list is complete and whether my
conclusions on all entries in it are correct):
benchtests/bench-malloc-simple.c
benchtests/bench-string.h
elf/sotruss-lib.c
math/libm-test-support.c
nptl/perf.c
nscd/nscd_conf.c
nss/nss_files/files-parse.c
posix/tst-fnmatch.c
posix/wordexp.c
resolv/inet_addr.c
rt/tst-mqueue7.c
soft-fp/testit.c
stdlib/fmtmsg.c
support/support_test_main.c
support/test-container.c
sysdeps/pthread/tst-mutex10.c
I think all of these places are OK with the new semantics, except for
resolv/inet_addr.c, where the POSIX semantics of inet_addr do not
allow for binary constants; thus, I changed that file (to use
__strtoul_internal, whose semantics are unchanged) and added a test
for this case. In the case of posix/wordexp.c I think accepting
binary constants is OK since POSIX explicitly allows additional forms
of shell arithmetic expressions, and in stdlib/fmtmsg.c SEV_LEVEL is
not in POSIX so again I think accepting binary constants is OK.
Functions such as __strtol_internal, which are only exported for
compatibility with old binaries from when those were used in inline
functions in headers, have unchanged semantics; the __*_l_internal
versions (purely internal to libc and not exported) have a new
argument to specify whether to accept binary constants.
As well as for the standard functions, the header redirection also
applies to the *_l versions (GNU extensions), and to legacy functions
such as strtoq, to avoid confusing inconsistency (the *q functions
redirect to __isoc23_*ll rather than needing their own __isoc23_*
entry points). For the functions that are only declared with
_GNU_SOURCE, this means the old versions are no longer available for
normal user programs at all. An internal __GLIBC_USE_C2X_STRTOL macro
is used to control the redirections in the headers, and cases in glibc
that wish to avoid the redirections - the function implementations
themselves and the tests of the old versions of the GNU functions -
then undefine and redefine that macro to allow the old versions to be
accessed. (There would of course be greater complexity should we wish
to make any of the old versions into compat symbols / avoid them being
defined at all for new glibc ABIs.)
strtol_l.c has some similarity to strtol.c in gnulib, but has already
diverged some way (and isn't listed at all at
https://sourceware.org/glibc/wiki/SharedSourceFiles unlike strtoll.c
and strtoul.c); I haven't made any attempts at gnulib compatibility in
the changes to that file.
I note incidentally that inttypes.h and wchar.h are missing the
__nonnull present on declarations of this family of functions in
stdlib.h; I didn't make any changes in that regard for the new
declarations added.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
<locale.h> is specified to define locale_t in POSIX.1-2008, and so are
all of the headers that define functions that take locale_t arguments.
Under _GNU_SOURCE, the additional headers that define such functions
have also always defined locale_t. Therefore, there is no need to use
__locale_t in public function prototypes, nor in any internal code.
* ctype/ctype-c99_l.c, ctype/ctype.h, ctype/ctype_l.c
* include/monetary.h, include/stdlib.h, include/time.h
* include/wchar.h, locale/duplocale.c, locale/freelocale.c
* locale/global-locale.c, locale/langinfo.h, locale/locale.h
* locale/localeinfo.h, locale/newlocale.c
* locale/nl_langinfo_l.c, locale/uselocale.c
* localedata/bug-usesetlocale.c, localedata/tst-xlocale2.c
* stdio-common/vfscanf.c, stdlib/monetary.h, stdlib/stdlib.h
* stdlib/strfmon_l.c, stdlib/strtod_l.c, stdlib/strtof_l.c
* stdlib/strtol.c, stdlib/strtol_l.c, stdlib/strtold_l.c
* stdlib/strtoll_l.c, stdlib/strtoul_l.c, stdlib/strtoull_l.c
* string/strcasecmp.c, string/strcoll_l.c, string/string.h
* string/strings.h, string/strncase.c, string/strxfrm_l.c
* sysdeps/ieee754/float128/strtof128_l.c
* sysdeps/ieee754/float128/wcstof128.c
* sysdeps/ieee754/float128/wcstof128_l.c
* sysdeps/ieee754/ldbl-128ibm/strtold_l.c
* sysdeps/ieee754/ldbl-64-128/strtold_l.c
* sysdeps/ieee754/ldbl-opt/nldbl-compat.c
* sysdeps/ieee754/ldbl-opt/nldbl-strfmon_l.c
* sysdeps/ieee754/ldbl-opt/nldbl-strtold_l.c
* sysdeps/ieee754/ldbl-opt/nldbl-wcstold_l.c
* sysdeps/powerpc/powerpc32/power7/strcasecmp.S
* sysdeps/powerpc/powerpc64/power7/strcasecmp.S
* sysdeps/x86_64/strcasecmp_l-nonascii.c
* sysdeps/x86_64/strncase_l-nonascii.c, time/strftime_l.c
* time/strptime_l.c, time/time.h, wcsmbs/mbsrtowcs_l.c
* wcsmbs/wchar.h, wcsmbs/wcscasecmp.c, wcsmbs/wcsncase.c
* wcsmbs/wcstod.c, wcsmbs/wcstod_l.c, wcsmbs/wcstof.c
* wcsmbs/wcstof_l.c, wcsmbs/wcstol_l.c, wcsmbs/wcstold.c
* wcsmbs/wcstold_l.c, wcsmbs/wcstoll_l.c, wcsmbs/wcstoul_l.c
* wcsmbs/wcstoull_l.c, wctype/iswctype_l.c
* wctype/towctrans_l.c, wctype/wcfuncs_l.c
* wctype/wctrans_l.c, wctype/wctype.h, wctype/wctype_l.c:
Change all uses of __locale_t to locale_t.
xlocale.h is already a single-type micro-header, defining struct
__locale_struct and the typedefs __locale_t and locale_t. This patch
brings it into the bits/types/ scheme: there are now
bits/types/__locale_t.h which defines only __locale_struct and
__locale_t, and bits/types/locale_t.h which defines locale_t as well
as the other two. None of *our* headers need __locale_t.h, but it
appears to me that libstdc++ could make use of it.
There are a lot of external uses of xlocale.h, but all the uses I
checked had an autoconf test or equivalent for its existence. It has
never been available from other C libraries, and it has always
contained a comment reading "This file is not standardized, don't rely
on it, it can go away without warning" so I think dropping it is
pretty safe.
I also took the opportunity to clean up comments in various public
header files that still talk about the *_l interfaces as though they
were completely nonstandard. There are a few of them, notably the
strtoX_l and wcstoX_l families, that haven't been standardized, but
the bulk are in POSIX.1-2008.
* locale/xlocale.h: Rename to...
* locale/bits/types/__locale_t.h: ...here. Adjust commentary.
Only define struct __locale_struct and __locale_t, not locale_t.
* locale/bits/types/locale_t.h: New file; define locale_t here.
* locale/Makefile (headers): Update to match.
* include/xlocale.h: Delete wrapper.
* include/bits/types/__locale_t.h: New wrapper.
* include/bits/types/locale_t.h: New wrapper.
* ctype/ctype.h, include/printf.h, include/time.h
* locale/langinfo.h, locale/locale.h, stdlib/monetary.h
* stdlib/stdlib.h, string/string.h, string/strings.h, time/time.h
* wcsmbs/wchar.h, wctype/wctype.h: Use bits/types/locale_t.h.
Correct outdated comments regarding the standardization status of
the functions that take locale_t arguments.
* stdlib/strtod_l.c, stdlib/strtof_l.c, stdlib/strtol_l.c
* stdlib/strtold_l.c, stdlib/strtoul_l.c, stdlib/strtoull_l.c
* sysdeps/ieee754/ldbl-128ibm/strtold_l.c
* sysdeps/ieee754/ldbl-64-128/strtold_l.c
* wcsmbs/wcstod.c, wcsmbs/wcstod_l.c, wcsmbs/wcstof.c
* wcsmbs/wcstof_l.c, wcsmbs/wcstold.c, wcsmbs/wcstold_l.c:
Don't include xlocale.h. If necessary, include locale.h instead.
* stdlib/strtold_l.c: Unconditionally include wchar.h.
The implementations of strtol and related functions use
locale-specific conversions to upper case before determining whether a
character is a valid letter in the argument. This means that in
Turkish locales such as tr_TR.UTF-8 and tr_TR.ISO-8859-9, "i" is
interpreted as not being a valid number, when if the base passed to
strtol is 19 or more it should be interpreted as the number 18.
ISO C explicitly says "The letters from a (or A) through z (or Z) are
ascribed the values 10 through 35", so clearly intends the standard
ASCII letters (otherwise you wouldn't generally have exactly 26
letters to ascribe such values) (whereas white-space must be
identified according to the locale). In particular, 'i' and 'I' must
be understood to be in that sequence.
This patch makes the code do the case conversions and classification
in the C locale; the user's locale remains used for whitespace testing
(explicitly correct according to ISO C). Note that the way the code
worked, the only non-ASCII letter that would previously have been
accepted would have been the Turkish 'ı' (dotless 'i'), because the
uppercase version of that in Turkish locales is 'I'. This patch means
that will no longer be accepted, which seems appropriate.
Tested for x86_64 and x86.
[BZ #19242]
* stdlib/strtol_l.c (ISALPHA): Use _nl_C_locobj_ptr for locale.
(TOUPPER): Likewise.
* stdlib/tst-strtol-locale-main.c: New file.
* stdlib/tst-strtol-locale.c: Likewise.
* stdlib/Makefile (tests): Add tst-strtol-locale.
[$(run-built-tests) = yes] (LOCALES): Add tr_TR.ISO-8859-9.
[$(run-built-tests) = yes] ($(objpfx)tst-strtol-locale.out):
Depend on $(gen-locales).
* wcsmbs/tst-wcstol-locale.c: New file.
* wcsmbs/Makefile (tests): Add tst-wcstol-locale.
[$(run-built-tests) = yes] (LOCALES): Add tr_TR.UTF-8 and
tr_TR.ISO-8859-9.
[$(run-built-tests) = yes] ($(objpfx)tst-wcstol-locale.out):
Depend on $(gen-locales).
__strtol_ull_max_tab, __strtol_ull_rem_tab): Declare.
(DEF): Don't put the var into .gnu.linkonce.r.* section.
Only provide var definitions in strtol_l (or for *ull*
in strtoll_l).
* stdio-common/bug16.c (tests): New array.
(do_tests): Allow the first hexadecimal digit
to be 1, 2, 4 or 8. Do 3 additional tests.
* sysdeps/s390/fpu/libm-test-ulps: Update.
* sysdeps/unix/sysv/linux/s390/s390-32/fchownat.c (fchownat): Use
fchownat syscall if available.
* sysdeps/unix/sysv/linux/powerpc/fchownat.c (fchownat): Likewise.
* sysdeps/unix/sysv/linux/sh/fchownat.c (fchownat): Likewise.
* sysdeps/unix/sysv/linux/i386/fchownat.c (fchownat): Likewise.
(rec_dirsearch) [case HIGHER_NAME]: Correctly size ndomain array.
1999-05-25 Ulrich Drepper <drepper@cygnus.com>
* stdlib/strtol.c: Moved to ...
* sysdeps/generic/strtol.c: ...here.
* stdlib/strtoul.c: Moved to ...
* sysdeps/generic/strtoul.c: ...here.
* stdlib/strtoll.c: Moved to ...
* sysdeps/generic/strtoll.c: ...here.
* stdlib/strtoull.c: Moved to ...
* sysdeps/generic/strtoull.c: ...here.
* stdlib/strtol_l.c: Moved to ...
* sysdeps/generic/strtol_l.c: ...here.
* stdlib/strtoul_l.c: Moved to ...
* sysdeps/generic/strtoul_l.c: ...here.
* stdlib/strtoll_l.c: Moved to ...
* sysdeps/generic/strtoll_l.c: ...here.
* stdlib/strtoull_l.c: Moved to ...
* sysdeps/generic/strtoull_l.c: ...here.
* sysdeps/wordexp-64/strtol.c: New file.
* sysdeps/wordexp-64/strtoul.c: New file.
* sysdeps/wordexp-64/strtoll.c: New file.
* sysdeps/wordexp-64/strtoull.c: New file.
* sysdeps/wordexp-64/strtol_l.c: New file.
* sysdeps/wordexp-64/strtoul_l.c: New file.
* sysdeps/wordexp-64/strtoll_l.c: New file.
* sysdeps/wordexp-64/strtoull_l.c: New file.
1997-05-24 03:51 Ulrich Drepper <drepper@cygnus.com>
* stdlib/Makefile (routines): Add strtol_l, strtoul_l, strtoll_l,
strtoull_l, strtof_l, strtod_l, and strtold_l.
* stdlib/stdlib.h: Add prototypes for new functions.
* stdlib/strtod.c: Change for compiling as strtoX_l.
* stdlib/strtol.c: Likewise.
* stdlib/strtof.c: Likewise.
* stdlib/strtold.c: Likewise.
* stdlib/strtod_l.c: New file.
* stdlib/strtof_l.c: New file.
* stdlib/strtold_l.c: New file.
* stdlib/strtol_l.c: New file.
* stdlib/strtoul_l.c: New file.
* stdlib/strtoll_l.c: New file.
* stdlib/strtoull_l.c: New file.
* string/Makefile (routines): Add strcasecmp_l and strncase_l.
* string/string.h: Add prototypes for new functions.
* sysdeps/generic/strcasecmp.c: Change for compiling as strcasecmp_l.
* sysdeps/generic/strncase.c: Change for compiling as strncasecmp_l.
* sysdeps/generic/strcasecmp_l.c: New file.
* sysdeps/generic/strncase_l.c: New file.
* wcsmbs/Makefile (routines): Add wcstol_l, wcstoul_l, wcstoll_l,
wcstoull_l, wcstod_l, wcstold_l, wcstof_l, wcscasecmp_l, and
wcsncase_l.
* wcsmbs/wchar.h: Add prototypes for new functions.
* wcsmbs/wcscasecmp.c: Change for compiling as wcscasecmp_l.
* wcsmbs/wcsncase.c: Change for compiling as wcsncasecmp_l.
* wcsmbs/wcscasecmp_l.c: New file.
* wcsmbs/wcsncase_l.c: New file.
* wcsmbs/wcstof.c: Change for compiling as wcstof_l.c
* wcsmbs/wcstold.c: Change for compiling as wcstold_l.c
* wcsmcs/wcstod_l.c: New file.
* wcsmcs/wcstof_l.c: New file.
* wcsmcs/wcstold_l.c: New file.
* wcsmcs/wcstol_l.c: New file.
* wcsmcs/wcstoul_l.c: New file.
* wcsmcs/wcstoll_l.c: New file.
* wcsmcs/wcstoull_l.c: New file.
* Makeconfig (binfmt-subdir): New variable. Set to `elf' if
$(elf) is defined. More to come later when other binary formats
are supported.
* Makefile (subdirs): Remove elf. Add $(binfmt-subdir).
Suggested by Philip Blundell.
* stdlib/Makefile (headers): Add fmtmsg.h.
(routines): Add fmtmsg.
* stdlib/fmtmsg.c: New file.
* stdlib/fmtmsg.h: New file.
* manual/stdio.texi: Add description of fmtmsg and addseverity.
* manual/examples/fmtmsgexpl.c: Example program for fmtmsg
documentation.
1997-05-23 15:26 Philip Blundell <pjb27@cam.ac.uk>
* resolv/res_query.c (res_querydomain): Avoid potential buffer
overrun. Reported by Dan A. Dickey <ddickey@transition.com>.
1997-05-22 18:36 Thomas Bushnell, n/BSG <thomas@gnu.ai.mit.edu>
* elf/dl-support.c (_dl_sysdep_open_zero_fill,
_dl_sysdep_read_whole_file): Moved functions to ...
* elf/dl-misc.c: This new file.
* sysdeps/generic/dl-sysdepio.c: Delete file and move functions...
* elf/dl-misc.c: ... here.
* sysdeps/generic/dl-sysdep.c (_dl_sysdep_open_zero_fill,
_dl_sysdep_read_whole_file): Delete functions; they now come from
elf/dl-misc.c (dl-support.c had contained identical versions).
* sysdeps/mach/hurd/dl-sysdepio.c: Delete file; move functions...
* sysdeps/mach/hurd/dl-sysdep.c: ... here, but mark them weak so
that the regular ones in dl-misc work once we've initialized.
* elf/Makefile (dl-routines): Remove dl-sysdepio.c. Add dl-misc.c.
1997-05-22 21:55 Philip Blundell <pjb27@cam.ac.uk>
* inet/Makefile (headers): Add netinet/inbits.h.
* inet/netinet/in.h: New file.
* sysdeps/generic/netinet/inbits.h: Likewise.
* sysdeps/unix/sysv/linux/netinet/inbits.h: Likewise.
* sysdeps/generic/netinet/ip6.h: Move to...
* inet/netinet/ip6.h: ... here.
* sysdeps/generic/netinet/icmp6.h: Move to...
* inet/netinet/icmp6.h: ... here.
* sysdeps/unix/sysv/linux/netinet/in.h: Remove.
* sysdeps/generic/netinet/in.h: Remove.
1997-05-22 05:40 Richard Henderson <rth@tamu.edu>
* sysdeps/alpha/dl-machine.h (elf_machine_runtime_setup): If we are
not looking at the new thread-safe .plt, don't be lazy about relocs.
(_dl_runtime_resolve): Fix up arithmetic for new .plt layout.
(elf_alpha_fix_plt): Insert wmb as appropriate to ensure safety.
* elf/dynamic-link.h (ELF_DYNAMIC_RELOCATE): Let
elf_machine_runtime_setup() decide if we can actually be lazy.
* elf/rtld.c (_dl_start): So don't call it.
* elf/dl-reloc.c (_dl_relocate_object): Likewise.
* sysdeps/i386/dl-machine.h (elf_machine_runtime_setup): Return lazy.
* sysdeps/m68k/dl-machine.h (elf_machine_runtime_setup): Likewise.
* sysdeps/mips/dl-machine.h (elf_machine_runtime_setup): Likewise.
* sysdeps/powerpc/dl-machine.h (elf_machine_runtime_setup): Likewise.
* sysdeps/sparc/dl-machine.h (elf_machine_runtime_setup): Likewise.
* sysdeps/stub/dl-machine.h (elf_machine_runtime_setup): Update
skeleton definition.
1997-05-22 18:45 Ulrich Drepper <drepper@cygnus.com>
* sysdeps/i386/fpu/__math.h (logb): Remove second value placed on
stack by fxtract.
1997-05-22 13:07 Andreas Jaeger <aj@arthur.rhein-neckar.de>
* sunrpc/rpcsvc/rusers.x: Provide and correct prototypes,
add cast to (xdrproc_t) where necessary to prevent warnings.
1997-05-22 12:18 Ulrich Drepper <drepper@cygnus.com>
* sysdeps/unix/sysv/linux/powerpc/dl-sysdep.c: Remove I/O functions.
* sunrpc/rpcinfo.c (get_inet_address): Use INADDR_NONE and INADDR_ANY
* sysdeps/libm-ieee754/s_cexp.c: Fix typo: string_alias ->
* nss/XXX-lookup.c: Add missing explanation.
