The type does not depend on wide vs narrow preprocessor macros,
so it does not need to be customized in stdio-common/printf-parse.h.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This code path is exercised indirectly by some of the DNS stub
resolver tests, via their own use of xopen_memstream for constructing
strings describing result data. The relative lack of test suite
coverage became apparent when these tests starting failing after a
printf changes uncovered bug 28949.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
form_character and form_string processing a sufficiently similar
that the logic can be shared.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This simplies formatting and helps with debugging. It also allows
the use of localized COMPILE_WPRINTF preprocessor conditionals.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The siglist.c is built with -fno-toplevel-reorder to avoid compiler
to reorder the compat assembly directives due an assembler
issue [1] (fixed on 2.39).
This patch removes the compiler flags by split the compat symbol
generation in two phases. First the __sys_siglist and __sys_sigabbrev
without any compat symbol directive is preprocessed to generate an
assembly source code. This generate assembly is then used as input
on a platform agnostic siglist.S which then creates the compat
definitions. This prevents compiler to move any compat directive
prior the _sys_errlist definition itself.
Checked on a make check run-built-tests=no on all affected ABIs.
Reviewed-by: Fangrui Song <maskray@google.com>
The errlist.c is built with -fno-toplevel-reorder to avoid compiler to
reorder the compat assembly directives due an assembler issue [1]
(fixed on 2.39).
This patch removes the compiler flags by split the compat symbol
generation in two phases. First the _sys_errlist_internal internal
without any compat symbol directive is preprocessed to generate an
assembly source code. This generate assembly is then used as input
on a platform agnostic errlist-data.S which then creates the compat
definitions. This prevents compiler to move any compat directive
prior the _sys_errlist_internal definition itself.
Checked on a make check run-built-tests=no on all affected ABIs.
[1] https://sourceware.org/bugzilla/show_bug.cgi?id=29012
The loader does not need to pull all __get_errlist definitions
and its size is decreased:
Before:
$ size elf/ld.so
text data bss dec hex filename
197774 11024 456 209254 33166 elf/ld.so
After:
$ size elf/ld.so
text data bss dec hex filename
191510 9936 456 201902 314ae elf/ld.so
Checked on x86_64-linux-gnu.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
%#m prints errno as an error constant if one is available, or
a decimal number as a fallback. This intends to address the gap
that strerrorname_np does not work well with printf for unknown
error codes due to its NULL return values in those cases.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
C2X adds a printf %b format (see
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2630.pdf>, accepted
for C2X), for outputting integers in binary. It also has recommended
practice for a corresponding %B format (like %b, but %#B starts the
output with 0B instead of 0b). Add support for these formats to
glibc.
One existing test uses %b as an example of an unknown format, to test
how glibc printf handles unknown formats; change that to %v. Use of
%b and %B as user-registered format specifiers continues to work (and
we already have a test that covers that, tst-printfsz.c).
Note that C2X also has scanf %b support, plus support for binary
constants starting 0b in strtol (base 0 and 2) and scanf %i (strtol
base 0 and scanf %i coming from a previous paper that added binary
integer literals). I intend to implement those features in a separate
patch or patches; as discussed in the thread starting at
<https://sourceware.org/pipermail/libc-alpha/2020-December/120414.html>,
they will be more complicated because they involve adding extra public
symbols to ensure compatibility with existing code that might not
expect 0b constants to be handled by strtol base 0 and 2 and scanf %i,
whereas simply adding a new format specifier poses no such
compatibility concerns.
Note that the actual conversion from integer to string uses existing
code in _itoa.c. That code has special cases for bases 8, 10 and 16,
probably so that the compiler can optimize division by an integer
constant in the code for those bases. If desired such special cases
could easily be added for base 2 as well, but that would be an
optimization, not actually needed for these printf formats to work.
Tested for x86_64 and x86. Also tested with build-many-glibcs.py for
aarch64-linux-gnu with GCC mainline to make sure that the test does
indeed build with GCC 12 (where format checking warnings are enabled
for most of the test).
A recent GCC mainline change introduces errors of the form:
vfprintf-internal.c: In function 'group_number':
vfprintf-internal.c:2093:15: error: 'memmove' specified bound between 9223372036854775808 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
2093 | memmove (w, s, (front_ptr -s) * sizeof (CHAR_T));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is a genuine bug in the glibc code: s > front_ptr is always true
at this point in the code, and the intent is clearly for the
subtraction to be the other way round. The other arguments to the
memmove call here also appear to be wrong; w and s point just *after*
the destination and source for copying the rest of the number, so the
size needs to be subtracted to get appropriate pointers for the
copying. Adjust the memmove call to conform to the apparent intent of
the code, so fixing the -Wstringop-overflow error.
Now, if the original code were ever executed, a buffer overrun would
result. However, I believe this code (introduced in commit
edc1686af0, "vfprintf: Reuse work_buffer
in group_number", so in glibc 2.26) is unreachable in prior glibc
releases (so there is no need for a bug in Bugzilla, no need to
consider any backports unless someone wants to build older glibc
releases with GCC 12 and no possibility of this buffer overrun
resulting in a security issue).
work_buffer is 1000 bytes / 250 wide characters. This case is only
reachable if an initial part of the number, plus a grouped copy of the
rest of the number, fail to fit in that space; that is, if the grouped
number fails to fit in the space. In the wide character case,
grouping is always one wide character, so even with a locale (of which
there aren't any in glibc) grouping every digit, a number would need
to occupy at least 125 wide characters to overflow, and a 64-bit
integer occupies at most 23 characters in octal including a leading 0.
In the narrow character case, the multibyte encoding of the grouping
separator would need to be at least 42 bytes to overflow, again
supposing grouping every digit, but MB_LEN_MAX is 16. So even if we
admit the case of artificially constructed locales not shipped with
glibc, given that such a locale would need to use one of the character
sets supported by glibc, this code cannot be reached at present. (And
POSIX only actually specifies the ' flag for grouping for decimal
output, though glibc acts on it for other bases as well.)
With binary output (if you consider use of grouping there to be
valid), you'd need a 15-byte multibyte character for overflow; I don't
know if any supported character set has such a character (if, again,
we admit constructed locales using grouping every digit and a grouping
separator chosen to have a multibyte encoding as long as possible, as
well as accepting use of grouping with binary), but given that we have
this code at all (clearly it's not *correct*, or in accordance with
the principle of avoiding arbitrary limits, to skip grouping on
running out of internal space like that), I don't think it should need
any further changes for binary printf support to go in.
On the other hand, support for large sizes of _BitInt in printf (see
the N2858 proposal) *would* require something to be done about such
arbitrary limits (presumably using dynamic allocation in printf again,
for sufficiently large _BitInt arguments only - currently only
floating-point uses dynamic allocation, and, as previously discussed,
that could actually be replaced by bounded allocation given smarter
code).
Tested with build-many-glibcs.py for aarch64-linux-gnu (GCC mainline).
Also tested natively for x86_64.
My glibc bot shows failures building the testsuite with GCC mainline
across all architectures:
tst-vfprintf-width-prec.c: In function 'do_test':
tst-vfprintf-width-prec.c:90:16: error: the comparison will always evaluate as 'false' for the address of 'result' will never be NULL [-Werror=address]
90 | if (result == NULL)
| ^~
tst-vfprintf-width-prec.c:89:13: note: 'result' declared here
89 | wchar_t result[100];
| ^~~~~~
This is clearly a correct warning; the comparison against NULL is
clearly a cut-and-paste mistake from an earlier case in the test that
does use calloc. Thus, remove the unnecessary check for NULL shown up
by the warning.
Similarly, two other tests have bogus comparisons against NULL; remove
those as well:
scanf14a.c:95:13: error: the comparison will always evaluate as 'false' for the address of 'fname' will never be NULL [-Werror=address]
95 | if (fname == NULL)
| ^~
scanf14a.c:93:8: note: 'fname' declared here
93 | char fname[strlen (tmpdir) + sizeof "/tst-scanf14.XXXXXX"];
| ^~~~~
scanf16a.c:125:13: error: the comparison will always evaluate as 'false' for the address of 'fname' will never be NULL [-Werror=address]
125 | if (fname == NULL)
| ^~
scanf16a.c:123:8: note: 'fname' declared here
123 | char fname[strlen (tmpdir) + sizeof "/tst-scanf16.XXXXXX"];
| ^~~~~
Tested with build-many-glibcs.py (GCC mainline) for aarch64-linux-gnu.
Instead of checking a pointer argument for NULL, use helper macros
defined differently in the non-positional and positional cases.
This avoids frequent conditional checks and a GCC 12 warning
about comparing pointers against NULL which cannot be NULL.
A lot of the code is unique to the positional and non-positional
code. Also unify the decimal and hexadecimal cases via the new
helper function __printf_fp_spec.
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
On the Hurd, the errno values don't start at 0, so _sys_errlist_internal
needs index remapping. The _sys_errlist_internal definition already properly
uses ERR_MAP, but __get_errlist and __get_errname were not.
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so. With this, the hooks now no
longer have any effect on the core library.
libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.
Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.
The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.
Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
The symbol has never been exported, so no compatibility symbol is
needed. Removing this file prevents ld from creation an exported
symbol in case GLIBC_2_0 expands to a symbol version which
does not have a local: *; directive in the symbol version map file.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The nptl version is used as default, since now with symbol always
present the single-thread optimization is tricky.
Hurd is not change, it is used it own lock scheme (which call
_cthreads_funlockfile).
Checked on x86_64-linux-gnu.
The nptl version is used as default, since now with symbol always
present the single-thread optimization is tricky.
Hurd is not change, it is used it own lock scheme (which call
_cthreads_ftrylockfile).
Checked on x86_64-linux-gnu.
The nptl version is used as default, since now with symbol always
present the single-thread optimization is tricky.
Hurd is not change, it is used it own lock scheme (which call
_cthreads_flockfile).
Checked on x86_64-linux-gnu.
This change continues the improvements to compile-time out of bounds
checking by decorating more APIs with either attribute access, or by
explicitly providing the array bound in APIs such as tmpnam() that
expect arrays of some minimum size as arguments. (The latter feature
is new in GCC 11.)
The only effects of the attribute and/or the array bound is to check
and diagnose calls to the functions that fail to provide a sufficient
number of elements, and the definitions of the functions that access
elements outside the specified bounds. (There is no interplay with
_FORTIFY_SOURCE here yet.)
Tested with GCC 7 through 11 on x86_64-linux.
Patterns like %*[ can safely be used to match a great many characters,
and it's quite realisitic to use them for more than INT_MAX characters
from an IO stream.
With the previous approach, after INT_MAX characters (v)fscanf would
return successfully, indicating an end to the match, even though there
wasn't one.
'this' can be understood as the current parameter, but in this case it
is meaning the other one, the one holding the width/precission.
'it' better describes that parameter, differentiating it from the
one corresponding to the current specifier.
Signed-off-by: Alejandro Colomar <alx.manpages@gmail.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The printf tests have no coverage for long double. Duplicate the
double tests so that we have some basic coverage.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
As noted in <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97264>, the cast
in the call to the read_int function is an aliasing violation. Change the
type of local variable f to a pointer to unsigned, which allows to
eliminate most casts while only adding three new ones.
It returns the string of the error constant, not its description (as
strerrordesc_np). To handle the Hurd error mapping, the ERR_MAP was
removed from errlist.h to errlist.c.
Also, the testcase test-strerr (added on 325081b9eb) was not added
on the check build neither it builds correctly. This patch also
changed it to decouple from errlist.h, the expected return values
are added explicitly for both both strerrorname_np and strerrordesc_np
directly.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also run a make
check for i686-gnu.
__printf_fp_l has a memory leak in the case of some I/O errors, where
both buffer and wbuffer have been malloced but the handling of I/O
errors only frees wbuffer. This patch fixes this by moving the
declaration of buffer to an outer scope and ensuring that it is freed
when wbuffer is freed.
Tested for x86_64 and x86.
__printf_fp_l has a double free bug in the case where it allocates
memory with malloc internally, then has an I/O error while outputting
trailing padding and tries to free that already-freed memory when the
error occurs. This patch fixes this by setting the relevant pointer
to NULL after the first free (the only free of this pointer that isn't
immediately followed by returning from the function).
Tested for x86_64 and x86.
The strerrorname_np returns error number name (e.g. "EINVAL" for EINVAL)
while strerrordesc_np returns string describing error number (e.g
"Invalid argument" for EINVAL). Different than strerror,
strerrordesc_np does not attempt to translate the return description,
both functions return NULL for an invalid error number.
They should be used instead of sys_errlist and sys_nerr, both are
thread and async-signal safe. These functions are GNU extensions.
Checked on x86-64-linux-gnu, i686-linux-gnu, powerpc64le-linux-gnu,
and s390x-linux-gnu.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The sigabbrev_np returns the abbreviated signal name (e.g. "HUP" for
SIGHUP) while sigdescr_np returns the string describing the error
number (e.g "Hangup" for SIGHUP). Different than strsignal,
sigdescr_np does not attempt to translate the return description and
both functions return NULL for an invalid signal number.
They should be used instead of sys_siglist or sys_sigabbrev and they
are both thread and async-signal safe. They are added as GNU
extensions on string.h header (same as strsignal).
Checked on x86-64-linux-gnu, i686-linux-gnu, powerpc64le-linux-gnu,
and s390x-linux-gnu.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The symbol is deprecated by strerror since its usage imposes some issues
such as copy relocations.
Its internal name is also changed to _sys_errlist_internal to avoid
static linking usage. The compat code is also refactored by removing
the over enginered errlist-compat.c generation from manual entried and
extra comment token in linker script file. It disantangle the code
generation from manual and simplify both Linux and Hurd compat code.
The definitions from errlist.c are moved to errlist.h and a new test
is added to avoid a new errno entry without an associated one in manual.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also run a check-abi
on all affected platforms.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The symbol was deprecated by strsignal and its usage imposes issues
such as copy relocations.
Its internal name is changed to __sys_siglist and __sys_sigabbrev to
avoid static linking usage. The compat code is also refactored, since
both Linux and Hurd usage the same strategy: export the same array with
different object sizes.
The libSegfault change avoids calling strsignal on the SIGFAULT signal
handler (the current usage is already sketchy, adding a call that
potentially issue locale internal function is even sketchier).
Checked on x86_64-linux-gnu and i686-linux-gnu. I also run a check-abi
on all affected platforms.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The vfprintf implementation (used for all printf-family functions)
contains complicated logic to allocate internal buffers of a size
depending on the width and precision used for a format, using either
malloc or alloca depending on that size, and with consequent checks
for size overflow and allocation failure.
As noted in bug 26211, the version of that logic used when '$' plus
argument number formats are in use is missing the overflow checks,
which can result in segfaults (quite possibly exploitable, I didn't
try to work that out) when the width or precision is in the range
0x7fffffe0 through 0x7fffffff (maybe smaller values as well in the
wprintf case on 32-bit systems, when the multiplication by sizeof
(CHAR_T) can overflow).
All that complicated logic in fact appears to be useless. As far as I
can tell, there has been no need (outside the floating-point printf
code, which does its own allocations) for allocations depending on
width or precision since commit
3e95f6602b ("Remove limitation on size
of precision for integers", Sun Sep 12 21:23:32 1999 +0000). Thus,
this patch removes that logic completely, thereby fixing both problems
with excessive allocations for large width and precision for
non-floating-point formats, and the problem with missing overflow
checks with such allocations. Note that this does have the
consequence that width and precision up to INT_MAX are now allowed
where previously INT_MAX / sizeof (CHAR_T) - EXTSIZ or more would have
been rejected, so could potentially expose any other overflows where
the value would previously have been rejected by those removed checks.
I believe this completely fixes bugs 14231 and 26211.
Excessive allocations are still possible in the floating-point case
(bug 21127), as are other integer or buffer overflows (see bug 26201).
This does not address the cases where a precision larger than INT_MAX
(embedded in the format string) would be meaningful without printf's
return value overflowing (when it's used with a string format, or %g
without the '#' flag, so the actual output will be much smaller), as
mentioned in bug 17829 comment 8; using size_t internally for
precision to handle that case would be complicated by struct
printf_info being a public ABI. Nor does it address the matter of an
INT_MIN width being negated (bug 17829 comment 7; the same logic
appears a second time in the file as well, in the form of multiplying
by -1). There may be other sources of memory allocations with malloc
in printf functions as well (bug 24988, bug 16060). From inspection,
I think there are also integer overflows in two copies of "if ((width
-= len) < 0)" logic (where width is int, len is size_t and a very long
string could result in spurious padding being output on a 32-bit
system before printf overflows the count of output characters).
Tested for x86-64 and x86.
Improve the commentary to aid future developers who will stumble
upon this novel, yet not always perfect, mechanism to support
alternative formats for long double.
Likewise, rename __LONG_DOUBLE_USES_FLOAT128 to
__LDOUBLE_REDIRECTS_TO_FLOAT128_ABI now that development work
has settled down. The command used was
git grep -l __LONG_DOUBLE_USES_FLOAT128 ':!./ChangeLog*' | \
xargs sed -i 's/__LONG_DOUBLE_USES_FLOAT128/__LDOUBLE_REDIRECTS_TO_FLOAT128_ABI/g'
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
The ldbl redirects for ieee128 have some jagged edges when
inspecting and manipulating symbols directly.
e.g asprintf is unconditionally redirected to __asprintfieee128
thus any tests relying on GCC's redirect behavior will encounter
problems if they inspect the symbol names too closely.
I've mitigated tests which expose the limitations of the
ldbl -> f128 redirects by giving them knowledge about the
redirected symbol names.
Hopefully there isn't much user code which depends on this
implementation specific behavior.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
