There have been multiple requests to provide more detail on how the
_FORTIFY_SOURCE macro works, so this patch adds a new node in the
Library Maintenance section that does this. A lot of the description is
implementation detail, which is why I put this in the appendix and not
in the main documentation.
Resolves: BZ #28998.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
I've updated copyright dates in glibc for 2023. This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files.
For some reason this causes a pre-commit check error for a copyright
date update commit, even though that commit doesn't touch anything
near the line with this whitespace.
<tgmath.h> implements semantics for integer generic arguments that
handle cases involving _FloatN / _FloatNx types as specified in TS
18661-3 plus some defect fixes.
C2x has further changes to the semantics for <tgmath.h> macros with
such types, which should also be considered defect fixes (although
handled through the integration of TS 18661-3 in C2x rather than
through an issue tracking process). Specifically, the rules were
changed because of problems raised with using the macros with the
evaluation format types such as float_t and _Float32_t: the older
version of the rules didn't allow passing _FloatN / _FloatNx types to
the narrowing macros returning float or double, or passing float /
double / long double to the narrowing macros returning _FloatN /
_FloatNx, which was a problem with the evaluation format types which
could be either kind of type depending on the value of
FLT_EVAL_METHOD.
Thus the new rules allow cases of mixing types which were not allowed
before, and, as part of the changes, the handling of integer arguments
was also changed: if there is any _FloatNx generic argument, integer
generic arguments are treated as _Float32x (not double), while the
rule about treating integer arguments to narrowing macros returning
_FloatN or _FloatNx as _Float64 not double was removed (no longer
needed now double is a valid argument to such macros).
I've implemented the changes in GCC's __builtin_tgmath, which thus
requires updates to glibc's test expectations so that the tests
continue to build with GCC 13 (the test is also updated to test the
argument types that weren't allowed before but are now valid under C2x
rules).
Given those test changes, it's then also necessary to fix the
implementations in <tgmath.h> to have appropriate semantics with older
GCC so that the tests pass with GCC versions before GCC 13 as well.
For some cases (non-narrowing macros with two or three generic
arguments; narrowing macros returning _Float32x), the older version of
__builtin_tgmath doesn't correspond sufficiently well to C2x
semantics, so in those cases <tgmath.h> is adjusted to use the older
macro implementation instead of __builtin_tgmath. The older macro
implementation is itself adjusted to give the desired semantics, with
GCC 7 and later. (It's not possible to get the right semantics in all
cases for the narrowing macros with GCC 6 and before when the _FloatN
/ _FloatNx names are typedefs rather than distinct types.)
Tested as follows: with the full glibc testsuite for x86_64, GCC 6, 7,
11, 13; with execution of the math/tests for aarch64, arm, powerpc and
powerpc64le, GCC 6, 7, 12 and 13 (powerpc64le only with GCC 12 and
13); with build-many-glibcs.py with GCC 6, 7, 12 and 13.
The daylight variable is supposed to be set to 1 if DST is ever in
use for the current time zone. But __tzfile_read used to do this:
__daylight = rule_stdoff != rule_dstoff;
This check can fail to set __daylight to 1 if the DST and non-DST
offsets happen to be the same.
Fix the following issues with built-in function use in
sysdeps/ieee754/ldbl-128 and sysdeps/ieee754/float128:
* fabsl used __builtin_fabsf128 unconditionally, breaking the build
with GCC 6 for several architectures; it should use __builtin_fabsl
with an appropriate redirection in float128_private.h. (I'm not
particularly concerned with building glibc with GCC 6; rather, I
want to be able to run the tgmath.h tests with GCC 6, which is a
significantly different case for tgmath.h compared to GCC 7 and
later because of the lack of _FloatN / _FloatNx support in the
compiler, and at present running the tests with a compiler means
building glibc with that compiler.)
* Some (conditional) uses of built-in functions had been added to
ldbl-128 without appropriate float128_private.h remapping (there was
remapping for the macros controlling whether the built-in functions
are used, just not for the functions themselves).
* s_llrintl.c called __builtin_round not __builtin_llrintl, which is
obviously wrong.
Tested with build-many-glibcs.py for aarch64-linux-gnu, GCC 6 (where
it fixes the glibc build) and GCC 12, and with the glibc testsuite for
x86_64.
The minimum non_temporal_threshold is 0x4040. non_temporal_threshold may
be set to less than the minimum value when the shared cache size isn't
available (e.g., in an emulator) or by the tunable. Add checks for
minimum and maximum of non_temporal_threshold.
This fixes BZ #29953.
If the value changes between sem_wait's read and the gsync_wait call,
the kernel will return KERN_INVALID_ARGUMENT, which we have to interpret
as the value having already changed.
This fixes applications (e.g. libgo) seeing sem_wait erroneously return
KERN_INVALID_ARGUMENT.
This patch is analogous to commit
a3708cf6b0.
atoi has undefined behavior on out-of-range input, which makes it
problematic to use anywhere in glibc that might be processing input
out-of-range for atoi but not specified to produce undefined behavior
for the function calling atoi. In conjunction with the C2x strtol
changes, use of atoi in libc can also result in localplt test failures
because the redirection for strtol does not interact properly with the
libc_hidden_proto call for __isoc23_strtol for the call in the inline
atoi implementation.
In malloc/arena.c, this issue shows up for atoi calls that are only
compiled for --disable-tunables (thus with the
x86_64-linux-gnu-minimal configuration of build-many-glibcs.py, for
example). Change those atoi calls to use strtol directly, as in the
previous such changes.
Tested for x86_64 (--disable-tunables).
The kernel actually verifies it, and a garbage value in the register
causes improper system call failures.
Fixes commit c1c0dea388 ("Linux: Remove epoll_create,
inotify_init from syscalls.list") and commit d1d23b1342
("Lninux: consolidate epoll_create implementation").
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
With modern ssh clients and daemons it is required to use AcceptEnv and
SendEnv configuration options to correctly support testing the DSO sort
ordering tests. This requirement is present because
scripts/dso-ordering-test.py injects GLIBC_TUNABLES to the left of the
${test_wrapper_env} and so it must both be sent by the ssh client and
accepted by the ssh daemon. This requirement is removed in this change
and the injected GLIBC_TUNABLES is placed after ${run_program_env} and
so still correctly provides the override that the test requires.
This is similar to existing tests like elf/tst-pathopt.sh,
elf/tst-rtld-load-self.sh, and locale/tst-locale-locpath.sh.
Tested that it fixes two failures when cross-testing on aarch64 with
scripts/cross-test-ssh.sh and an ssh client and daemon that do not pass
GLIBC_TUNABLES. Without this fix such a configuration will report the
following failures (since the GLIBC_TUNABLES not preserved):
FAIL: elf/tst-bz15311
FAIL: elf/tst-bz28937
Tested without regression on native x86_64 and aarch64 builds.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
This patch increases the value of SIGSTKSZ and MINSIGSTKSZ
for powerpc64 similar to the kernel commit
2f82ec19757f58549467db568c56e7dfff8af283 to allow
further expansion of the signal stack frame size.
This patch updates the kernel version in the tests tst-mman-consts.py,
tst-mount-consts.py and tst-pidfd-consts.py to 6.1. (There are no new
constants covered by these tests in 6.1 that need any other header
changes.)
Tested with build-many-glibcs.py.
Always null-terminate the buffer and set E2BIG if the buffer is too
small. This fixes bug 27857.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The internal buffer size is set to 2048 bytes. This is less than
the original BUFSIZ value used by buffered_vfprintf before
the conversion, but it hopefully covers all cases where write
boundaries matter.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The buffer resizing algorithm is slightly different. The initial
buffer is on the stack, and small buffers are directly allocated
on the heap using the exact required size. The overhead of the
additional copy is compensated by the lowered setup cost for buffers
compared to libio streams.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
After the rewrite and the implicit unbuffered streams handling, this
is very straightforward to add.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
vfprintf is entangled with vfwprintf (of course), __printf_fp,
__printf_fphex, __vstrfmon_l_internal, and the strfrom family of
functions. The latter use the internal snprintf functionality,
so vsnprintf is converted as well.
The simples conversion is __printf_fphex, followed by
__vstrfmon_l_internal and __printf_fp, and finally
__vfprintf_internal and __vfwprintf_internal. __vsnprintf_internal
and strfrom* are mostly consuming the new interfaces, so they
are comparatively simple.
__printf_fp is a public symbol, so the FILE *-based interface
had to preserved.
The __printf_fp rewrite does not change the actual binary-to-decimal
conversion algorithm, and digits are still not emitted directly to
the target buffer. However, the staging buffer now uses bytes
instead of wide characters, and one buffer copy is eliminated.
The changes are at least performance-neutral in my testing.
Floating point printing and snprintf improved measurably, so that
this Lua script
for i=1,5000000 do
print(i, i * math.pi)
end
runs about 5% faster for me. To preserve fprintf performance for
a simple "%d" format, this commit has some logic changes under
LABEL (unsigned_number) to avoid additional function calls. There
are certainly some very easy performance improvements here: binary,
octal and hexadecimal formatting can easily avoid the temporary work
buffer (the number of digits can be computed ahead-of-time using one
of the __builtin_clz* built-ins). Decimal formatting can use a
specialized version of _itoa_word for base 10.
The existing (inconsistent) width handling between strfmon and printf
is preserved here. __print_fp_buffer_1 would have to use
__translated_number_width to achieve ISO conformance for printf.
Test expectations in libio/tst-vtables-common.c are adjusted because
the internal staging buffer merges all virtual function calls into
one.
In general, stack buffer usage is greatly reduced, particularly for
unbuffered input streams. __printf_fp can still use a large buffer
in binary128 mode for %g, though.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This function will be used to compute the width of a number
after i18n digit translation.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
And __wprintf_function_invoke. These functions will be used to
to call registered printf specifier callbacks on printf buffers
after vfprintf and vfwprintf have been converted to buffers. The new
implementation avoids alloca/variable length arrays.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
These buffers will eventually be used instead of FILE * objects
to implement printf functions. The multibyte buffer is struct
__printf_buffer, the wide buffer is struct __wprintf_buffer.
To enable writing type-generic code, the header files
printf_buffer-char.h and printf_buffer-wchar_t.h define the
Xprintf macro differently, enabling Xprintf (buffer) to stand
for __printf_buffer and __wprintf_buffer as appropriate. For
common cases, macros like Xprintf_buffer are provided as a more
syntactically convenient shortcut.
Buffer-specific flush callbacks are implemented with a switch
statement instead of a function pointer, to avoid hardening issues
similar to those of libio vtables. struct __printf_buffer_as_file
is needed to support custom printf specifiers because the public
interface for that requires passing a FILE *, which is why there
is a trapdoor back from these buffers to FILE * streams.
Since the immediate user of these interfaces knows when processing
has finished, there is no flush callback for the end of processing,
only a flush callback for the intermediate buffer flush.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The iterator allows grouping while scanning forward through
the digits. This enables emitting digits as they are processed.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch is split out of
<https://sourceware.org/pipermail/libc-alpha/2022-December/144122.html>.
atoi has undefined behavior on out-of-range input, which makes it
problematic to use anywhere in glibc that might be processing input
out-of-range for atoi but not specified to produce undefined behavior
for the function calling atoi. Change some uses of atoi to call
strtol instead; this avoids the undefined behavior, though there is no
guarantee that the overflow handling of strtol is really right in
those places either. This also serves to avoid localplt test failures
given an installed header redirection for strtol (which means that the
call from the inline atoi implementation doesn't end up at a hidden
alias from libc_hidden_proto).
Certainly, the use of atoi is questionable in argp-help.c (shared with
gnulib, so shouldn't depend on glibc implementation details, and
processing user-provided input), and maybe also in argp-parse.c (I'm
not sure what that code in argp-parse.c is meant to be used for). I
also changed inet/rexec.c and resolv/res_init.c similarly to use
strtol to avoid such localplt failures, although given those files (in
those versions) are only used in glibc it's not problematic for them
to rely on the specific behavior of glibc's atoi on out-of-range input
(in the absence of compiler optimizations based on the undefined
behavior) in the same way it's problematic for gnulib code to do so.
There may be other uses of atoi (or atol or atoll), in any of glibc's
installed code, for which it would also be appropriate to avoid the
undefined behavior on out-of-range input; this patch only fixes the
specific cases needed to avoid localplt failures.
Tested for x86_64.
Their presence causes stub warnings to be created on architectures
which do not implement them.
Fixes commit d1d23b1342 ("Lninux: consolidate
epoll_create implementation") and commit 842128f160
("Linux: consolidate inotify_init implementation").
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
8b8c768e3c ("Force use of -ffreestanding when checking for gnumach
headers") was passing -ffreestanding to CFLAGS only, but headers checks are
performed with the preprocessor, so we rather need to pass it to CPPFLAGS.
Without this ./configure assumes that we are in a fully hosted
environment, which might not be the case. After this patch, we can rely on
the freestanding header files provided by GCC such as stdint.h.
Message-Id: <Y5+0V9osFc/zXMq0@mars>
Current scheme only consideres the first argument for both --required
and --optional, where the idea is to append a new item.
Checked on x86_64-linux-gnu.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
In the case of INCORRECT usage of `memcmp(a, b, N)` where `a` and `b`
are concurrently modified as `memcmp` runs, there can be a SIGSEGV
in `L(ret_nonzero_vec_end_0)` because the sequential logic
assumes that `(rdx - 32 + rax)` is a positive 32-bit integer.
To be clear, this change does not mean the usage of `memcmp` is
supported. The program behaviour is undefined (UB) in the
presence of data races, and `memcmp` is incorrect when the values
of `a` and/or `b` are modified concurrently (data race). This UB
may manifest itself as a SIGSEGV. That being said, if we can
allow the idiomatic use cases, like those in yottadb with
opportunistic concurrency control (OCC), to execute without a
SIGSEGV, at no cost to regular use cases, then we can aim to
minimize harm to those existing users.
The fix replaces a 32-bit `addl %edx, %eax` with the 64-bit variant
`addq %rdx, %rax`. The 1-extra byte of code size from using the
64-bit instruction doesn't contribute to overall code size as the
next target is aligned and has multiple bytes of `nop` padding
before it. As well all the logic between the add and `ret` still
fits in the same fetch block, so the cost of this change is
basically zero.
The relevant sequential logic can be seen in the following
pseudo-code:
```
/*
* rsi = a
* rdi = b
* rdx = len - 32
*/
/* cmp a[0:15] and b[0:15]. Since length is known to be [17, 32]
in this case, this check is also assumed to cover a[0:(31 - len)]
and b[0:(31 - len)]. */
movups (%rsi), %xmm0
movups (%rdi), %xmm1
PCMPEQ %xmm0, %xmm1
pmovmskb %xmm1, %eax
subl %ecx, %eax
jnz L(END_NEQ)
/* cmp a[len-16:len-1] and b[len-16:len-1]. */
movups 16(%rsi, %rdx), %xmm0
movups 16(%rdi, %rdx), %xmm1
PCMPEQ %xmm0, %xmm1
pmovmskb %xmm1, %eax
subl %ecx, %eax
jnz L(END_NEQ2)
ret
L(END2):
/* Position first mismatch. */
bsfl %eax, %eax
/* The sequential version is able to assume this value is a
positive 32-bit value because the first check included bytes in
range a[0:(31 - len)] and b[0:(31 - len)] so `eax` must be
greater than `31 - len` so the minimum value of `edx` + `eax` is
`(len - 32) + (32 - len) >= 0`. In the concurrent case, however,
`a` or `b` could have been changed so a mismatch in `eax` less or
equal than `(31 - len)` is possible (the new low bound is `(16 -
len)`. This can result in a negative 32-bit signed integer, which
when zero extended to 64-bits is a random large value this out
out of bounds. */
addl %edx, %eax
/* Crash here because 32-bit negative number in `eax` zero
extends to out of bounds 64-bit offset. */
movzbl 16(%rdi, %rax), %ecx
movzbl 16(%rsi, %rax), %eax
```
This fix is quite simple, just make the `addl %edx, %eax` 64 bit (i.e
`addq %rdx, %rax`). This prevents the 32-bit zero extension
and since `eax` is still a low bound of `16 - len` the `rdx + rax`
is bound by `(len - 32) - (16 - len) >= -16`. Since we have a
fixed offset of `16` in the memory access this must be in bounds.
A recent GCC change resulted in localplt test failures on sparc64
because of references to _Qp_fgt. This is analogous to all the other
floating-point symbols allowed in localplt.data, so it seems
appropriate to allow this one as well.
Tested with build-many-glibcs.py for sparc64-linux-gnu (GCC mainline),
where it fixes the test failure.
