This patch modifies the current Power9 implementation of strcpy and
stpcpy to optimize it for Power9 and Power10.
No new Power10 instructions are used, so the original Power9 strcpy
is modified instead of creating a new implementation for Power10.
The changes also affect stpcpy, which uses the same implementation
with some additional code before returning.
Improvements compared to the old Power9 version:
Use simple comparisons for the first ~512 bytes:
The main loop is good for long strings, but comparing 16B each time is
better for shorter strings. After aligning the address to 16 bytes, we
unroll the loop four times, checking 128 bytes each time. There may be
some overlap with the main loop for unaligned strings, but it is better
for shorter strings.
Loop with 64 bytes for longer bytes:
Use 4 consecutive lxv/stxv instructions.
Showed an average improvement of 13%.
Reviewed-by: Paul E. Murphy <murphyp@linux.ibm.com>
Reviewed-by: Peter Bergner <bergner@linux.ibm.com>
The current racy approach is to enable asynchronous cancellation
before making the syscall and restore the previous cancellation
type once the syscall returns, and check if cancellation has happen
during the cancellation entrypoint.
As described in BZ#12683, this approach shows 2 problems:
1. Cancellation can act after the syscall has returned from the
kernel, but before userspace saves the return value. It might
result in a resource leak if the syscall allocated a resource or a
side effect (partial read/write), and there is no way to program
handle it with cancellation handlers.
2. If a signal is handled while the thread is blocked at a cancellable
syscall, the entire signal handler runs with asynchronous
cancellation enabled. This can lead to issues if the signal
handler call functions which are async-signal-safe but not
async-cancel-safe.
For the cancellation to work correctly, there are 5 points at which the
cancellation signal could arrive:
[ ... )[ ... )[ syscall ]( ...
1 2 3 4 5
1. Before initial testcancel, e.g. [*... testcancel)
2. Between testcancel and syscall start, e.g. [testcancel...syscall start)
3. While syscall is blocked and no side effects have yet taken
place, e.g. [ syscall ]
4. Same as 3 but with side-effects having occurred (e.g. a partial
read or write).
5. After syscall end e.g. (syscall end...*]
And libc wants to act on cancellation in cases 1, 2, and 3 but not
in cases 4 or 5. For the 4 and 5 cases, the cancellation will eventually
happen in the next cancellable entrypoint without any further external
event.
The proposed solution for each case is:
1. Do a conditional branch based on whether the thread has received
a cancellation request;
2. It can be caught by the signal handler determining that the saved
program counter (from the ucontext_t) is in some address range
beginning just before the "testcancel" and ending with the
syscall instruction.
3. SIGCANCEL can be caught by the signal handler and determine that
the saved program counter (from the ucontext_t) is in the address
range beginning just before "testcancel" and ending with the first
uninterruptable (via a signal) syscall instruction that enters the
kernel.
4. In this case, except for certain syscalls that ALWAYS fail with
EINTR even for non-interrupting signals, the kernel will reset
the program counter to point at the syscall instruction during
signal handling, so that the syscall is restarted when the signal
handler returns. So, from the signal handler's standpoint, this
looks the same as case 2, and thus it's taken care of.
5. For syscalls with side-effects, the kernel cannot restart the
syscall; when it's interrupted by a signal, the kernel must cause
the syscall to return with whatever partial result is obtained
(e.g. partial read or write).
6. The saved program counter points just after the syscall
instruction, so the signal handler won't act on cancellation.
This is similar to 4. since the program counter is past the syscall
instruction.
So The proposed fixes are:
1. Remove the enable_asynccancel/disable_asynccancel function usage in
cancellable syscall definition and instead make them call a common
symbol that will check if cancellation is enabled (__syscall_cancel
at nptl/cancellation.c), call the arch-specific cancellable
entry-point (__syscall_cancel_arch), and cancel the thread when
required.
2. Provide an arch-specific generic system call wrapper function
that contains global markers. These markers will be used in
SIGCANCEL signal handler to check if the interruption has been
called in a valid syscall and if the syscalls has side-effects.
A reference implementation sysdeps/unix/sysv/linux/syscall_cancel.c
is provided. However, the markers may not be set on correct
expected places depending on how INTERNAL_SYSCALL_NCS is
implemented by the architecture. It is expected that all
architectures add an arch-specific implementation.
3. Rewrite SIGCANCEL asynchronous handler to check for both canceling
type and if current IP from signal handler falls between the global
markers and act accordingly.
4. Adjust libc code to replace LIBC_CANCEL_ASYNC/LIBC_CANCEL_RESET to
use the appropriate cancelable syscalls.
5. Adjust 'lowlevellock-futex.h' arch-specific implementations to
provide cancelable futex calls.
Some architectures require specific support on syscall handling:
* On i386 the syscall cancel bridge needs to use the old int80
instruction because the optimized vDSO symbol the resulting PC value
for an interrupted syscall points to an address outside the expected
markers in __syscall_cancel_arch. It has been discussed in LKML [1]
on how kernel could help userland to accomplish it, but afaik
discussion has stalled.
Also, sysenter should not be used directly by libc since its calling
convention is set by the kernel depending of the underlying x86 chip
(check kernel commit 30bfa7b3488bfb1bb75c9f50a5fcac1832970c60).
* mips o32 is the only kABI that requires 7 argument syscall, and to
avoid add a requirement on all architectures to support it, mips
support is added with extra internal defines.
Checked on aarch64-linux-gnu, arm-linux-gnueabihf, powerpc-linux-gnu,
powerpc64-linux-gnu, powerpc64le-linux-gnu, i686-linux-gnu, and
x86_64-linux-gnu.
[1] https://lkml.org/lkml/2016/3/8/1105
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The test io/tst-mkdirat doesn't verify the permissions on the created
directory (thus, doesn't verify at all anything about how mkdirat uses
the mode argument). Add checks of this to the existing test.
Tested for x86_64.
There is very little test coverage for getline (only a minimal
stdio-common/tstgetln.c which doesn't verify anything about the
results of the getline calls). Add some more thorough tests
(generally using fopencookie for convenience in testing various cases
for what the input and possible errors / EOF in the file read might
look like).
Note the following regarding testing of error cases:
* Nothing is said in the specifications about what if anything might
be written into the buffer, and whether it might be reallocated, in
error cases. The expectation of the tests (required to avoid memory
leaks on error) is that at least on error cases, the invariant that
lineptr points to at least n bytes is maintained.
* The optional EOVERFLOW error case specified in POSIX, "The number of
bytes to be written into the buffer, including the delimiter
character (if encountered), would exceed {SSIZE_MAX}.", doesn't seem
practically testable, as any case reading so many characters (half
the address space) would also be liable to run into allocation
failure along (ENOMEM) the way.
* If a read error occurs part way through reading an input line, it
seems unclear whether a partial line should be returned by getline
(avoid input getting lost), which is what glibc does at least in the
fopencookie case used in this test, or whether getline should return
-1 (error) (so avoiding the program misbehaving by processing a
truncated line as if it were complete). (There was a short,
inconclusive discussion about this on the Austin Group list on 9-10
November 2014.)
* The POSIX specification of getline inherits errors from fgetc. I
didn't try to cover fgetc errors systematically, just one example of
such an error.
Tested for x86_64 and x86.
This will avoid in the future cases like a57cbbd853 ("malloc: Link
threading tests with $(shared-thread-library") missing the memcheck
cases added in 251843e16f ("malloc: Link threading tests with
$(shared-thread-library)")
Tests for if_nameindex, if_name2index, and if_index2name
Tests that valid results are consistent.
Tests that invalid parameters fail correctly.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
With ia64 removal, the function descriptor supports is only used
by HPPA and new architectures do not seem leaning towards this
design.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
This commit uses a common implementation 'strnlen-evex-base.S' for both
'strnlen-evex' and 'strnlen-evex512'
This patch serves both to reduce the number of implementations, and it also does some small optimizations that benefit strnlen-evex and strnlen-evex512.
All tests pass on x86.
Benchmarks were taken on SKX.
https://www.intel.com/content/www/us/en/products/sku/123613/intel-core-i97900x-xseries-processor-13-75m-cache-up-to-4-30-ghz/specifications.html
Geometric mean for strnlen-evex over all benchmarks (N=10) was (new/old) 0.881
Geometric mean for strnlen-evex512 over all benchmarks (N=10) was (new/old) 0.953
Code Size Changes:
strnlen-evex : +31 bytes
strnlen-evex512 : +156 bytes
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Secondary namespaces have a different malloc. Allocating the
buffer in one namespace and freeing it another results in
heap corruption. Fix this by using a static string (potentially
translated) in secondary namespaces. It would also be possible
to use the malloc from the initial namespace to manage the
buffer, but these functions would still not be safe to use in
auditors etc. because a call to strerror could still free a
buffer while it is used by the application. Another approach
could use proper initial-exec TLS, duplicated in secondary
namespaces, but that would need a callback interface for freeing
libc resources in namespaces on thread exit, which does not exist
today.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Macros will automatically use the correct types, without
having to fiddle with internal glibc macros. It's also
impossible to get the types wrong due to aliasing because
support_check_stat_fd and support_check_stat_path do not
depend on the struct stat* types.
The changes reveal some inconsistencies in tests.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This avoids the need to define struct_statx to an appropriate
struct stat type variant because struct statx does not change
based on time/file offset flags.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This is currently implied by the internal headers, but it makes
sense not to rely on this.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This is not needed: include/intprops.h has its own detection logic.
It makes building these files outside of glibc easer.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Fix an issue with commit 8f4632deb3 ("Linux: rseq registration tests")
and prevent testing from being run in the process of the test driver
itself rather than just the test child where one has been forked. The
problem here is the unguarded use of a destructor to call a part of the
testing. The destructor function, 'do_rseq_destructor_test' is called
implicitly at program completion, however because it is associated with
the executable itself rather than an individual process, it is called
both in the test child *and* in the test driver itself.
Prevent this from happening by providing a guard variable that only
enables test invocation from 'do_rseq_destructor_test' in the process
that has first run 'do_test'. Consequently extra testing is invoked
from 'do_rseq_destructor_test' only once and in the correct process,
regardless of the use or the lack of of the '--direct' option. Where
called in the controlling test driver process that has neved called
'do_test' the destructor function silently returns right away without
taking any further actions, letting the test driver fail gracefully
where applicable.
This arrangement prevents 'tst-rseq-nptl' from ever causing testing to
hang forever and never complete, such as currently happening with the
'mips-linux-gnu' (o32 ABI) target.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Previously if the setaffinity wrapper failed the rest of the subtest
would not execute and the current subtest would be reported as passing.
Now if the setaffinity wrapper fails the subtest is correctly reported
as faling. Tested manually by changing the conditions of the affinity
call including setting size to zero, or checking the wrong condition.
No regressions on x86_64.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
If a file descriptor is left unclosed and is cleaned up by _IO_cleanup
on exit, its backup buffer remains unfreed, registering as a leak in
valgrind. This is not strictly an issue since (1) the program should
ideally be closing the stream once it's not in use and (2) the program
is about to exit anyway, so keeping the backup buffer around a wee bit
longer isn't a real problem. Free it anyway to keep valgrind happy
when the streams in question are the standard ones, i.e. stdout, stdin
or stderr.
Also, the _IO_have_backup macro checks for _IO_save_base,
which is a roundabout way to check for a backup buffer instead of
directly looking for _IO_backup_base. The roundabout check breaks when
the main get area has not been used and user pushes a char into the
backup buffer with ungetc. Fix this to use the _IO_backup_base
directly.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
When ungetc is called on an unused stream, the backup buffer is
allocated without the main get area being present. This results in
every subsequent ungetc (as the stream remains in the backup area)
checking uninitialized memory in the backup buffer when trying to put a
character back into the stream.
Avoid comparing the input character with buffer contents when in backup
to avoid this uninitialized read. The uninitialized read is harmless in
this context since the location is promptly overwritten with the input
character, thus fulfilling ungetc functionality.
Also adjust wording in the manual to drop the paragraph that says glibc
cannot do multiple ungetc back to back since with this change, ungetc
can actually do this.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The goal of this flag is to allow targets which don't prefer/have ERMS
to still access the non-temporal memset implementation.
There are 4 cases for tuning memset:
1) `Avoid_STOSB && Avoid_Non_Temporal_Memset`
- Memset with temporal stores
2) `Avoid_STOSB && !Avoid_Non_Temporal_Memset`
- Memset with temporal/non-temporal stores. Non-temporal path
goes through `rep stosb` path. We accomplish this by setting
`x86_rep_stosb_threshold` to
`x86_memset_non_temporal_threshold`.
3) `!Avoid_STOSB && Avoid_Non_Temporal_Memset`
- Memset with temporal stores/`rep stosb`
3) `!Avoid_STOSB && !Avoid_Non_Temporal_Memset`
- Memset with temporal stores/`rep stosb`/non-temporal stores.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
This is just a refactor and there should be no behavioral change from
this commit.
The goal is to make `Avoid_Non_Temporal_Memset` a more universal knob
for controlling whether we use non-temporal memset rather than having
extra logic based on vendor.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Issue was we were expecting not matches with CHAR before the start of
the string in the page cross case.
The check code in the page cross case:
```
and $0xffffffffffffffc0,%rax
vmovdqa64 (%rax),%zmm17
vpcmpneqb %zmm17,%zmm16,%k1
vptestmb %zmm17,%zmm17,%k0{%k1}
kmovq %k0,%rax
inc %rax
shr %cl,%rax
je L(continue)
```
expects that all characters that neither match null nor CHAR will be
1s in `rax` prior to the `inc`. Then the `inc` will overflow all of
the 1s where no relevant match was found.
This is incorrect in the page-cross case, as the
`vmovdqa64 (%rax),%zmm17` loads from before the start of the input
string.
If there are matches with CHAR before the start of the string, `rax`
won't properly overflow.
The fix is quite simple. Just replace:
```
inc %rax
shr %cl,%rax
```
With:
```
sar %cl,%rax
inc %rax
```
The arithmetic shift will clear any matches prior to the start of the
string while maintaining the signbit so the 1s can properly overflow
to zero in the case of no matches.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
We have no tests that errno is set to ERANGE on overflow of
strtod-family functions (we do have some tests for underflow, in
tst-strtod-underflow). Add such tests to tst-strtod-round.
Tested for x86_64.
In _dl_tlsdesc_dynamic, there are three 'addi.d sp, sp, -size'
instructions to allocate stack size for Float/LSX/LASX registers.
Every 'addi.d sp, sp, -size' needs a cfi_adjust_cfa_offset because
of sp is used to compute CFA. But only one 'addi.d sp, sp, -size'
will be run according to HWCAP value. And all cfi_adjust_cfa_offset
will be executed in stack unwinding, it result in incorrect CFA.
Change _dl_tlsdesc_dynamic to _dl_tlsdesc_dynamic,
_dl_tlsdesc_dynamic_lsx and _dl_tlsdesc_dynamic_lasx.
Conflicting cfi instructions can be distributed to the three functions.
And cfi instructions can correspond to stack down instructions.
Fix an issue with commit b74121ae4b ("Update.") and prevent a stray
process from being left behind by tst-cancel7 (and also tst-cancelx7,
which is the same test built with '-fexceptions' additionally supplied
to the compiler), which then blocks remote testing until the process has
been killed by hand.
This test case creates a thread that runs an extra copy of the test via
system(3) and using the '--direct' option so that the test wrapper does
not interfere with this instance. This extra copy executes its business
and calls sigsuspend(2) and then never terminates by itself. Instead it
relies on being killed by the main test process directly via a thread
cancellation request or, should that fail, by issuing SIGKILL either at
the conclusion of 'do_test' or by the test driver via 'do_cleanup' where
the test timeout has been hit or the test driver interrupted.
However if the main test process has been instead killed by a signal,
such as due to incorrect execution, before it had a chance to kill the
extra copy of the test case, then the test wrapper will terminate
without running 'do_cleanup' and consequently the extra copy of the test
case will remain forever in its suspended state, and in the remote case
in particular it means that the remote test wrapper will wait forever
for the SSH command to complete.
This has been observed with the 'alpha-linux-gnu' target, where the main
test process triggers SIGSEGV and the test wrapper correctly records:
Didn't expect signal from child: got `Segmentation fault'
in nptl/tst-cancel7.out and terminates, but then the calling SSH command
continues waiting for the remaining process started in the same session
on the remote target to complete.
Address this problem by also registering 'do_cleanup' via atexit(3),
observing that 'support_delete_temp_files' is registered by the test
wrapper before the test initializing function 'do_prepare' is called and
that we call all the functions registered in the reverse of the order in
which they were registered, so it is safe to refer to 'pidfilename' in
'do_cleanup' invoked by exit(3) because by that time temporary files
have not yet been deleted.
A minor inconvenience is that if 'signal_handler' is invoked in the test
wrapper as a result of SIGALRM rather than SIGINT, then 'do_cleanup'
will be called twice, once as a cleanup handler and again by exit(3).
In reality it is harmless though, because issuing SIGKILL is guarded by
a record lock, so if the first call has succeeded in killing the extra
copy of the test case, then the subsequent call will do nothing.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Move the release of the semaphore used to synchronize between an extra
copy of the test run as a separate process and the main test process
until after the PID file has been locked. It is so that if the cleanup
function gets called by the test driver due to premature termination of
the main test process, then the function does not get at the PID file
before it has been locked and conclude that the extra copy of the test
has already terminated. This won't usually happen due to a relatively
high amount of time required to elapse before timeout triggers in the
test driver, but it will change with the next change.
There is still a small time window remaining with this change in place
where the main test process gets killed for some reason between the
extra copy of the test has been already started by pthread_create(3) and
a successful return from the call to sem_wait(3), in which case the
cleanup function can be reached before PID has been written to the PID
file and the file locked. It seems that with the test case structured
as it is now and PID-based process management we have no means to avoid
it.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
