clone already uses r31 to temporarily save input arguments before doing the
syscall, so we use a different register to read from the TCB. We can also avoid
allocating another stack frame, which is not needed since we can simply extend
the usage of the red zone.
Tested-by: Lucas A. M. Magalhães <lamm@linux.ibm.com>
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
Linux kernel v5.9 added support for system calls using the scv
instruction for POWER9 and later. The new codepath provides better
performance (see below) if compared to using sc. For the
foreseeable future, both sc and scv mechanisms will co-exist, so this
patch enables glibc to do a runtime check and use scv when it is
available.
Before issuing the system call to the kernel, we check hwcap2 in the TCB
for PPC_FEATURE2_SCV to see if scv is supported by the kernel. If not,
we fallback to sc and keep the old behavior.
The kernel implements a different error return convention for scv, so
when returning from a system call we need to handle the return value
differently depending on the instruction we used to enter the kernel.
For syscalls implemented in ASM, entry and exit are implemented by
different macros (PSEUDO and PSEUDO_RET, resp.), which may be used in
sequence (e.g. for templated syscalls) or with other instructions in
between (e.g. clone). To avoid accessing the TCB a second time on
PSEUDO_RET to check which instruction we used, the value read from
hwcap2 is cached on a non-volatile register.
This is not needed when using INTERNAL_SYSCALL macro, since entry and
exit are bundled into the same inline asm directive.
The dynamic loader may issue syscalls before the TCB has been setup
so it always uses sc with no extra checks. For the static case, there
is no compile-time way to determine if we are inside startup code,
so we also check the value of the thread pointer before effectively
accessing the TCB. For such situations in which the availability of
scv cannot be determined, sc is always used.
Support for scv in syscalls implemented in their own ASM file (clone and
vfork) will be added later. For now simply use sc as before.
Average performance over 1M calls for each syscall "type":
- stat: C wrapper calling INTERNAL_SYSCALL
- getpid: templated ASM syscall
- syscall: call to gettid using syscall function
Standard:
stat : 1.573445 us / ~3619 cycles
getpid : 0.164986 us / ~379 cycles
syscall : 0.162743 us / ~374 cycles
With scv:
stat : 1.537049 us / ~3535 cycles <~ -84 cycles / -2.32%
getpid : 0.109923 us / ~253 cycles <~ -126 cycles / -33.25%
syscall : 0.116410 us / ~268 cycles <~ -106 cycles / -28.34%
Tested on powerpc, powerpc64, powerpc64le (with and without scv)
Tested-by: Lucas A. M. Magalhães <lamm@linux.ibm.com>
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
Linux 5.10 has one new syscall, process_madvise. Update
syscall-names.list and regenerate the arch-syscall.h headers with
build-many-glibcs.py update-syscalls.
Tested with build-many-glibcs.py.
It removes all the arch-specific assembly implementation. The
outliers are alpha, where its kernel ABI explict return -ENOMEM
in case of failure; and i686, where it can't use
"call *%gs:SYSINFO_OFFSET" during statup in static PIE.
Also some ABIs exports an additional ___brk_addr symbol and to
handle it an internal HAVE_INTERNAL_BRK_ADDR_SYMBOL is added.
Checked on x86_64-linux-gnu, i686-linux-gnu, adn with builsd for
the affected ABIs.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
Linux 5.9 has one new syscall, close_range. Update syscall-names.list
and regenerate the arch-syscall.h headers with build-many-glibcs.py
update-syscalls.
Tested with build-many-glibcs.py.
This patch removes the mknod and mknodat static wrapper and add the
symbols on the libc with the expected names.
Both the prototypes of the internal symbol linked by the static
wrappers and the inline redirectors are also removed from the installed
sys/stat.h header file. The wrapper implementation license LGPL
exception is also removed since it is no longer statically linked to
binaries.
Internally the _STAT_VER* definitions are moved to the arch-specific
xstatver.h file.
Checked with a build for all affected ABIs. I also checked on x86_64,
i686, powerpc, powerpc64le, sparcv9, sparc64, s390, and s390x.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
This patch removes the stat, stat64, lstat, lstat64, fstat, fstat64,
fstatat, and fstatat64 static wrapper and add the symbol on the libc
with the expected names.
Both the prototypes of the internal symbol linked by the static
wrappers and the inline redirectors are also removed from the installed
sys/stat.h header file. The wrapper implementation license LGPL
exception is also removed since it is no longer statically linked to
binaries.
Internally the _STAT_VER* definitions are moved to a arch-specific
xstatver.h file. The internal defines that redirects internals
{f}stat{at} to their {f}xstat{at} counterparts are removed for Linux
(!NO_RTLD_HIDDEN). Hurd still requires them since {f}stat{at} pulls
extra objects that makes the loader build fail otherwise (I haven't
dig into why exactly).
Checked with a build for all affected ABIs. I also checked on x86_64,
i686, powerpc, powerpc64le, sparcv9, sparc64, s390, and s390x.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
This patch adds the ABI-related bits to reflect the new mallinfo2
function, and adds a test case to verify basic functionality.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
It indicates that the glibc export stat64 is similar in size and
layout of the kernel stat64 used on the syscall. It is not currently
used on stat implementation, but the idea is to indicate whether
to use the kernel_stat to issue on the syscall on the *stat*64
variant (more specifically on mips which its exported ABI does not
match the kernel).
Reviewed-by: Lukasz Majewski <lukma@denx.de>
Linux 5.8 has one new syscall, faccessat2. Update syscall-names.list
and regenerate the arch-syscall.h headers with build-many-glibcs.py
update-syscalls.
Tested with build-many-glibcs.py.
The kernel ABI is not finalized, and there are now various proposals
to change the size of struct rseq, which would make the glibc ABI
dependent on the version of the kernels used for building glibc.
This is of course not acceptable.
This reverts commit 48699da1c4 ("elf:
Support at least 32-byte alignment in static dlopen"), commit
8f4632deb3 ("Linux: rseq registration
tests"), commit 6e29cb3f61 ("Linux: Use
rseq in sched_getcpu if available"), and commit
0c76fc3c2b ("Linux: Perform rseq
registration at C startup and thread creation"), resolving the conflicts
introduced by the ARC port and the TLS static surplus changes.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Teach the linker that __mcount_internal, __sigjmp_save_symbol,
__syscall_error and __GI_exit do not use r2, so that it does not need to
recover r2 after the call.
Test at configure time if the assembler supports @notoc and define
USE_PPC64_NOTOC.
The strerrorname_np returns error number name (e.g. "EINVAL" for EINVAL)
while strerrordesc_np returns string describing error number (e.g
"Invalid argument" for EINVAL). Different than strerror,
strerrordesc_np does not attempt to translate the return description,
both functions return NULL for an invalid error number.
They should be used instead of sys_errlist and sys_nerr, both are
thread and async-signal safe. These functions are GNU extensions.
Checked on x86-64-linux-gnu, i686-linux-gnu, powerpc64le-linux-gnu,
and s390x-linux-gnu.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The sigabbrev_np returns the abbreviated signal name (e.g. "HUP" for
SIGHUP) while sigdescr_np returns the string describing the error
number (e.g "Hangup" for SIGHUP). Different than strsignal,
sigdescr_np does not attempt to translate the return description and
both functions return NULL for an invalid signal number.
They should be used instead of sys_siglist or sys_sigabbrev and they
are both thread and async-signal safe. They are added as GNU
extensions on string.h header (same as strsignal).
Checked on x86-64-linux-gnu, i686-linux-gnu, powerpc64le-linux-gnu,
and s390x-linux-gnu.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The variable is placed in libc.so, and it can be true only in
an outer libc, not libcs loaded via dlmopen or static dlopen.
Since thread creation from inner namespaces does not work,
pthread_create can update __libc_single_threaded directly.
Using __libc_early_init and its initial flag, implementation of this
variable is very straightforward. A future version may reset the flag
during fork (but not in an inner namespace), or after joining all
threads except one.
Reviewed-by: DJ Delorie <dj@redhat.com>
Register rseq TLS for each thread (including main), and unregister for
each thread (excluding main). "rseq" stands for Restartable Sequences.
See the rseq(2) man page proposed here:
https://lkml.org/lkml/2018/9/19/647
Those are based on glibc master branch commit 3ee1e0ec5c.
The rseq system call was merged into Linux 4.18.
The TLS_STATIC_SURPLUS define is increased to leave additional room for
dlopen'd initial-exec TLS, which keeps elf/tst-auditmany working.
The increase (76 bytes) is larger than 32 bytes because it has not been
increased in quite a while. The cost in terms of additional TLS storage
is quite significant, but it will also obscure some initial-exec-related
dlopen failures.
This patch changes the exp10f error handling semantics to only set
errno according to POSIX rules. New symbol version is introduced at
GLIBC_2.32. The old wrappers are kept for compat symbols.
There are some outliers that need special handling:
- ia64 provides an optimized implementation of exp10f that uses ia64
specific routines to set SVID compatibility. The new symbol version
is aliased to the exp10f one.
- m68k also provides an optimized implementation, and the new version
uses it instead of the sysdeps/ieee754/flt32 one.
- riscv and csky uses the generic template implementation that
does not provide SVID support. For both cases a new exp10f
version is not added, but rather the symbols version of the
generic sysdeps/ieee754/flt32 is adjusted instead.
Checked on aarch64-linux-gnu, x86_64-linux-gnu, i686-linux-gnu,
powerpc64le-linux-gnu.
This is part of the libpthread removal project:
<https://sourceware.org/ml/libc-alpha/2019-10/msg00080.html>
Use __getline instead of __getdelim to avoid a localplt failure.
Likewise for __getrlimit/getrlimit.
The abilist updates were performed by:
git ls-files 'sysdeps/unix/sysv/linux/**/libc.abilist' \
| while read x ; do
echo "GLIBC_2.32 pthread_getattr_np F" >> $x
done
python3 scripts/move-symbol-to-libc.py --only-linux pthread_getattr_np
The private export of __pthread_getaffinity_np is no longer needed, but
the hidden alias still necessary so that the symbol can be exported with
versioned_symbol.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
This is part of the libpthread removal project:
<https://sourceware.org/ml/libc-alpha/2019-10/msg00080.html>
The abilist updates were performed by:
git ls-files 'sysdeps/unix/sysv/linux/**/libc.abilist' \
| while read x ; do
echo "GLIBC_2.32 pthread_getaffinity_np F" >> $x
done
python3 scripts/move-symbol-to-libc.py pthread_getaffinity_np
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
This is part of the libpthread removal project:
<https://sourceware.org/ml/libc-alpha/2019-10/msg00080.html>
The symbol did not previously exist in libc, so a new GLIBC_2.32
symbol is needed, to get correct dependency for binaries which
use the symbol but no longer link against libpthread.
The abilist updates were performed by:
git ls-files 'sysdeps/unix/sysv/linux/**/libc.abilist' \
| while read x ; do
echo "GLIBC_2.32 pthread_attr_setaffinity_np F" >> $x
done
python3 scripts/move-symbol-to-libc.py pthread_attr_setaffinity_np
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
On platforms where long double may have two different formats, i.e.: the
same format as double (64-bits) or something else (128-bits), building
with -mlong-double-128 is the default and function calls in the user
program match the name of the function in Glibc. When building with
-mlong-double-64, Glibc installed headers redirect such calls to the
appropriate function.
Likewise, the internals of glibc are now built against IEEE long double.
However, the only (minimally) notable usage of long double is difftime.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
This is part of the libpthread removal project:
<https://sourceware.org/ml/libc-alpha/2019-10/msg00080.html>
A new symbol version is added on libc to force loading failure
instead of lazy binding one for newly binaries with old loaders.
Checked with a build against all affected ABIs.
Linux 5.5 remove the system call in commit
61a47c1ad3a4dc6882f01ebdc88138ac62d0df03 ("Linux: Remove
<sys/sysctl.h>"). Therefore, the compat function is just a stub that
sets ENOSYS.
Due to SHLIB_COMPAT, new ports will not add the sysctl function anymore
automatically.
x32 already lacks the sysctl function, so an empty sysctl.c file is
used to suppress it. Otherwise, a new compat symbol would be added.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Linux 5.6 has new openat2 and pidfd_getfd syscalls. This patch adds
them to syscall-names.list and regenerates the arch-syscall.h files.
Tested with build-many-glibcs.py.
It is necessary to export __pthread_cond_init from libc because
the C11 condition variable needs it and is still left in libpthread.
This is part of the libpthread removal project:
<https://sourceware.org/ml/libc-alpha/2019-10/msg00080.html>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
It is necessary to export __pthread_cond_destroy from libc because
the C11 condition variable needs it and is still left in libpthread.
This is part of the libpthread removal project:
<https://sourceware.org/ml/libc-alpha/2019-10/msg00080.html>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Exporting functions and relying on symbol interposition from libc.so
makes the choice of implementation dependent on DT_NEEDED order, which
is not what some compiler drivers expect.
This commit replaces one magic mechanism (symbol interposition) with
another one (preprocessor-/compiler-based redirection). This makes
the hand-over from the minimal malloc to the full malloc more
explicit.
Removing the ABI symbols is backwards-compatible because libc.so is
always in scope, and the dynamic loader will find the malloc-related
symbols there since commit f0b2132b35
("ld.so: Support moving versioned symbols between sonames
[BZ #24741]").
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
With all Linux ABIs using the expected Linux kABI to indicate
syscalls errors, the INTERNAL_SYSCALL_DECL is an empty declaration
on all ports.
This patch removes the 'err' argument on INTERNAL_SYSCALL* macro
and remove the INTERNAL_SYSCALL_DECL usage.
Checked with a build against all affected ABIs.
The diferences between powerpc64{le} and powerpc32 Linux sysdep.h
are:
1. On both vDSO and syscall macros the volatile registers r9, r10,
r11, and r12 are used as input operands on powerpc32 and as
clobber registers on powerpc64. However the outcome is essentially
the same, it advertise the register might be clobbered by the
kernel (although Linux won't leak register information to userland
in such case).
2. The LOADARGS* macros uses a different size to check for invalid
types.
3. The pointer mangling support guard pointer loading uses ABI
specific instruction and register.
This patch consolidates on only one sysdep by using the the powerpc64
version as default and add the adjustments required for powerpc32.
Checked on powerpc64-linux-gnu, powerpc64le-linux-gnu, and
powerpc-linux-gnu-power4.
The 32-bit protection key behavior is somewhat unclear on 32-bit powerpc,
so this change is restricted to the 64-bit variants.
Flag translation is needed because of hardware differences between the
POWER implementation (read and write flags) and the Intel implementation
(write and read+write flags).
The new tables are currently only used for consistency checks
with the installed kernel headers and the architecture-independent
system call names table. They are based on Linux 5.4.
The goal is to use these architecture-specific tables to ensure
that system call wrappers are available irrespective of the version
of the installed kernel headers.
The tables are formatted in the form of C header files so that they
can be used directly in an #include directive, without external
preprocessing. (External preprocessing of a plain table file
would introduce cross-subdirectory dependency issues.) However,
the intent is that they can still be treated as tables and can be
processed by simple tools.
The irregular system call names on 32-bit arm add a complication.
The <fixup-asm-unistd.h> header is introduced to work around that,
and the system calls are listed under regular names in the
<arch-syscall.h> file.
A make target, update-syscalls-list, is added to patch the glibc
sources with data from the current kernel headers.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Checked on x86_64-linux-gnu and powerpc64le-linux-gnu. I also checked
the libpthread.so .gnu.version_d entries for every ABI affected and
all of them contains the required versions (including for architectures
which exports __nanosleep with a different version).
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Introduce pthread_clockjoin_np as a version of pthread_timedjoin_np that
accepts a clockid_t parameter to indicate which clock the timeout should be
measured against. This mirrors the recently-added POSIX-proposed "clock"
wait functions.
Checked on x86_64-linux-gnu.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch simplifies the powerpc internal macros for vDSO calls
by:
- Removing INTERNAL_VSYSCALL_NO_SYSCALL_FALLBACK, used solely on
get_timebase_freq.
- Adjust INTERNAL_VSYSCALL_CALL_TYPE powerpc32 to follow powerpc64
argument ordering.
- Use HAVE_*_VSYSCALL instead of explicit strings.
- Make powerpc libc-vdso.h include generic implementation.
No semantic change expected, checked on powerpc-linux-gnu-power4,
powerpc64-linux-gnu, and powerpc64le-linux-gnu.
* sysdeps/unix/sysv/linux/libc-vdso.h (VDSO_IFUNC_RET): Define if not
defined.
* sysdeps/unix/sysv/linux/powerpc/get_timebase_freq.c
(__get_timebase_freq): Remove use of
INTERNAL_VSYSCALL_NO_SYSCALL_FALLBACK.
(get_timebase_freq_fallback): New symbol.
* sysdeps/unix/sysv/linux/powerpc/gettimeofday.c (time): Use
HAVE_GETTIMEOFDAY_VSYSCALL.
* sysdeps/unix/sysv/linux/powerpc/time.c (gettimeofday): Use
HAVE_TIME_VSYSCALL.
* sysdeps/unix/sysv/linux/powerpc/libc-vdso.h: Include generic
implementation.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/sysdep.h
(INTERNAL_VSYSCALL_CALL_TYPE): Make calling convention similar to
powerpc64.
(INTERNAL_VSYSCALL_NO_SYSCALL_FALLBACK): Remove macro.
* .../sysv/linux/powerpc/powerpc64/sysdep.h
(INTERNAL_VSYSCALL_NO_SYSCALL_FALLBACK): Likewise.
* sysdeps/unix/sysv/linux/powerpc/sysdep.h
(HAVE_GETTIMEOFDAY_VSYSCALL): Define.
