All the required code already existed, and some of it was already
running.
AT_SYSINFO_EHDR is processed if NEED_DL_SYSINFO_DSO is defined, but it
looks like it always is. The call to setup_vdso is also unconditional,
so all that was left to do was setup the function pointers and use
them. This patch just deletes some #ifdef to enable that.
[BZ #19767]
* nptl/Makefile (tests-static): Add tst-cond11-static.
(tests): Likewise.
* nptl/tst-cond11-static.c: New File.
* sysdeps/unix/sysv/linux/Makefile (tests-static): Add
tst-affinity-static.
(tests): Likewise.
* sysdeps/unix/sysv/linux/sysdep-vdso.h: Check USE_VSYSCALL
instead of SHARED.
* sysdeps/unix/sysv/linux/sysdep.h (ALWAYS_USE_VSYSCALL): New.
(USE_VSYSCALL): Likewise.
* sysdeps/unix/sysv/linux/tst-affinity-static.c: New file.
* sysdeps/unix/sysv/linux/x86/libc-vdso.h: Check USE_VSYSCALL
instead of SHARED.
* sysdeps/unix/sysv/linux/x86_64/init-first.c: Don't check
SHARED.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h (ALWAYS_USE_VSYSCALL):
New.
This patch adds a field to ucontext_t to save shadow stack:
1. getcontext and swapcontext are updated to save the caller's shadow
stack pointer and return addresses.
2. setcontext and swapcontext are updated to restore shadow stack and
jump to new context directly.
3. makecontext is updated to allocate a new shadow stack and set the
caller's return address to __start_context.
Since makecontext allocates a new shadow stack when making a new
context and kernel allocates a new shadow stack for clone/fork/vfork
syscalls, we track the current shadow stack base. In setcontext and
swapcontext, if the target shadow stack base is the same as the current
shadow stack base, we unwind the shadow stack. Otherwise it is a stack
switch and we look for a restore token.
We enable shadow stack at run-time only if program and all used shared
objects, including dlopened ones, are shadow stack enabled, which means
that they must be compiled with GCC 8 or above and glibc 2.28 or above.
We need to save and restore shadow stack only if shadow stack is enabled.
When caller of getcontext, setcontext, swapcontext and makecontext is
compiled with smaller ucontext_t, shadow stack won't be enabled at
run-time. We check if shadow stack is enabled before accessing the
extended field in ucontext_t.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* sysdeps/unix/sysv/linux/x86/sys/ucontext.h (ucontext_t): Add
__ssp.
* sysdeps/unix/sysv/linux/x86_64/__start_context.S: Include
<asm/prctl.h> and "ucontext_i.h" when shadow stack is enabled.
(__push___start_context): New.
* sysdeps/unix/sysv/linux/x86_64/getcontext.S: Include
<asm/prctl.h>.
(__getcontext): Record the current shadow stack base. Save the
caller's shadow stack pointer and base.
* sysdeps/unix/sysv/linux/x86_64/makecontext.c: Include
<pthread.h>, <libc-pointer-arith.h> and <sys/prctl.h>.
(__push___start_context): New prototype.
(__makecontext): Call __push___start_context to allocate a new
shadow stack, push __start_context onto the new stack as well
as the new shadow stack.
* sysdeps/unix/sysv/linux/x86_64/setcontext.S: Include
<asm/prctl.h>.
(__setcontext): Restore the target shadow stack.
* sysdeps/unix/sysv/linux/x86_64/swapcontext.S: Include
<asm/prctl.h>.
(__swapcontext): Record the current shadow stack base. Save
the caller's shadow stack pointer and base. Restore the target
shadow stack.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h
(STACK_SIZE_TO_SHADOW_STACK_SIZE_SHIFT): New.
* sysdeps/unix/sysv/linux/x86_64/ucontext_i.sym (oSSP): New.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h: Always include
<dl-sysdep.h>. Test for value of RTLD_PRIVATE_ERRNO instead of
testing whether it is defined.
Current GLIBC has two ways to implement the single thread optimization
on syscalls to avoid calling the cancellation path: either by using
global variables (__{libc,pthread}_multiple_thread) or by accessing
the TCB field (defined by TLS_MULTIPLE_THREADS_IN_TCB). Both the
variables and the macros to acces its value are defined in the
architecture sysdep-cancel.h header.
This patch consolidates its definition on only one header,
sysdeps/unix/sysv/linux/sysdep-cancel.h, and adds a new define
(SINGLE_THREAD_BY_GLOBAL) which the architecture defines if it prefer
to use the global variables instead of the TCB field. This is an
optimization, so if the architecture does not define it, the TCB
method will be used as default.
Checked on x86_64-linux-gnu and on a build with major touched
ABIs (aarch64-linux-gnu, alpha-linux-gnu, arm-linux-gnueabihf,
hppa-linux-gnu, i686-linux-gnu, m68k-linux-gnu, microblaze-linux-gnu,
mips-linux-gnu, mips64-linux-gnu, powerpc-linux-gnu,
powerpc64le-linux-gnu, s390-linux-gnu, s390x-linux-gnu, sh4-linux-gnu,
sparcv9-linux-gnu, sparc64-linux-gnu, tilegx-linux-gnu).
* sysdeps/unix/sysv/linux/aarch64/sysdep-cancel.h: Remove file.
* sysdeps/unix/sysv/linux/alpha/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/arm/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/hppa/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/mips/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/nios2/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/powerpc/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/sh/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/sparc/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/tile/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/sysdep.h
(SINGLE_THREAD_BY_GLOBAL): Define.
* sysdeps/unix/sysv/linux/aarch64/sysdep.h (SINGLE_THREAD_BY_GLOBAL):
Likewise.
* sysdeps/unix/sysv/linux/alpha/sysdep.h (SINGLE_THREAD_BY_GLOBAL):
Likewise.
* sysdeps/unix/sysv/linux/arm/sysdep.h (SINGLE_THREAD_BY_GLOBAL):
Likewise.
* sysdeps/unix/sysv/linux/hppa/sysdep.h (SINGLE_THREAD_BY_GLOBAL):
Likewise.
* sysdeps/unix/sysv/linux/microblaze/sysdep.h
(SINGLE_THREAD_BY_GLOBAL): Likewise.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h (SINGLE_THREAD_BY_GLOBAL):
Likewise.
The problem for x32 is the {INTERNAL,INLINE}_SYSCALL C macros explicit
cast the arguments to 'long int', thus passing as 32 bits arguments
that should be passed to 64 bits.
Previous x32 implementation uses the auto-generated syscalls from
assembly macros (syscalls.list), so the {INTERNAL,INLINE}_SYSCALL
macros are never used with 64 bit argument in x32 (which are
internally broken for this ILP).
To fix it I used a strategy similar to MIPS64n32 (although both
ABI differs for some syscalls on how top pass 64-bits arguments)
where argument types for kernel call are defined using GCC extension
'typeof' with a arithmetic operation. This allows 64-bits arguments
to be defined while 32-bits argument will still passed as 32-bits.
I also cleanup the {INLINE,INTERNAL}_SYSCALL definition by defining
'inline_syscallX' instead of constructing the argument passing using
macros (it adds some readability) and removed the ununsed
INTERNAL_SYSCALL_NCS_TYPES define (since the patch idea is exactly to
avoid requiric explicit types passing).
Tested on x86_64 and x32.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h
(INTERNAL_SYSCALL_NCS_TYPES): Remove define.
(LOAD_ARGS_0): Likewise.
(LOAD_ARGS_1): Likewise.
(LOAD_ARGS_2): Likewise.
(LOAD_ARGS_3): Likewise.
(LOAD_ARGS_4): Likewise.
(LOAD_ARGS_5): Likewise.
(LOAD_ARGS_6): Likewise.
(LOAD_REGS_0): Likewise.
(LOAD_REGS_1): Likewise.
(LOAD_REGS_2): Likewise.
(LOAD_REGS_3): Likewise.
(LOAD_REGS_4): Likewise.
(LOAD_REGS_5): Likewise.
(LOAD_REGS_6): Likewise.
(ASM_ARGS_0): Likewise.
(ASM_ARGS_1): Likewise.
(ASM_ARGS_2): Likewise.
(ASM_ARGS_3): Likewise.
(ASM_ARGS_4): Likewise.
(ASM_ARGS_5): Likewise.
(ASM_ARGS_6): Likewise.
(LOAD_ARGS_TYPES_1): Likewise.
(LOAD_ARGS_TYPES_2): Likewise.
(LOAD_ARGS_TYPES_3): Likewise.
(LOAD_ARGS_TYPES_4): Likewise.
(LOAD_ARGS_TYPES_5): Likewise.
(LOAD_ARGS_TYPES_6): Likewise.
(LOAD_REGS_TYPES_1): Likewise.
(LOAD_REGS_TYPES_2): Likewise.
(LOAD_REGS_TYPES_3): Likewise.
(LOAD_REGS_TYPES_4): Likewise.
(LOAD_REGS_TYPES_5): Likewise.
(LOAD_REGS_TYPES_6): Likewise.
(TYPEFY): New define.
(ARGIFY): Likewise.
(internal_syscall0): Likewise.
(internal_syscall1): Likewise.
(internal_syscall2): Likewise.
(internal_syscall3): Likewise.
(internal_syscall4): Likewise.
(internal_syscall5): Likewise.
(internal_syscall6): Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/times.c
(INTERNAL_SYSCALL_NCS): Remove define.
(internal_syscall1): Add define.
The kernel interface for p{readv,writev}{64}v is
(unsigned long fd, {const }struct iovec *iov, unsigned long vlen,
unsigned long pos_l, unsigned long pos_h)
Except for targets which define __ARCH_WANT_COMPAT_SYS_PREADV64 and
__ARCH_WANT_COMPAT_SYS_PWRITEV64,
(unsigned long fd, {const }struct iovec *iov, unsigned long vlen,
off64_t pos)
is used for p{readv,writev}{64}v. X32 is the only such target. The
LO_HI_LONG macro is used to pass offset to the pos_l and pos_h pair.
Since pos_h is ignored when size of offset == sizeof of pos_l, x86-64
has
#define LO_HI_LONG(val) (val)
But the kernel interface for p{readv,writev}{64}v2 is
(unsigned long fd, {const }struct iovec *iov, unsigned long vlen,
unsigned long pos_l, unsigned long pos_h, int flags)
Except for targets which define __ARCH_WANT_COMPAT_SYS_PREADV64V2 and
__ARCH_WANT_COMPAT_SYS_PWRITEV64V2,
(unsigned long fd, {const }struct iovec *iov, unsigned long vlen,
off64_t pos, int flags)
is used for p{readv,writev}{64}v2. X32 is the only such target. Update
x86-64 LO_HI_LONG to pass 0 as the high part of the offset argument for
p{readv,writev}{64}v2 and define a different LO_HI_LONG for x32 to only
pass one argument for offset.
Tested on x32 and x86-64.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h (LO_HI_LONG): Pass
0 as the high part of offset.
* sysdeps/unix/sysv/linux/x86_64/x32/sysdep.h (LO_HI_LONG): New.
Define LO_HI_LONG to skip pos_h since it is ignored by kernel:
static inline loff_t pos_from_hilo(unsigned long high, unsigned long low)
{
#define HALF_LONG_BITS (BITS_PER_LONG / 2)
return (((loff_t)high << HALF_LONG_BITS) << HALF_LONG_BITS) | low;
}
where size of loff_t == size of long.
[BZ #20349]
* sysdeps/unix/sysv/linux/x86_64/sysdep.h (LO_HI_LONG): New.
X86-64 system calls use a different calling convention, which clobbers
CC, %r11 an %rcx registers. Define REGISTERS_CLOBBERED_BY_SYSCALL for
x86-64 inline asm statements.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h
(REGISTERS_CLOBBERED_BY_SYSCALL): New.
(INTERNAL_SYSCALL_NCS): Use it.
(INTERNAL_SYSCALL_NCS_TYPES): Likewise.
This patch consolidates the sched_getcpu implementations across all
arches (except tile, which requires its own). This patch removes
the powerpc, x86_64 and x32 specific files and change the default
linux one to use INLINE_VSYSCALL where possible (for ports that
implements it).
This patch consolidate the Linux vDSO define and usage across all ports
that uses it. The common vDSO definitions and calling through
{INLINE/INTERNAL}_VSYSCALL macros are moved to a common header
sysdep-vdso.h and vDSO name declaration and prototype is defined
using a common macro.
Also PTR_{MANGLE,DEMANGLE} is added to ports that does not use them
for vDSO calls (aarch64, powerpc, s390, and tile) and thus it will
reflect in code changes. For ports that already implement pointer
mangling/demangling in vDSO system (i386, x32, x86_64) this patch
is mainly a code refactor.
Checked on x32, x86_64, x32, ppc64le, and aarch64.
(_libc_vdso_platform_setup): Mangle function pointers before storing
them.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h (INLINE_VSYSCALL):
Demangle vdso pointer before use.
(INTERNAL_VSYSCALL): Likewise.
* elf/cache.c (primes): Mark as const.
Noted by Roland McGrath.
returning.
(PTR_DEMANGLE): Real definition now that it's not the same as
PRT_MANGLE anymore.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h: Likewise.
* string/strerror_l.c: New file.
* string/Makefile (routines): Add strerror_l.
* string/string.h: Declare strerror_l.
* string/Versions: Export strerror_l for GLIBC_2.6.
* nscd/connections.c (nscd_run): Don't define MAXKEYLEN here.
* elf/rtld.c: Define __pointer_chk_guard_local and if necessary
__pointer_chk_guard.
(_rtld_global_ro): Initialize _dl_pointer_guard.
(dl_main): Initialize __pointer_chk_guard_local and either
__pointer_chk_guard or TLS value if necessary.
(process_envvars): Recognize and handle LD_POINTER_GUARD.
* sysdeps/generic/ldsodefs.h (rtld_global_ro): Add _dl_pointer_guard.
* sysdeps/i386/__longjmp.S: Use PTR_DEMANGLE for PC if defined.
* sysdeps/x86_64/__longjmp.S: Likewise.
* sysdeps/i386/bsd-_setjmp.S: Use PTR_MANGLE for PC if defined.
* sysdeps/i386/bsd-_setjmp.S: Likewise.
* sysdeps/i386/setjmp.S: Likewise.
[IS_IN_rtld]: Avoid call to __sigjmp_save.
* sysdeps/i386/setjmp.S: Likewise.
* sysdeps/unix/sysv/linux/i386/sysdep.h: Define PTR_MANGLE and
PTR_DEMANGLE.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h: Likewise.
* sysdeps/i386/elf/setjmp.S: Removed.
* sysdeps/i386/elf/bsd-setjmp.S: Removed.
2003-08-15 Ulrich Drepper <drepper@redhat.com>
* io/Versions [GLIBC_2.3.3]: Add posix_fadvise64.
* io/fcntl.h (posix_fadvise64): Change type of third parameter to
off_t.
* sysdeps/unix/sysv/linux/posix_fadvise.c: Adjust definition.
* sysdeps/unix/make-syscalls.sh: Recognize V prefix to the parameter
description indicating the error value is returned, not -1.
* sysdeps/unix/sysv/linux/Makefile [subdir=io] (subdir_routines): Add
posix_fadvise64_64.
* sysdeps/unix/sysv/linux/kernel-features.h: Add definition of
__ASSUME_FADVISE64_64_SYSCALL.
* sysdeps/unix/sysv/linux/syscalls.list: Don't define madvise and
posix_madvise in one file. Have posix_madvise defined with error
value returned.
Define posix_fadvise64_64 entry. Add version info to posix_fadvise64
entry.
* sysdeps/unix/sysv/linux/alpha/syscalls.list: Define
posix_fadvise64_64 entry. Add version info to posix_fadvise64 entry.
* sysdeps/unix/sysv/linux/ia64/syscalls.list: Likewise.
* sysdeps/unix/sysv/linux/x86_64/syscalls.list: Likewise.
* sysdeps/unix/sysv/linux/i386/posix_fadvise.S: New file.
* sysdeps/unix/sysv/linux/i386/sysdep.h: Define PSEUDO_ERRVAL,
PSEUDO_END_ERRVAL, and ret_ERRVAL.
* sysdeps/unix/sysv/linux/ia64/sysdep.h: Likewise.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h: Likewise.
* posix/Makefile (routines): Add posix_madvise.
* sysdeps/generic/madvise.c: Don't define posix_madvise.
* sysdeps/generic/posix_madvise.c: New file.
* sysdeps/unix/sysv/aix/posix_madvise.c: New file.
