feature_1 has X86_FEATURE_1_IBT and X86_FEATURE_1_SHSTK bits for CET
run-time control.
CET_ENABLED, IBT_ENABLED and SHSTK_ENABLED are defined to 1 or 0 to
indicate that if CET, IBT and SHSTK are enabled.
<tls-setup.h> is added to set up thread-local data.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
[BZ #22563]
* nptl/pthread_create.c: Include <tls-setup.h>.
(__pthread_create_2_1): Call tls_setup_tcbhead.
* sysdeps/generic/tls-setup.h: New file.
* sysdeps/x86/nptl/tls-setup.h: Likewise.
* sysdeps/i386/nptl/tcb-offsets.sym (FEATURE_1_OFFSET): New.
* sysdeps/x86_64/nptl/tcb-offsets.sym (FEATURE_1_OFFSET):
Likewise.
* sysdeps/i386/nptl/tls.h (tcbhead_t): Rename __glibc_reserved1
to feature_1.
* sysdeps/x86_64/nptl/tls.h (tcbhead_t): Likewise.
* sysdeps/x86/sysdep.h (X86_FEATURE_1_IBT): New.
(X86_FEATURE_1_SHSTK): Likewise.
(CET_ENABLED): Likewise.
(IBT_ENABLED): Likewise.
(SHSTK_ENABLED): Likewise.
The pad array in struct pthread_unwind_buf is used by setjmp to save
shadow stack register. We assert that size of struct pthread_unwind_buf
is no less than offset of shadow stack pointer + shadow stack pointer
size.
Since functions, like LIBC_START_MAIN, START_THREAD_DEFN as well as
these with thread cancellation, call setjmp, but never return after
__libc_unwind_longjmp, __libc_unwind_longjmp, which is defined as
__libc_longjmp on x86, doesn't need to restore shadow stack register.
__libc_longjmp, which is a private interface for thread cancellation
implementation in libpthread, is changed to call __longjmp_cancel,
instead of __longjmp. __longjmp_cancel is a new internal function
in libc, which is similar to __longjmp, but doesn't restore shadow
stack register.
The compatibility longjmp and siglongjmp in libpthread.so are changed
to call __libc_siglongjmp, instead of __libc_longjmp, so that they will
restore shadow stack register.
Tested with build-many-glibcs.py.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* nptl/pthread_create.c (START_THREAD_DEFN): Clear previous
handlers after setjmp.
* setjmp/longjmp.c (__libc_longjmp): Don't define alias if
defined.
* sysdeps/unix/sysv/linux/x86/setjmpP.h: Include
<libc-pointer-arith.h>.
(_JUMP_BUF_SIGSET_BITS_PER_WORD): New.
(_JUMP_BUF_SIGSET_NSIG): Changed to 96.
(_JUMP_BUF_SIGSET_NWORDS): Changed to use ALIGN_UP and
_JUMP_BUF_SIGSET_BITS_PER_WORD.
* sysdeps/x86/Makefile (sysdep_routines): Add __longjmp_cancel.
* sysdeps/x86/__longjmp_cancel.S: New file.
* sysdeps/x86/longjmp.c: Likewise.
* sysdeps/x86/nptl/pt-longjmp.c: Likewise.
This patch adds two new internal defines to set the internal
pthread_mutex_t layout required by the supported ABIS:
1. __PTHREAD_MUTEX_NUSERS_AFTER_KIND which control whether to define
__nusers fields before or after __kind. The preferred value for
is 0 for new ports and it sets __nusers before __kind.
2. __PTHREAD_MUTEX_USE_UNION which control whether internal __spins and
__list members will be place inside an union for linuxthreads
compatibility. The preferred value is 0 for ports and it sets
to not use an union to define both fields.
It fixes the wrong offsets value for __kind value on x86_64-linux-gnu-x32.
Checked with a make check run-built-tests=no on all afected ABIs.
[BZ #22298]
* nptl/allocatestack.c (allocate_stack): Check if
__PTHREAD_MUTEX_HAVE_PREV is non-zero, instead if
__PTHREAD_MUTEX_HAVE_PREV is defined.
* nptl/descr.h (pthread): Likewise.
* nptl/nptl-init.c (__pthread_initialize_minimal_internal):
Likewise.
* nptl/pthread_create.c (START_THREAD_DEFN): Likewise.
* sysdeps/nptl/fork.c (__libc_fork): Likewise.
* sysdeps/nptl/pthread.h (PTHREAD_MUTEX_INITIALIZER): Likewise.
* sysdeps/nptl/bits/thread-shared-types.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION): New
defines.
(__pthread_internal_list): Check __PTHREAD_MUTEX_USE_UNION instead
of __WORDSIZE for internal layout.
(__pthread_mutex_s): Check __PTHREAD_MUTEX_NUSERS_AFTER_KIND instead
of __WORDSIZE for internal __nusers layout and __PTHREAD_MUTEX_USE_UNION
instead of __WORDSIZE whether to use an union for __spins and __list
fields.
(__PTHREAD_MUTEX_HAVE_PREV): Define also for __PTHREAD_MUTEX_USE_UNION
case.
* sysdeps/aarch64/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION): New
defines.
* sysdeps/alpha/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/arm/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/hppa/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/ia64/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/m68k/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/microblaze/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/mips/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/nios2/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/powerpc/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/s390/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/sh/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/sparc/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/tile/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/x86/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch removes all the replicated pthread definition accross the
architectures and consolidates it on shared headers. The new
organization is as follow:
* Architecture specific definition (such as pthread types sizes) are
place in the new pthreadtypes-arch.h header in arch specific path.
* All shared structure definition are moved to a common NPTL header
at sysdeps/nptl/bits/pthreadtypes.h (with now includes the arch
specific one for internal definitions).
* Also, for C11 future thread support, both mutex and condition
definition are placed in a common header at
sysdeps/nptl/bits/thread-shared-types.h.
It is also a refactor patch without expected functional changes.
Checked with a build for all major ABI (aarch64-linux-gnu, alpha-linux-gnu,
arm-linux-gnueabi, i386-linux-gnu, ia64-linux-gnu,
m68k-linux-gnu, microblaze-linux-gnu, mips{64}-linux-gnu, nios2-linux-gnu,
powerpc{64le}-linux-gnu, s390{x}-linux-gnu, sparc{64}-linux-gnu,
tile{pro,gx}-linux-gnu, and x86_64-linux-gnu).
* posix/Makefile (headers): Add pthreadtypes-arch.h and
thread-shared-types.h.
* sysdeps/aarch64/nptl/bits/pthreadtypes-arch.h: New file: arch
specific thread definition.
* sysdeps/alpha/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/arm/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/hppa/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/ia64/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/m68k/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/microblaze/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/mips/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/nios2/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/powerpc/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/s390/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/sh/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/sparc/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/tile/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/x86/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/nptl/bits/thread-shared-types.h: New file: shared
thread definition between POSIX and C11.
* sysdeps/aarch64/nptl/bits/pthreadtypes.h.: Remove file.
* sysdeps/alpha/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/arm/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/hppa/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/m68k/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/microblaze/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/mips/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/nios2/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/ia64/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/powerpc/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/s390/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/sh/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/sparc/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/tile/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/x86/nptl/bits/pthreadtypes.h: Likewise.
* sysdeps/nptl/bits/pthreadtypes.h: New file: common thread
definitions shared across all architectures.
This patch moves all arch specific pthreadtypes.h to a similar path
for all architectures (sysdeps/unix/sysv/<arch>/bits). No functional
or build change is expected. The idea is mainly to organize the
header placement for all architectures.
Checked with a build for all major ABI (aarch64-linux-gnu, alpha-linux-gnu,
arm-linux-gnueabi, i386-linux-gnu, ia64-linux-gnu,
m68k-linux-gnu, microblaze-linux-gnu [1], mips{64}-linux-gnu, nios2-linux-gnu,
powerpc{64le}-linux-gnu, s390{x}-linux-gnu, sparc{64}-linux-gnu,
tile{pro,gx}-linux-gnu, and x86_64-linux-gnu).
* sysdeps/unix/sysv/linux/x86/Implies: New file.
* sysdeps/unix/sysv/linux/alpha/bits/pthreadtypes.h: Move to ...
* sysdeps/alpha/nptl/bits/pthreadtypes.h: ... here.
* sysdeps/unix/sysv/linux/powerpc/bits/pthreadtypes.h: Move to ...
* sysdeps/powerpc/nptl/bits/pthreadtypes.h: ... here.
* sysdeps/x86/bits/pthreadtypes.h: Move to ...
* sysdeps/x86/nptl/bits/pthreadtypes.h: ... here.
The first argument of elision_adapt and that of ELISION_*LOCK have
different signs since __elision_rwcount is signed char * and the
argument of elision_adapt is uint8_t *. Modified elision_adapt to
accept signed char * instead of uint8_t *.
Since long is 4 bytes for x32, we should use 3 bytes for __pad1 when
a long __pad1 is replaced by a byte __rwelision and __pad1.
* sysdeps/x86/nptl/bits/pthreadtypes.h (pthread_rwlock_t): Use
3 bytes for __pad1 for x32.
(__PTHREAD_RWLOCK_ELISION_EXTRA): Likewise.
This patch relies on the C version of the rwlocks posted earlier.
With C rwlocks it is very straight forward to do adaptive elision
using TSX. It is based on the infrastructure added earlier
for mutexes, but uses its own elision macros. The macros
are fairly general purpose and could be used for other
elision purposes too.
This version is much cleaner than the earlier assembler based
version, and in particular implements adaptation which makes
it safer.
I changed the behavior slightly to not require any changes
in the test suite and fully conform to all expected
behaviors (generally at the cost of not eliding in
various situations). In particular this means the timedlock
variants are not elided. Nested trylock aborts.
