Linux from 3.9 through 4.2 does not abort HTM transaction on syscalls,
instead it suspend and resume it when leaving the kernel. The
side-effects of the syscall will always remain visible, even if the
transaction is aborted. This is an issue when transaction is used along
with futex syscall, on pthread_cond_wait for instance, where the futex
call might succeed but the transaction is rolled back leading the
pthread_cond object in an inconsistent state.
Glibc used to prevent it by always aborting a transaction before issuing
a syscall. Linux 4.2 also decided to abort active transaction in
syscalls which makes the glibc workaround superfluous. Worse, glibc
transaction abortion leads to a performance issue on recent kernels
where the HTM state is saved/restore lazily (v4.9). By aborting a
transaction on every syscalls, regardless whether a transaction has being
initiated before, GLIBS makes the kernel always save/restore HTM state
(it can not even lazily disable it after a certain number of syscall
iterations).
Because of this shortcoming, Transactional Lock Elision is just enabled
when it has been explicitly set (either by tunables of by a configure
switch) and if kernel aborts HTM transactions on syscalls
(PPC_FEATURE2_HTM_NOSC). It is reported that using simple benchmark [1],
the context-switch is about 5% faster by not issuing a tabort in every
syscall in newer kernels.
Checked on powerpc64le-linux-gnu with 4.4.0 kernel (Ubuntu 16.04).
* NEWS: Add note about new TLE support on powerpc64le.
* sysdeps/powerpc/nptl/tcb-offsets.sym (TM_CAPABLE): Remove.
* sysdeps/powerpc/nptl/tls.h (tcbhead_t): Rename tm_capable to
__ununsed1.
(TLS_INIT_TP, TLS_DEFINE_INIT_TP): Remove tm_capable setup.
(THREAD_GET_TM_CAPABLE, THREAD_SET_TM_CAPABLE): Remove macros.
* sysdeps/powerpc/powerpc32/sysdep.h,
sysdeps/powerpc/powerpc64/sysdep.h (ABORT_TRANSACTION_IMPL,
ABORT_TRANSACTION): Remove macros.
* sysdeps/powerpc/sysdep.h (ABORT_TRANSACTION): Likewise.
* sysdeps/unix/sysv/linux/powerpc/elision-conf.c (elision_init): Set
__pthread_force_elision iff PPC_FEATURE2_HTM_NOSC is set.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/sysdep.h,
sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h
sysdeps/unix/sysv/linux/powerpc/syscall.S (ABORT_TRANSACTION): Remove
usage.
* sysdeps/unix/sysv/linux/powerpc/not-errno.h: Remove file.
Reported-by: Breno Leitão <leitao@debian.org>
The tunables framework needs to execute syscall early in process
initialization, before the TCB is available for consumption. This
behavior conflicts with powerpc{|64|64le}'s lock elision code, that
checks the TCB before trying to abort transactions immediately before
executing a syscall.
This patch adds a powerpc-specific implementation of __access_noerrno
that does not abort transactions before the executing syscall.
Tested on powerpc{|64|64le}.
[BZ #22685]
* sysdeps/powerpc/powerpc32/sysdep.h (ABORT_TRANSACTION_IMPL): Renamed
from ABORT_TRANSACTION.
(ABORT_TRANSACTION): Redirect to ABORT_TRANSACTION_IMPL.
* sysdeps/powerpc/powerpc64/sysdep.h (ABORT_TRANSACTION,
ABORT_TRANSACTION_IMPL): Likewise.
* sysdeps/unix/sysv/linux/powerpc/not-errno.h: New file. Reuse
Linux code, but remove the code that aborts transactions.
Signed-off-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
Tested-by: Aurelien Jarno <aurelien@aurel32.net>
This patch adds several new tunables to control the behavior of
elision on supported platforms[1]. Since elision now depends
on tunables, we should always *compile* with elision enabled,
and leave the code disabled, but available for runtime
selection. This gives us *much* better compile-time testing of
the existing code to avoid bit-rot[2].
Tested on ppc, ppc64, ppc64le, s390x and x86_64.
[1] This part of the patch was initially proposed by
Paul Murphy but was "staled" because the framework have changed
since the patch was originally proposed:
https://patchwork.sourceware.org/patch/10342/
[2] This part of the patch was inititally proposed as a RFC by
Carlos O'Donnell. Make sense to me integrate this on the patch:
https://sourceware.org/ml/libc-alpha/2017-05/msg00335.html
* elf/dl-tunables.list: Add elision parameters.
* manual/tunables.texi: Add entries about elision tunable.
* sysdeps/unix/sysv/linux/powerpc/elision-conf.c:
Add callback functions to dynamically enable/disable elision.
Add multiple callbacks functions to set elision parameters.
Deleted __libc_enable_secure check.
* sysdeps/unix/sysv/linux/s390/elision-conf.c: Likewise.
* sysdeps/unix/sysv/linux/x86/elision-conf.c: Likewise.
* configure: Regenerated.
* configure.ac: Option enable_lock_elision was deleted.
* config.h.in: ENABLE_LOCK_ELISION flag was deleted.
* config.make.in: Remove references to enable_lock_elision.
* manual/install.texi: Elision configure option was removed.
* INSTALL: Regenerated to remove enable_lock_elision.
* nptl/Makefile:
Disable elision so it can verify error case for destroying a mutex.
* sysdeps/powerpc/nptl/elide.h:
Cleanup ENABLE_LOCK_ELISION check.
Deleted macros for the case when ENABLE_LOCK_ELISION was not defined.
* sysdeps/s390/configure: Regenerated.
* sysdeps/s390/configure.ac: Remove references to enable_lock_elision..
* nptl/tst-mutex8.c:
Deleted all #ifndef ENABLE_LOCK_ELISION from the test.
* sysdeps/powerpc/powerpc32/sysdep.h:
Deleted all ENABLE_LOCK_ELISION checks.
* sysdeps/powerpc/powerpc64/sysdep.h: Likewise.
* sysdeps/powerpc/sysdep.h: Likewise.
* sysdeps/s390/nptl/bits/pthreadtypes-arch.h: Likewise.
* sysdeps/unix/sysv/linux/powerpc/force-elision.h: Likewise.
* sysdeps/unix/sysv/linux/s390/elision-conf.h: Likewise.
* sysdeps/unix/sysv/linux/s390/force-elision.h: Likewise.
* sysdeps/unix/sysv/linux/s390/lowlevellock.h: Likewise.
* sysdeps/unix/sysv/linux/s390/Makefile: Remove references to
enable-lock-elision.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
.align on some targets takes a byte alignment, on others like powerpc,
log2 of the byte alignment. It's a good idea to avoid .align,
particularly since x86 and powerpc are different. This patch fixes
the occurrences of .align in powerpc64/sysdep.h, renames DOT_LABEL
since the macro doesn't have anything to do with adding dots, removes
extraneous semicolons, and fixes some formatting.
* sysdeps/powerpc/powerpc64/sysdep.h: Formatting.
(FUNC_LABEL): Rename from DOT_LABEL.
(ENTRY_1): Use FUNC_LABEL and remove leading space from label.
Use .p2align rather than .align.
(TRACEBACK, TRACEBACK_MASK): Use .p2align rather than .align.
(ABORT_TRANSACTION): Likewise.
(ENTRY_1, ENTRY_2, END_2, LOCALENTRY): Remove unnecessary semicolons,
particularly at end. Add semicolon at invocation as necessary.
(TRACEBACK, TRACEBACK_MASK, PSEUDO, PSEUDO_NOERRNO): Likewise.
(PSEUDO_ERRVAL, PPC64_LOAD_FUNCPTR, OPD_ENT): Likewise.
* sysdeps/powerpc/powerpc64/multiarch/strrchr-power8.S (ENTRY,
END): Adjust to suit.
I think FRAME_PARM[1-9]_SAVE confuse the code, particularly
FRAME_PARM9_SAVE. There are only 8 parameter save slots!
* sysdeps/powerpc/powerpc64/sysdep.h: (FRAME_BACKCHAIN,
FRAME_CR_SAVE, FRAME_LR_SAVE): Move out of conditional.
(FRAME_PARM1_SAVE, FRAME_PARM2_SAVE, FRAME_PARM3_SAVE,
FRAME_PARM4_SAVE, FRAME_PARM5_SAVE, FRAME_PARM6_SAVE,
FRAME_PARM7_SAVE, FRAME_PARM8_SAVE, FRAME_PARM9_SAVE): Delete.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/makecontext.S: Replace
uses of FRAME_PARM[1-9]_SAVE with FRAME_PARM_SAVE plus offset.
The macros used in assembly were broken on powerpc64 ELFv1.
* sysdeps/powerpc/powerpc64/sysdep.h: (call_mcount_parm_offset): Delete.
(SAVE_ARG, REST_ARG, CFI_SAVE_ARG): Correct.
Fix usage of tabort in generated syscalls. r0 has special meaning
when used with this instruction, thus it will not generate
persistent errors, nor return an error code. This mitigates poor
CPU usage when performing elided critical sections.
Additionally, transactions should be aborted when entering a user
invoked syscall. Otherwise the results of the transaction may be
undefined.
2015-08-25 Paul E. Murphy <murphyp@linux.vnet.ibm.com>
* sysdeps/powerpc/powerpc32/sysdep.h (ABORT_TRANSACTION): Use
register other than r0 for tabort, it has special meaning.
* sysdeps/powerpc/powerpc64/sysdep.h (ABORT_TRANSACTION): Likewise
* sysdeps/unix.sysv/linux/powerpc/syscall.S (syscall): Abort
transaction before starting syscall.
With AIX port deprecated there is no need to check/define
HAVE_ASM_GLOBAL_DOT_NAME anymore since the current minimum binutils
supported (2.22) does not emit global symbol with dot.
This patch removes all the HAVE_ASM_GLOBAL_DOT_NAME definition and
checks for powerpc64 port.
Linux kernel powerpc documentation states issuing a syscall inside a
transaction is not recommended and may lead to undefined behavior. It
also states syscalls does not abort transactoin neither they run in
transactional state.
To avoid side-effects being visible outside transactions, GLIBC with
lock elision enabled will issue a transaction abort instruction just
before all syscalls if hardware supports hardware transactions.
This updates glibc for the changes in the ELFv2 relating to the
stack frame layout. These are described in more detail here:
http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01149.htmlhttp://gcc.gnu.org/ml/gcc-patches/2013-11/msg01146.html
Specifically, the "compiler and linker doublewords" were removed,
which has the effect that the save slot for the TOC register is
now at offset 24 rather than 40 to the stack pointer.
In addition, a function may now no longer necessarily assume that
its caller has set up a 64-byte register save area its use.
To address the first change, the patch goes through all assembler
files and replaces immediate offsets in instructions accessing the
ABI-defined stack slots by symbolic offsets. Those already were
defined in ucontext_i.sym and used in some of the context routines,
but that doesn't really seem like the right place for those defines.
The patch instead defines those symbolic offsets in sysdeps.h,
in two variants for the old and new ABI, and uses them systematically
in all assembler files, not just the context routines.
The second change only affected a few assembler files that used
the save area to temporarily store some registers. In those
cases where this happens within a leaf function, this patch
changes the code to store those registers to the "red zone"
below the stack pointer. Otherwise, the functions already allocate
a stack frame, and the patch changes them to add extra space in
these frames as temporary space for the ELFv2 ABI.
This patch adds support for the ELFv2 ABI feature to remove function
descriptors. See this GCC patch for in-depth discussion:
http://gcc.gnu.org/ml/gcc-patches/2013-11/msg01141.html
This mostly involves two types of changes: updating assembler source
files to the new logic, and updating the dynamic loader.
After the refactoring in the previous patch, most of the assembler source
changes can be handled simply by providing ELFv2 versions of the
macros in sysdep.h. One somewhat non-obvious change is in __GI__setjmp:
this used to "fall through" to the immediately following __setjmp ENTRY
point. This is no longer safe in the ELFv2 since ENTRY defines both
a global and a local entry point, and you cannot simply fall through
to a global entry point as it requires r12 to be set up.
Also, makecontext needs to be updated to set up registers according to
the new ABI for calling into the context's start routine.
The dynamic linker changes mostly consist of removing special code
to handle function descriptors. We also need to support the new PLT
and glink format used by the the ELFv2 linker, see:
https://sourceware.org/ml/binutils/2013-10/msg00376.html
In addition, the dynamic linker now verifies that the dynamic libraries
it loads match its own ABI.
The hack in VDSO_IFUNC_RET to "synthesize" a function descriptor
for vDSO routines is also no longer necessary for ELFv2.
This is the first patch to support the new ELFv2 ABI in glibc.
As preparation, this patch simply refactors some of the powerpc64 assembler
code to move all code related to creating function descriptors (.opd section)
or using function descriptors (function pointer call) into a central place
in sysdep.h.
Note that most locations creating .opd entries were already using macros
in sysdep.h, this patch simply extends this to the remaining places.
No relevant change in generated code expected.
directive to ENTRY macros.
(END): Add cfi_endproc directive to END macro.
* sysdeps/powerpc/powerpc64/sysdep.h (ENTRY, EALIGN): Add cfi_startproc
directive to ENTRY macros.
(END, END_GEN_TB): Add cfi_endproc directive to END macros.
* sysdeps/powerpc/powerpc32/dl-start.S: Remove ENTRY()s for
_dl_start_user and _dl_main_dispatch.
* sysdeps/powerpc/powerpc32/fpu/fprrest.S: Use END macro.
* sysdeps/powerpc/powerpc32/fpu/fprsave.S: Use Likewise.
* sysdeps/powerpc/powerpc32/gprrest0.S: Likewise.
* sysdeps/powerpc/powerpc32/gprrest1.S: Likewise.
* sysdeps/powerpc/powerpc32/gprsave0.S: Likewise.
* sysdeps/powerpc/powerpc32/gprsave1.S: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/socket.S: Remove redundant
cfi_startproc and cfi_endproc.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/socket.S: Likewise.
2006-01-04 Ulrich Drepper <drepper@redhat.com>
* sysdeps/pthread/aio_misc.h (struct waitlist): Add result member.
* sysdeps/pthread/aio_notify.c (__aio_notify): For synchronous waiters,
failed I/O, and a result pointer, set value to -1.
* sysdeps/pthread/aio_suspend.c: Initialize result pointer to NULL.
* sysdeps/pthread/lio_listio.c: For LIO_WAIT, point result pointer in
wait list to local variable result. If nonzero afterwards, set errno
to EIO. For LIO_NOWAIT set result to NULL.
* rt/Makefile (tests): Add tst-aio8.
* rt/tst-aio8.c: New file.
* sysdeps/posix/getaddrinfo.c (gaih_inet): Set no_inet6_date alse
if only PF_INET address is needed.
* nss/getXXbyYY_r.c: Make sure we always return a nonzero value in
case of an error.
2003-01-11 Jim Meyering <jim@meyering.net>
* io/ftw.c [HAVE_CONFIG_H]: Include <config.h>.
[HAVE_SYS_PARAM_H || _LIBC]: Guard inclusion of <sys/param.h>.
Include <sys/stat.h>, not <include/sys/stat.h>, if !_LIBC.
[!_LIBC] (__chdir, __closedir, __fchdir, __getcwd, __opendir): Define.
[!_LIBC] (__readdir64, __tdestroy, __tfind, __tsearch): Define.
[!_LIBC] (internal_function, dirent64, MAX): Define.
(__set_errno): Define if not already defined.
(open_dir_stream): When FTW_CHDIR is enabled, invoke opendir on
the basename, not the entire file name.
(process_entry): When FTW_CHDIR is enabled, invoke XSTAT or LXSTAT on
the basename, not the entire file name.
2003-01-12 Ulrich Drepper <drepper@redhat.com>
* string/tester.c (test_strcpy): Disable last added strcpy until
it is fixed.
2003-01-11 Philip Blundell <philb@gnu.org>
* sysdeps/unix/sysv/linux/arm/socket.S: Add cancellation support.
2003-01-11 Andreas Schwab <schwab@suse.de>
* Makerules: Add vpath for %.dynsym and %.so so that the
implicit rule chaining for check-abi works.
2003-01-11 Kaz Kojima <kkojima@rr.iij4u.or.jp>
* sysdeps/unix/sysv/linux/sh/sysdep.h (SYSCALL_ERROR_HANDLER):
Add non-PIC case.
2003-01-11 Jakub Jelinek <jakub@redhat.com>
* elf/tls-macros.h [__ia64__] (__TLS_CALL_CLOBBERS): Define.
[__ia64__] (TLS_LE, TLS_IE): Fix typos. Add ;; at start of asm if
gp is used early.
[__ia64__] (TLS_LD, TLS_GD): Likewise. Use __TLS_CALL_CLOBBERS.
* elf/Makefile ($(objpfx)tst-tlsmod5.so, $(objpfx)tst-tlsmod6.so):
Ensure libc.so in DT_NEEDED.
* sysdeps/alpha/dl-machine.h (elf_machine_rela): Move
CHECK_STATIC_TLS before l_tls_offset use.
* sysdeps/i386/dl-machine.h (elf_machine_rel, elf_machine_rela):
Likewise.
* sysdeps/sh/dl-machine.h (elf_machine_rela): Likewise.
* sysdeps/generic/dl-tls.c (_dl_allocate_tls_storage) [TLS_DTV_AT_TP]:
Allocate TLS_PRE_TCB_SIZE bytes below result.
(_dl_deallocate_tls) [TLS_DTV_AT_TP]: Adjust before freeing.
* sysdeps/generic/libc-tls.c (__libc_setup_tls): If
TLS_INIT_TP_EXPENSIVE is not defined, allocate even if no PT_TLS
segment has been found. If TLS_DTV_AT_TP, allocate TLS_PRE_TCB_SIZE
bytes below result and add tcb_offset to memsz.
* sysdeps/ia64/dl-tls.h (__tls_get_addr): New prototype.
* sysdeps/ia64/dl-machine.h: Include tls.h.
(elf_machine_type_class): Return ELF_RTYPE_CLASS_PLT for TLS relocs
too.
(elf_machine_rela): Assume if sym_map != NULL sym is non-NULL too.
Handle R_IA64_DTPMOD*, R_IA64_DTPREL* and R_IA64_TPREL* relocations.
* sysdeps/ia64/libc-tls.c: New file.
2003-01-10 Steven Munroe <sjmunroe@us.ibm.com>
* sysdeps/powerpc/powerpc64/sysdep.h (PSEUDO_RET): Add branch hit.
* sysdeps/unix/sysv/linux/powerpc/bits/stat.h (STAT_VER_LINUX):
Fix type. Move definition out of #if.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/ftruncate64.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/pread.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/pread64.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/pwrite.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/pwrite64.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/socket.S: Add cancellation
support.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/syscalls.list: Remove
ftruncate64, pread64, pwrite64, truncate64 entries.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h
(INLINE_SYSCALL): New version that supports function call like
syscalls. Add __builtin_expect.
(LOADARGS_n): Add argument size safety checks.
(INTERNAL_SYSCALL): New Macro.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/truncate64.c: New file.
* sysdeps/unix/sysv/linux/powerpc/sys/procfs.h [__PPC_ELF_H]: Avoid
redefinition of elf_fpreg_t and elf_fpregset_t.
2003-01-12 Ulrich Drepper <drepper@redhat.com>
* elf/dl-close.c (_dl_close): Add several asserts. Correct and
simplify test for unloading. If loader of a DSO is unloaded do not
use its scope anymore. Fall back to own scope and adjust opencounts.
Fix several comments.
* elf/dl-deps.c (_dl_map_object_deps): Always allocate memory for
the l_searchlist, not only for l_initfini.
* elf/dl-lookup.c (add_dependencies): Avoid creating relocation
dependencies if objects cannot be removed. Remove object with the
definition as not unloadable if necessary.
* elf/reldep6.c: Create relocation dependency before closing the first
module.
2003-01-10 Guido Gnther <agx@sigxcpu.org>
* elf/Makefile: Add rules to build and run reldep9 test.
* elf/reldep9.c: New file.
* elf/reldep9mod1.c: New file.
* elf/reldep9mod2.c: New file.
* elf/reldep9mod3.c: New file.
2003-01-09 Jakub Jelinek <jakub@redhat.com>
* elf/Makefile: Add rules to build and run nodelete2 test.
* elf/nodelete2.c: New file.
* elf/nodel2mod1.c: New file.
* elf/nodel2mod2.c: New file.
* elf/nodel2mod3.c: New file.
2003-01-09 Jakub Jelinek <jakub@redhat.com>
* sysdeps/powerpc/powerpc32/sysdep.h: New file. Define powerpc32
specific macros copied from sysdeps/powerpc/sysdep.h.
* sysdeps/powerpc/powerpc64/sysdep.h: New file. Define powerpc64
specific macros.
* sysdeps/unix/powerpc/sysdep.h: Include <bits/wordsize.h>.
[__WORDSIZE==64]: Include sysdeps/powerpc/powerpc64/sysdep.h.
[!__WORDSIZE==64]: Include sysdeps/powerpc/powerpc32/sysdep.h.
