glibc/sysdeps/unix/sysv/linux/s390/elision-trylock.c

99 lines
3.9 KiB
C
Raw Normal View History

/* Elided pthread mutex trylock.
Copyright (C) 2014-2021 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
Prefer https to http for gnu.org and fsf.org URLs Also, change sources.redhat.com to sourceware.org. This patch was automatically generated by running the following shell script, which uses GNU sed, and which avoids modifying files imported from upstream: sed -ri ' s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g ' \ $(find $(git ls-files) -prune -type f \ ! -name '*.po' \ ! -name 'ChangeLog*' \ ! -path COPYING ! -path COPYING.LIB \ ! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \ ! -path manual/texinfo.tex ! -path scripts/config.guess \ ! -path scripts/config.sub ! -path scripts/install-sh \ ! -path scripts/mkinstalldirs ! -path scripts/move-if-change \ ! -path INSTALL ! -path locale/programs/charmap-kw.h \ ! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \ ! '(' -name configure \ -execdir test -f configure.ac -o -f configure.in ';' ')' \ ! '(' -name preconfigure \ -execdir test -f preconfigure.ac ';' ')' \ -print) and then by running 'make dist-prepare' to regenerate files built from the altered files, and then executing the following to cleanup: chmod a+x sysdeps/unix/sysv/linux/riscv/configure # Omit irrelevant whitespace and comment-only changes, # perhaps from a slightly-different Autoconf version. git checkout -f \ sysdeps/csky/configure \ sysdeps/hppa/configure \ sysdeps/riscv/configure \ sysdeps/unix/sysv/linux/csky/configure # Omit changes that caused a pre-commit check to fail like this: # remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines git checkout -f \ sysdeps/powerpc/powerpc64/ppc-mcount.S \ sysdeps/unix/sysv/linux/s390/s390-64/syscall.S # Omit change that caused a pre-commit check to fail like this: # remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S
2019-09-07 05:40:42 +00:00
<https://www.gnu.org/licenses/>. */
#include <pthread.h>
#include <pthreadP.h>
#include <lowlevellock.h>
S390: Use own tbegin macro instead of __builtin_tbegin. This patch defines __libc_tbegin, __libc_tend, __libc_tabort and __libc_tx_nesting_depth in htm.h which replaces the direct usage of equivalent gcc builtins. We have to use an own inline assembly instead of __builtin_tbegin, as tbegin has to filter program interruptions which can't be done with the builtin. Before this change, e.g. a segmentation fault within a transaction, leads to a coredump where the instruction pointer points behind the tbegin instruction instead of real failing one. Now the transaction aborts and the code should be reexecuted by the fallback path without transactions. The segmentation fault will produce a coredump with the real failing instruction. The fpc is not saved before starting the transaction. If e.g. the rounging mode is changed and the transaction is aborting afterwards, the builtin will not restore the fpc. This is now done with the __libc_tbegin macro. Now the call saved fprs have to be saved / restored in the __libc_tbegin macro. Using the gcc builtin had forced the saving / restoring of fprs at begin / end of e.g. __lll_lock_elision function. The new macro saves these fprs before tbegin instruction and only restores them on a transaction abort. Restoring is not needed on a successfully started transaction. The used inline assembly does not clobber the fprs / vrs! Clobbering the latter ones would force the compiler to save / restore the call saved fprs as those overlap with the vrs, but they only need to be restored if the transaction fails. Thus the user of the tbegin macros has to compile the file / function with -msoft-float. It prevents gcc from using fprs / vrs. ChangeLog: * sysdeps/unix/sysv/linux/s390/Makefile (elision-CFLAGS): Add -msoft-float. * sysdeps/unix/sysv/linux/s390/htm.h: New File. * sysdeps/unix/sysv/linux/s390/elision-lock.c: Use __libc_t* transaction macros instead of __builtin_t*. * sysdeps/unix/sysv/linux/s390/elision-trylock.c: Likewise. * sysdeps/unix/sysv/linux/s390/elision-unlock.c: Likewise.
2016-12-20 14:12:48 +00:00
#include <htm.h>
#include <elision-conf.h>
#define aconf __elision_aconf
/* Try to elide a futex trylock. FUTEX is the futex variable. ADAPT_COUNT is
the adaptation counter in the mutex. */
int
__lll_trylock_elision (int *futex, short *adapt_count)
{
/* Implement POSIX semantics by forbiding nesting elided trylocks.
Sorry. After the abort the code is re-executed
non transactional and if the lock was already locked
return an error. */
S390: Use own tbegin macro instead of __builtin_tbegin. This patch defines __libc_tbegin, __libc_tend, __libc_tabort and __libc_tx_nesting_depth in htm.h which replaces the direct usage of equivalent gcc builtins. We have to use an own inline assembly instead of __builtin_tbegin, as tbegin has to filter program interruptions which can't be done with the builtin. Before this change, e.g. a segmentation fault within a transaction, leads to a coredump where the instruction pointer points behind the tbegin instruction instead of real failing one. Now the transaction aborts and the code should be reexecuted by the fallback path without transactions. The segmentation fault will produce a coredump with the real failing instruction. The fpc is not saved before starting the transaction. If e.g. the rounging mode is changed and the transaction is aborting afterwards, the builtin will not restore the fpc. This is now done with the __libc_tbegin macro. Now the call saved fprs have to be saved / restored in the __libc_tbegin macro. Using the gcc builtin had forced the saving / restoring of fprs at begin / end of e.g. __lll_lock_elision function. The new macro saves these fprs before tbegin instruction and only restores them on a transaction abort. Restoring is not needed on a successfully started transaction. The used inline assembly does not clobber the fprs / vrs! Clobbering the latter ones would force the compiler to save / restore the call saved fprs as those overlap with the vrs, but they only need to be restored if the transaction fails. Thus the user of the tbegin macros has to compile the file / function with -msoft-float. It prevents gcc from using fprs / vrs. ChangeLog: * sysdeps/unix/sysv/linux/s390/Makefile (elision-CFLAGS): Add -msoft-float. * sysdeps/unix/sysv/linux/s390/htm.h: New File. * sysdeps/unix/sysv/linux/s390/elision-lock.c: Use __libc_t* transaction macros instead of __builtin_t*. * sysdeps/unix/sysv/linux/s390/elision-trylock.c: Likewise. * sysdeps/unix/sysv/linux/s390/elision-unlock.c: Likewise.
2016-12-20 14:12:48 +00:00
if (__libc_tx_nesting_depth () > 0)
{
/* Note that this abort may terminate an outermost transaction that
was created outside glibc.
This persistently aborts the current transactions to force
them to use the default lock instead of retrying transactions
until their try_tbegin is zero.
*/
S390: Use own tbegin macro instead of __builtin_tbegin. This patch defines __libc_tbegin, __libc_tend, __libc_tabort and __libc_tx_nesting_depth in htm.h which replaces the direct usage of equivalent gcc builtins. We have to use an own inline assembly instead of __builtin_tbegin, as tbegin has to filter program interruptions which can't be done with the builtin. Before this change, e.g. a segmentation fault within a transaction, leads to a coredump where the instruction pointer points behind the tbegin instruction instead of real failing one. Now the transaction aborts and the code should be reexecuted by the fallback path without transactions. The segmentation fault will produce a coredump with the real failing instruction. The fpc is not saved before starting the transaction. If e.g. the rounging mode is changed and the transaction is aborting afterwards, the builtin will not restore the fpc. This is now done with the __libc_tbegin macro. Now the call saved fprs have to be saved / restored in the __libc_tbegin macro. Using the gcc builtin had forced the saving / restoring of fprs at begin / end of e.g. __lll_lock_elision function. The new macro saves these fprs before tbegin instruction and only restores them on a transaction abort. Restoring is not needed on a successfully started transaction. The used inline assembly does not clobber the fprs / vrs! Clobbering the latter ones would force the compiler to save / restore the call saved fprs as those overlap with the vrs, but they only need to be restored if the transaction fails. Thus the user of the tbegin macros has to compile the file / function with -msoft-float. It prevents gcc from using fprs / vrs. ChangeLog: * sysdeps/unix/sysv/linux/s390/Makefile (elision-CFLAGS): Add -msoft-float. * sysdeps/unix/sysv/linux/s390/htm.h: New File. * sysdeps/unix/sysv/linux/s390/elision-lock.c: Use __libc_t* transaction macros instead of __builtin_t*. * sysdeps/unix/sysv/linux/s390/elision-trylock.c: Likewise. * sysdeps/unix/sysv/linux/s390/elision-unlock.c: Likewise.
2016-12-20 14:12:48 +00:00
__libc_tabort (_HTM_FIRST_USER_ABORT_CODE | 1);
__builtin_unreachable ();
}
/* adapt_count can be accessed concurrently; these accesses can be both
inside of transactions (if critical sections are nested and the outer
critical section uses lock elision) and outside of transactions. Thus,
we need to use atomic accesses to avoid data races. However, the
value of adapt_count is just a hint, so relaxed MO accesses are
sufficient. */
if (atomic_load_relaxed (adapt_count) <= 0 && aconf.try_tbegin > 0)
{
int status = __libc_tbegin ((void *) 0);
if (__glibc_likely (status == _HTM_TBEGIN_STARTED))
{
/* Check the futex to make sure nobody has touched it in the
mean time. This forces the futex into the cache and makes
sure the transaction aborts if another thread acquires the lock
concurrently. */
if (__glibc_likely (atomic_load_relaxed (futex) == 0))
/* Lock was free. Return to user code in a transaction. */
return 0;
/* Lock was busy. Fall back to normal locking.
This can be the case if e.g. adapt_count was decremented to zero
by a former release and another thread has been waken up and
acquired it.
Since we are in a non-nested transaction there is no need to abort,
which is expensive. Simply end the started transaction. */
S390: Use own tbegin macro instead of __builtin_tbegin. This patch defines __libc_tbegin, __libc_tend, __libc_tabort and __libc_tx_nesting_depth in htm.h which replaces the direct usage of equivalent gcc builtins. We have to use an own inline assembly instead of __builtin_tbegin, as tbegin has to filter program interruptions which can't be done with the builtin. Before this change, e.g. a segmentation fault within a transaction, leads to a coredump where the instruction pointer points behind the tbegin instruction instead of real failing one. Now the transaction aborts and the code should be reexecuted by the fallback path without transactions. The segmentation fault will produce a coredump with the real failing instruction. The fpc is not saved before starting the transaction. If e.g. the rounging mode is changed and the transaction is aborting afterwards, the builtin will not restore the fpc. This is now done with the __libc_tbegin macro. Now the call saved fprs have to be saved / restored in the __libc_tbegin macro. Using the gcc builtin had forced the saving / restoring of fprs at begin / end of e.g. __lll_lock_elision function. The new macro saves these fprs before tbegin instruction and only restores them on a transaction abort. Restoring is not needed on a successfully started transaction. The used inline assembly does not clobber the fprs / vrs! Clobbering the latter ones would force the compiler to save / restore the call saved fprs as those overlap with the vrs, but they only need to be restored if the transaction fails. Thus the user of the tbegin macros has to compile the file / function with -msoft-float. It prevents gcc from using fprs / vrs. ChangeLog: * sysdeps/unix/sysv/linux/s390/Makefile (elision-CFLAGS): Add -msoft-float. * sysdeps/unix/sysv/linux/s390/htm.h: New File. * sysdeps/unix/sysv/linux/s390/elision-lock.c: Use __libc_t* transaction macros instead of __builtin_t*. * sysdeps/unix/sysv/linux/s390/elision-trylock.c: Likewise. * sysdeps/unix/sysv/linux/s390/elision-unlock.c: Likewise.
2016-12-20 14:12:48 +00:00
__libc_tend ();
/* Note: Changing the adapt_count here might abort a transaction on a
different CPU, but that could happen anyway when the futex is
acquired, so there's no need to check the nesting depth here.
See above for why relaxed MO is sufficient. */
if (aconf.skip_lock_busy > 0)
atomic_store_relaxed (adapt_count, aconf.skip_lock_busy);
}
else if (status != _HTM_TBEGIN_TRANSIENT)
{
/* A persistent abort (cc 1 or 3) indicates that a retry is
probably futile. Use the normal locking now and for the
next couple of calls.
Be careful to avoid writing to the lock. */
if (aconf.skip_trylock_internal_abort > 0)
*adapt_count = aconf.skip_trylock_internal_abort;
}
/* Could do some retries here. */
}
/* Use normal locking as fallback path if the transaction does not
succeed. */
return lll_trylock (*futex);
}
libc_hidden_def (__lll_trylock_elision)