The commit documents the ownership rules around 'struct pthread' and
when a thread can read or write to the descriptor. With those ownership
rules in place it becomes obvious that pd->stopped_start should not be
touched in several of the paths during thread startup, particularly so
for detached threads. In the case of detached threads, between the time
the thread is created by the OS kernel and the creating thread checks
pd->stopped_start, the detached thread might have already exited and the
memory for pd unmapped. As a regression test we add a simple test which
exercises this exact case by quickly creating detached threads with
large enough stacks to ensure the thread stack cache is bypassed and the
stacks are unmapped. Before the fix the testcase segfaults, after the
fix it works correctly and completes without issue.
For a detailed discussion see:
https://www.sourceware.org/ml/libc-alpha/2017-01/msg00505.html
lll_robust_unlock on i386 and x86_64 first sets the futex word to
FUTEX_WAITERS|0 before calling __lll_unlock_wake, which will set the
futex word to 0. If the thread is killed between these steps, then the
futex word will be FUTEX_WAITERS|0, and the kernel (at least current
upstream) will not set it to FUTEX_OWNER_DIED|FUTEX_WAITERS because 0 is
not equal to the TID of the crashed thread.
The lll_robust_lock assembly code on i386 and x86_64 is not prepared to
deal with this case because the fastpath tries to only CAS 0 to TID and
not FUTEX_WAITERS|0 to TID; the slowpath simply waits until it can CAS 0
to TID or the futex_word has the FUTEX_OWNER_DIED bit set.
This issue is fixed by removing the custom x86 assembly code and using
the generic C code instead. However, instead of adding more duplicate
code to the custom x86 lowlevellock.h, the code of the lll_robust* functions
is inlined into the single call sites that exist for each of these functions
in the pthread_mutex_* functions. The robust mutex paths in the latter
have been slightly reorganized to make them simpler.
This patch is meant to be easy to backport, so C11-style atomics are not
used.
[BZ #20985]
* nptl/Makefile: Adapt.
* nptl/pthread_mutex_cond_lock.c (LLL_ROBUST_MUTEX_LOCK): Remove.
(LLL_ROBUST_MUTEX_LOCK_MODIFIER): New.
* nptl/pthread_mutex_lock.c (LLL_ROBUST_MUTEX_LOCK): Remove.
(LLL_ROBUST_MUTEX_LOCK_MODIFIER): New.
(__pthread_mutex_lock_full): Inline lll_robust* functions and adapt.
* nptl/pthread_mutex_timedlock.c (pthread_mutex_timedlock): Inline
lll_robust* functions and adapt.
* nptl/pthread_mutex_unlock.c (__pthread_mutex_unlock_full): Likewise.
* sysdeps/nptl/lowlevellock.h (__lll_robust_lock_wait,
__lll_robust_lock, lll_robust_cond_lock, __lll_robust_timedlock_wait,
__lll_robust_timedlock, __lll_robust_unlock): Remove.
* sysdeps/unix/sysv/linux/i386/lowlevellock.h (lll_robust_lock,
lll_robust_cond_lock, lll_robust_timedlock, lll_robust_unlock): Remove.
* sysdeps/unix/sysv/linux/x86_64/lowlevellock.h (lll_robust_lock,
lll_robust_cond_lock, lll_robust_timedlock, lll_robust_unlock): Remove.
* sysdeps/unix/sysv/linux/sparc/lowlevellock.h (__lll_robust_lock_wait,
__lll_robust_lock, lll_robust_cond_lock, __lll_robust_timedlock_wait,
__lll_robust_timedlock, __lll_robust_unlock): Remove.
* nptl/lowlevelrobustlock.c: Remove file.
* nptl/lowlevelrobustlock.sym: Likewise.
* sysdeps/unix/sysv/linux/i386/lowlevelrobustlock.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/lowlevelrobustlock.S: Likewise.
This is a new implementation for condition variables, required
after http://austingroupbugs.net/view.php?id=609 to fix bug 13165. In
essence, we need to be stricter in which waiters a signal or broadcast
is required to wake up; this couldn't be solved using the old algorithm.
ISO C++ made a similar clarification, so this also fixes a bug in
current libstdc++, for example.
We can't use the old algorithm anymore because futexes do not guarantee
to wake in FIFO order. Thus, when we wake, we can't simply let any
waiter grab a signal, but we need to ensure that one of the waiters
happening before the signal is woken up. This is something the previous
algorithm violated (see bug 13165).
There's another issue specific to condvars: ABA issues on the underlying
futexes. Unlike mutexes that have just three states, or semaphores that
have no tokens or a limited number of them, the state of a condvar is
the *order* of the waiters. A waiter on a semaphore can grab a token
whenever one is available; a condvar waiter must only consume a signal
if it is eligible to do so as determined by the relative order of the
waiter and the signal.
Therefore, this new algorithm maintains two groups of waiters: Those
eligible to consume signals (G1), and those that have to wait until
previous waiters have consumed signals (G2). Once G1 is empty, G2
becomes the new G1. 64b counters are used to avoid ABA issues.
This condvar doesn't yet use a requeue optimization (ie, on a broadcast,
waking just one thread and requeueing all others on the futex of the
mutex supplied by the program). I don't think doing the requeue is
necessarily the right approach (but I haven't done real measurements
yet):
* If a program expects to wake many threads at the same time and make
that scalable, a condvar isn't great anyway because of how it requires
waiters to operate mutually exclusive (due to the mutex usage). Thus, a
thundering herd problem is a scalability problem with or without the
optimization. Using something like a semaphore might be more
appropriate in such a case.
* The scalability problem is actually at the mutex side; the condvar
could help (and it tries to with the requeue optimization), but it
should be the mutex who decides how that is done, and whether it is done
at all.
* Forcing all but one waiter into the kernel-side wait queue of the
mutex prevents/avoids the use of lock elision on the mutex. Thus, it
prevents the only cure against the underlying scalability problem
inherent to condvars.
* If condvars use short critical sections (ie, hold the mutex just to
check a binary flag or such), which they should do ideally, then forcing
all those waiter to proceed serially with kernel-based hand-off (ie,
futex ops in the mutex' contended state, via the futex wait queues) will
be less efficient than just letting a scalable mutex implementation take
care of it. Our current mutex impl doesn't employ spinning at all, but
if critical sections are short, spinning can be much better.
* Doing the requeue stuff requires all waiters to always drive the mutex
into the contended state. This leads to each waiter having to call
futex_wake after lock release, even if this wouldn't be necessary.
[BZ #13165]
* nptl/pthread_cond_broadcast.c (__pthread_cond_broadcast): Rewrite to
use new algorithm.
* nptl/pthread_cond_destroy.c (__pthread_cond_destroy): Likewise.
* nptl/pthread_cond_init.c (__pthread_cond_init): Likewise.
* nptl/pthread_cond_signal.c (__pthread_cond_signal): Likewise.
* nptl/pthread_cond_wait.c (__pthread_cond_wait): Likewise.
(__pthread_cond_timedwait): Move here from pthread_cond_timedwait.c.
(__condvar_confirm_wakeup, __condvar_cancel_waiting,
__condvar_cleanup_waiting, __condvar_dec_grefs,
__pthread_cond_wait_common): New.
(__condvar_cleanup): Remove.
* npt/pthread_condattr_getclock.c (pthread_condattr_getclock): Adapt.
* npt/pthread_condattr_setclock.c (pthread_condattr_setclock):
Likewise.
* npt/pthread_condattr_getpshared.c (pthread_condattr_getpshared):
Likewise.
* npt/pthread_condattr_init.c (pthread_condattr_init): Likewise.
* nptl/tst-cond1.c: Add comment.
* nptl/tst-cond20.c (do_test): Adapt.
* nptl/tst-cond22.c (do_test): Likewise.
* sysdeps/aarch64/nptl/bits/pthreadtypes.h (pthread_cond_t): Adapt
structure.
* sysdeps/arm/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/ia64/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/m68k/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/microblaze/nptl/bits/pthreadtypes.h (pthread_cond_t):
Likewise.
* sysdeps/mips/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/nios2/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/s390/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/sh/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/tile/nptl/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/unix/sysv/linux/alpha/bits/pthreadtypes.h (pthread_cond_t):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/bits/pthreadtypes.h (pthread_cond_t):
Likewise.
* sysdeps/x86/bits/pthreadtypes.h (pthread_cond_t): Likewise.
* sysdeps/nptl/internaltypes.h (COND_NWAITERS_SHIFT): Remove.
(COND_CLOCK_BITS): Adapt.
* sysdeps/nptl/pthread.h (PTHREAD_COND_INITIALIZER): Adapt.
* nptl/pthreadP.h (__PTHREAD_COND_CLOCK_MONOTONIC_MASK,
__PTHREAD_COND_SHARED_MASK): New.
* nptl/nptl-printers.py (CLOCK_IDS): Remove.
(ConditionVariablePrinter, ConditionVariableAttributesPrinter): Adapt.
* nptl/nptl_lock_constants.pysym: Adapt.
* nptl/test-cond-printers.py: Adapt.
* sysdeps/unix/sysv/linux/hppa/internaltypes.h (cond_compat_clear,
cond_compat_check_and_clear): Adapt.
* sysdeps/unix/sysv/linux/hppa/pthread_cond_timedwait.c: Remove file ...
* sysdeps/unix/sysv/linux/hppa/pthread_cond_wait.c
(__pthread_cond_timedwait): ... and move here.
* nptl/DESIGN-condvar.txt: Remove file.
* nptl/lowlevelcond.sym: Likewise.
* nptl/pthread_cond_timedwait.c: Likewise.
* sysdeps/unix/sysv/linux/i386/i486/pthread_cond_broadcast.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i486/pthread_cond_signal.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i486/pthread_cond_timedwait.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i486/pthread_cond_wait.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i586/pthread_cond_broadcast.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i586/pthread_cond_signal.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i586/pthread_cond_timedwait.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i586/pthread_cond_wait.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i686/pthread_cond_broadcast.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i686/pthread_cond_signal.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i686/pthread_cond_timedwait.S: Likewise.
* sysdeps/unix/sysv/linux/i386/i686/pthread_cond_wait.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/pthread_cond_broadcast.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/pthread_cond_signal.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S: Likewise.
The value of CPPFLAGS provided by the environment may have optimizations
that interfere with the pretty printer test requirements. To override
such optimizations the pretty printer tests must also specify CPPFLAGS.
The existing pretty printer tests are fixed and the
README.pretty-printers is updated with the new requirement.
This patch adds pretty printers for the following NPTL types:
- pthread_mutex_t
- pthread_mutexattr_t
- pthread_cond_t
- pthread_condattr_t
- pthread_rwlock_t
- pthread_rwlockattr_t
To load the pretty printers into your gdb session, do the following:
python
import sys
sys.path.insert(0, '/path/to/glibc/build/nptl/pretty-printers')
end
source /path/to/glibc/source/pretty-printers/nptl-printers.py
You can check which printers are registered and enabled by issuing the
'info pretty-printer' gdb command. Printers should trigger automatically when
trying to print a variable of one of the types mentioned above.
The printers are architecture-independent, and were tested on an AMD64 running
Ubuntu 14.04 and an x86 VM running Fedora 24.
In order to work, the printers need to know the values of various flags that
are scattered throughout pthread.h and pthreadP.h as enums and #defines. Since
replicating these constants in the printers file itself would create a
maintenance burden, I wrote a script called gen-py-const.awk that Makerules uses
to extract the constants. This script is pretty much the same as gen-as-const.awk,
except it doesn't cast the constant values to 'long' and is thorougly documented.
The constants need only to be enumerated in a .pysym file, which is then referenced
by a Make variable called gen-py-const-headers.
As for the install directory, I discussed this with Mike Frysinger and Siddhesh
Poyarekar, and we agreed that it can be handled in a separate patch, and shouldn't
block merging of this one.
In addition, I've written a series of test cases for the pretty printers.
Each lock type (mutex, condvar and rwlock) has two test programs, one for itself
and other for its related 'attributes' object. Each test program in turn has a
PExpect-based Python script that drives gdb and compares its output to the
expected printer's. The tests run on the glibc host, which is assumed to have
both gdb and PExpect; if either is absent the tests will fail with code 77
(UNSUPPORTED). For cross-testing you should use cross-test-ssh.sh as test-wrapper.
I've tested the printers on both native builds and a cross build using a Beaglebone
Black running Debian, with the build system's filesystem shared with the board
through NFS.
Finally, I've written a README that explains all this and more.
* INSTALL: Regenerated.
* Makeconfig: Add comments and whitespace to make the control flow
clearer.
(+link-printers-tests, +link-pie-printers-tests, CFLAGS-printers-tests,
installed-rtld-LDFLAGS, built-rtld-LDFLAGS, link-libc-rpath,
link-libc-tests-after-rpath-link, link-libc-printers-tests): New.
(rtld-LDFLAGS, rtld-tests-LDFLAGS, link-libc-tests-rpath-link,
link-libc-tests): Use the new variables as required.
* Makerules ($(py-const)): New rule.
generated: Add $(py-const).
* README.pretty-printers: New file.
* Rules (tests-printers-programs, tests-printers-out, py-env): New.
(others): Depend on $(py-const).
(tests): Depend on $(tests-printers-programs) or $(tests-printers-out),
as required. Pass $(tests-printers) to merge-test-results.sh.
* manual/install.texi: Add requirements for testing the pretty printers.
* nptl/Makefile (gen-py-const-headers, pretty-printers, tests-printers,
CFLAGS-test-mutexattr-printers.c CFLAGS-test-mutex-printers.c,
CFLAGS-test-condattr-printers.c, CFLAGS-test-cond-printers.c,
CFLAGS-test-rwlockattr-printers.c CFLAGS-test-rwlock-printers.c,
tests-printers-libs): Define.
* nptl/nptl-printers.py: New file.
* nptl/nptl_lock_constants.pysym: Likewise.
* nptl/test-cond-printers.c: Likewise.
* nptl/test-cond-printers.py: Likewise.
* nptl/test-condattr-printers.c: Likewise.
* nptl/test-condattr-printers.py: Likewise.
* nptl/test-mutex-printers.c: Likewise.
* nptl/test-mutex-printers.py: Likewise.
* nptl/test-mutexattr-printers.c: Likewise.
* nptl/test-mutexattr-printers.py: Likewise.
* nptl/test-rwlock-printers.c: Likewise.
* nptl/test-rwlock-printers.py: Likewise.
* nptl/test-rwlockattr-printers.c: Likewise.
* nptl/test-rwlockattr-printers.py: Likewise.
* scripts/gen-py-const.awk: Likewise.
* scripts/test_printers_common.py: Likewise.
* scripts/test_printers_exceptions.py: Likewise.
This patch consolidates all Linux lseek/lseek64/llseek implementation
in on on sysdeps/unix/sysv/linux/lseek{64}.c. It also removes the llseek
file and instead consolidate the LFS lseek implementation on lseek64.c
as for other LFS symbols implementations.
The general idea is:
- lseek: ABIs that not define __OFF_T_MATCHES_OFF64_T will preferable
use __NR__llseek if kernel supports it, otherwise they will use __NR_lseek.
ABIs that defines __OFF_T_MATCHES_OFF64_T won't produce any symbol.
- lseek64: ABIs with __OFF_T_MATCHES_OFF64_T will preferable use __NR_lseek
(since it will use 64-bit arguments without low/high splitting) and
__NR__llseek if __NR_lseek is not defined (for some ILP32 ports).
- llseek: files will be removed and symbols will be aliased ot lseek64.
ABI without __OFF_T_MATCHES_OFF64_T and without __NR_llseek (basically MIPS64n32
so far) are covered by building lseek with off_t as expected and lseek64
using __NR_lseek (as expected for off64_t being passed using 64-bit registers).
For this consolidation I mantained the x32 assembly specific implementation
because to correctly fix this it would required both the x32 fix for
{INLINE,INTERNAL}_SYSCALL [1] and a wrapper to correctly subscribe it to
return 64 bits instead of default 32 bits (as for times). It could a future
cleanup.
It is based on my previous {INTERNAL,INLINE}_SYSCALL_CALL macro [2],
although it is mainly for simplification.
Tested on x86_64, i686, aarch64, armhf, and powerpc64le.
* nptl/Makefile (libpthread-routines): Remove ptw-llseek and add
ptw-lseek64.
* sysdeps/unix/sysv/linux/Makefile (sysdeps_routines): Remove llseek.
* sysdeps/unix/sysv/linux/alpha/Makefile (sysdeps_routines):
Likewise.
* sysdeps/unix/sysv/linux/generic/wordsize-32/llseek.c: Remove file.
* sysdeps/unix/sysv/linux/generic/wordsize-32/lseek.c: Remove file.
* sysdeps/unix/sysv/linux/mips/mips64/llseek.c: Likewise.
* sysdeps/unix/sysv/linux/llseek.c: Remove file.
* sysdeps/unix/sysv/linux/lseek.c: New file.
* sysdeps/unix/sysv/linux/lseek64.c: Add default Linux implementation.
* sysdeps/unix/sysv/linux/mips/mips64/syscalls.list: Remove lseek and
__libc_lseek64 from auto-generation.
* sysdeps/unix/sysv/linux/wordsize-64/syscalls.list: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/lseek64.S: New file.
[1] https://sourceware.org/ml/libc-alpha/2016-08/msg00443.html
[2] https://sourceware.org/ml/libc-alpha/2016-08/msg00646.html
This patch correctly enable and disable asynchronous cancellation on
Linux posix_spawn. Current code invert the logic by enabling and
disabling instead. It also adds a new test to check if posix_spawn
is not a cancellation entrypoint.
Checked on x86_64, i686, powerpc64le, and aarch64.
* nptl/Makefile (tests): Add tst-exec5.
* nptl/tst-exec5.c: New file.
* sysdeps/unix/sysv/linux/spawni.c (__spawni): Correctly enable and disable
asynchronous cancellation.
This patch changes sem_open to not act as a cancellation point.
Cancellation is disable at start and reenable in function exit.
It fixes BZ #15765.
Tested on x86_64 and i686.
[BZ #15765]
* nptl/Makefile (tests): Add tst-sem16.
* nptl/tst-sem16.c: New file.
* nptl/sem_open.c (sem_open): Disable asynchronous cancellation.
Nothing depends on the PTW macro anymore, so the mechanism to define
PTW for recompliations of libc routines is no longer needed. The
source files are still recompiled for the nptl directory, just without
the “ptw-” prefix.
(Reducing the number of pattern rules in sysd-rules is critical for
improving make performance.)
Instead, call malloc and explicitly align the pointer.
There is no external location to store the original (unaligned)
pointer, and this commit increases the allocation size to store
the pointer at a fixed location relative to the TCB pointer.
The manual alignment means that some space goes unused which
was previously made available for subsequent allocations.
However, in the TLS_DTV_AT_TP case, the manual alignment code
avoids aligning the pre-TCB to the TLS block alignment. (Even
while using memalign, the allocation had some unused padding
in front.)
This concludes the removal of memalign calls from the TLS code,
and the new tst-tls3-malloc test verifies that only core malloc
routines are used.
This reverts commit 62ce266b0b.
The change is not mature enough because it needs the following fixes:
1. Redirect test output to a file like other tests
2. Eliminate the need to use a .gdbinit because distributions will
break without it. I should have caught that but I was in too much
of a hurry to get the patch in :/
3. Feature checking during configure to determine things like minimum
required gdb version, python-pexpect version, etc. to make sure
that tests work correctly.
This patch adds pretty printers for the following NPTL types:
- pthread_mutex_t
- pthread_mutexattr_t
- pthread_cond_t
- pthread_condattr_t
- pthread_rwlock_t
- pthread_rwlockattr_t
To load the pretty printers into your gdb session, do the following:
python
import sys
sys.path.insert(0, '/path/to/glibc/build/nptl/pretty-printers')
end
source /path/to/glibc/source/pretty-printers/nptl-printers.py
You can check which printers are registered and enabled by issuing the
'info pretty-printer' gdb command. Printers should trigger automatically when
trying to print a variable of one of the types mentioned above.
The printers are architecture-independent, and were manually tested on both
the gdb CLI and Eclipse CDT.
In order to work, the printers need to know the values of various flags that
are scattered throughout pthread.h and pthreadP.h as enums and #defines. Since
replicating these constants in the printers file itself would create a
maintenance burden, I wrote a script called gen-py-const.awk that Makerules uses
to extract the constants. This script is pretty much the same as gen-as-const.awk,
except it doesn't cast the constant values to 'long' and is thorougly documented.
The constants need only to be enumerated in a .pysym file, which is then referenced
by a Make variable called gen-py-const-headers.
As for the install directory, I discussed this with Mike Frysinger and Siddhesh
Poyarekar, and we agreed that it can be handled in a separate patch, and it shouldn't
block merging of this one.
In addition, I've written a series of test cases for the pretty printers.
Each lock type (mutex, condvar and rwlock) has two test programs, one for itself
and other for its related 'attributes' object. Each test program in turn has a
PExpect-based Python script that drives gdb and compares its output to the
expected printer's. The tests run on the glibc host, which is assumed to have
both gdb and PExpect; if either is absent the tests will fail with code 77
(UNSUPPORTED). For cross-testing you should use cross-test-ssh.sh as test-wrapper.
I've tested the printers on both a native build and a cross build using a Beaglebone
Black, with the build system's filesystem shared with the board through NFS.
Finally, I've written a README that explains all this and more.
Hopefully this should be good to go in now. Thanks.
ChangeLog:
2016-07-04 Martin Galvan <martin.galvan@tallertechnologies.com>
* Makeconfig (build-hardcoded-path-in-tests): Set to 'yes' for shared builds
if tests-need-hardcoded-path is defined.
(all-subdirs): Add pretty-printers.
* Makerules ($(py-const)): New rule.
* Rules (others): Add $(py-const), if defined.
* nptl/Makefile (gen-py-const-headers): Define.
* nptl/nptl-printers.py: New file.
* nptl/nptl_lock_constants.pysym: Likewise.
* pretty-printers/Makefile: Likewise.
* pretty-printers/README: Likewise.
* pretty-printers/test-condvar-attributes.c: Likewise.
* pretty-printers/test-condvar-attributes.p: Likewise.
* pretty-printers/test-condvar-printer.c: Likewise.
* pretty-printers/test-condvar-printer.py: Likewise.
* pretty-printers/test-mutex-attributes.c: Likewise.
* pretty-printers/test-mutex-attributes.py: Likewise.
* pretty-printers/test-mutex-printer.c: Likewise.
* pretty-printers/test-mutex-printer.py: Likewise.
* pretty-printers/test-rwlock-attributes.c: Likewise.
* pretty-printers/test-rwlock-attributes.py: Likewise.
* pretty-printers/test-rwlock-printer.c: Likewise.
* pretty-printers/test-rwlock-printer.py: Likewise.
* pretty-printers/test_common.py: Likewise.
* scripts/gen-py-const.awk: Likewise.
tst-cleanupx4 is linked with tst-cleanupx4.o and tst-cleanup4aux.o.
Since tst-cleanupx4.o is compiled from tst-cleanup4.c with -fexceptions,
tst-cleanup4aux.c should also be compiled with -fexceptions.
Tested on x86-64 and i686.
[BZ #18645]
* nptl/Makefile (extra-test-objs): Add tst-cleanupx4aux.o.
(test-extras): Add tst-cleanupx4aux.
(CFLAGS-tst-cleanupx4aux.c): New. Set to -fexceptions.
($(objpfx)tst-cleanupx4): Replace tst-cleanup4aux.o with
tst-cleanupx4aux.o.
* nptl/tst-cleanupx4aux.c: New file.
This patch adds cancellation tests for both sendmmsg and recvmmsg
syscalls. Since for some system configuration (x86_64/i686 on
older kernels and non-Linux platforms), the tests are added as
two independent that report as unsupported if the syscall is not
presented.
Both new tests uses the already tst-cancel4.c code, which as moved
to a common tst-cancel4-common{.c,h} files.
Tested on x86_64 and i686.
* nptl/Makefile (test): Add tst-cancel4_1 and tst-cancel4_2.
* nptl/tst-cancel4-common.c: New file.
* nptl/tst-cancel4-common.h: Likewise.
* nptl/tst-cancel4.c: Move common definitions to
tst-cancel4-common.{c,h} file.
* nptl/tst-cancel4_1.c: New test.
* nptl/tst-cancel4_2.c: New test.
After some discussion in libc-alpha about this POSIX compliance fix, I see
that GLIBC should indeed revert back to previous definition of msghdr and
cmsghdr and implementation of sendmsg, recvmsg, sendmmsg, recvmmsg due some
reasons:
* The possible issue where the syscalls wrapper add the compatibility
layer is quite limited in scope and range. And kernel current
also add some limits to the values on the internal msghdr and
cmsghdr fields:
- msghdr::msg_iovlen larger than UIO_MAXIOV (1024) returns
EMSGSIZE.
- msghdr::msg_controllen larger than INT_MAX returns ENOBUFS.
* There is a small performance hit for recvmsg/sendmsg/recmmsg which
is neglectable, but it is a big hit for sendmmsg since now instead
of calling the syscall for the packed structure, GLIBC is calling
multiple sendmsg. This defeat the very existence of the syscall.
* It currently breaks libsanitizer build on GCC [1] (I fixed on compiler-rt).
However the fix is incomplete because it does add any runtime check
since libsanitizer currently does not have any facility to intercept
symbols with multiple version [2].
This, along with incorret dlsym/dlvsym return for versioned symbol due
another bug [3], makes hard to interpose versioned symbols.
Also, current approach of fixing GCC PR#71445 leads to half-baked
solutions without versioned symbol interposing.
This patch basically reverts commits 2f0dc39029, 222c2d7f43,
af7f7c7ec8. I decided to not revert abf29edd4a (Adjust
kernel-features.h defaults for recvmsg and sendmsg) mainly because it
does not really address the POSIX compliance original issue and also
adds some cleanups.
Tested on x86, i386, s390, s390x, aarch64, and powerpc64le.
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71445
[2] https://github.com/google/sanitizers/issues/628
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=14932
* conform/data/sys/socket.h-data (msghdr.msg_iovlen): Add xfail-.
(msghdr.msg_controllen): Likewise.
(cmsghdr.cmsg_len): Likewise.
* nptl/Makefile (libpthread-routines): Remove ptw-oldrecvmsg and
ptw-oldsendmsg.
(CFLAGS-oldrecvmsg.c): Remove rule.
(CFLAGS-oldsendmsg.c): Likewise.
(CFLAGS-recvmsg.c): Add rule.
(CFLAGS-sendmsg.c): Likewise.
* sysdeps/unix/sysv/linux/Makefile (sysdep_routines): Remove
oldrecvmsg, oldsendmsg, oldrecvmmsg, oldsendmmsg.
(CFLAGS-recvmsg.c): Remove rule.
(CFLAGS-sendmsg.c): Likewise.
(CFLAGS-oldrecvmsg.c): Likewise.
(CFLAGS-oldsendmsg.c): Likewise.
(CFLAGS-recvmmsg.c): Likewise.
* sysdeps/unix/sysv/linux/bits/socket.h (msghdr.msg_iovlen): Revert
to kernel defined interfaces.
(msghdr.msg_controllen): Likewise.
(cmsghdr.cmsg_len): Likewise.
(msghdr.__glibc_reserved1): Remove member.
(msghdr.__glibc_reserved2): Likewise.
(cmsghdr.__glibc_reserved1): Likewise.
* sysdeps/unix/sysv/linux/oldrecvmmsg.c: Remove file.
* sysdeps/unix/sysv/linux/oldrecvmsg.c: Likewise.
* sysdeps/unix/sysv/linux/oldsendmmsg.c: Likewise.
* sysdeps/unix/sysv/linux/oldsendmsg.c: Likewise.
* sysdeps/unix/sysv/linux/recvmmsg.c: Revert back to previous
version.
* sysdeps/unix/sysv/linux/recvmsg.c: Likewise.
* sysdeps/unix/sysv/linux/sendmmsg.c: Likewise.
* sysdeps/unix/sysv/linux/sendmsg.c: Likewise.
* sysdeps/unix/sysv/linux/aarch64/Versions [libc] (GLIBC_2.24):
Remove recvmsg and sendmsg.
* sysdeps/unix/sysv/linux/alpha/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/hppa/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/i386/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/m68k/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/microblaze/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/Versions
[libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/nios2/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/Versions
[libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/sh/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/sparc/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/tile/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/x86_64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/Versions: Remove file
* sysdeps/unix/sysv/linux/x86_64/64/Versions: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/Versions: Likewise.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Remove new 2.24
version for {recv,send,recm,sendm}msg.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilepro/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
This patch fixes wrong/missing bits from the Fix {recv,send}{m}msg
standard compliance (BZ#16919) patches:
* nptl/Makefile sets CFLAGS-oldrecvfrom.c, but there's no such file as
oldrecvfrom.c. It should be oldsendmsg.c as defined by ChangeLog.
* sysdeps/unix/sysv/linux/hppa/Versions and
sysdeps/unix/sysv/linux/i386/Versions list a symbol recvms instead of
recvmsg at version GLIBC_2.24.
* nptl/Makefile (CFLAGS-oldrecvfrom.c): Remove rule.
(CFLAGS-oldsendmsg.c): Add rule.
* sysdeps/unix/sysv/linux/hppa/Versions [libc] (GLIBC_2.24):
Correct recvmsg symbol name.
* sysdeps/unix/sysv/linux/i386/Versions [libc] (GLIBC_2.24):
Likewise.
Error checking mutexes are not supposed to be subject to lock elision.
That would defeat the error checking nature of the mutex because lock
elision doesn't record ownership.
The previous barrier implementation did not fulfill the POSIX requirements
for when a barrier can be destroyed. Specifically, it was possible that
threads that haven't noticed yet that their round is complete still access
the barrier's memory, and that those accesses can happen after the barrier
has been legally destroyed.
The new algorithm does not have this issue, and it avoids using a lock
internally.
NPTL has tests that initializers work with various -std= options. Now
that we can rely on -std=gnu11 and -std=c11 being available, this
patch adds versions of those tests for those options.
Tested for x86_64 and x86 (testsuite).
* nptl/tst-initializers1-c11.c: New file.
* nptl/tst-initializers1-gnu11.c: Likewise.
* nptl/Makefile (tests): Add these new tests.
(CFLAGS-tst-initializers1-c11.c): New variable.
(CFLAGS-tst-initializers1-gnu11.c): Likewise.
sysdeps/nptl/configure.ac tests for forced unwind support and the C
cleanup attribute, giving errors if either is unsupported. It does
nothing beyond running those two tests.
Both the attribute, and _Unwind_GetCFA which is used in the forced
unwind test, were added in GCC 3.3. Thus these tests are long
obsolete, and this patch removes the configure fragment running them,
along with associated conditionals.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
* sysdeps/nptl/configure.ac: Remove file.
* sysdeps/nptl/configure: Remove generated file.
* configure.ac (libc_cv_forced_unwind): Do not substitute.
* configure: Regenerated.
* config.h.in (HAVE_FORCED_UNWIND): Remove #undef.
* config.make.in (have-forced-unwind): Remove variable.
* nptl/Makefile [$(have-forced-unwind) = yes]: Make code
unconditional.
* nptl/descr.h [HAVE_FORCED_UNWIND]: Likewise.
* nptl/unwind.c [HAVE_FORCED_UNWIND]: Likewise.
(__pthread_unwind) [!HAVE_FORCED_UNWIND]: Remove conditional code.
* nptl/version.c [HAVE_FORCED_UNWIND]: Make code unconditional.
* sysdeps/nptl/Makefile [$(have-forced-unwind) = yes]: Make code
unconditional.
This patch pthread cancellation tests to check for failures cases
wherer the syscall cancel wrapper should both set the error and
the errno values.
Tested on i686, x86_64, x32, powerpc64le, and aarch64.
* nptl/Makefile (tests): Add tst-cancel26.c and tst-cancel27.c.
* nptl/tst-cancel26.c: New file.
* nptl/tst-cancel27.c: Likewise.
The recently introduced TLS variables in the thread-local destructor
implementation (__cxa_thread_atexit_impl) used the default GD access
model, resulting in a call to __tls_get_addr. This causes a deadlock
with recent changes to the way TLS is initialized because DTV
allocations are delayed and hence despite knowing the offset to the
variable inside its TLS block, the thread has to take the global rtld
lock to safely update the TLS offset.
This causes deadlocks when a thread is instantiated and joined inside
a destructor of a dlopen'd DSO. The correct long term fix is to
somehow not take the lock, but that will need a lot deeper change set
to alter the way in which the big rtld lock is used.
Instead, this patch just eliminates the call to __tls_get_addr for the
thread-local variables inside libc.so, libpthread.so and rtld by
building all of their units with -mtls-model=initial-exec.
There were concerns that the static storage for TLS is limited and
hence we should not be using it. Additionally, dynamically loaded
modules may result in libc.so looking for this static storage pretty
late in static binaries. Both concerns are valid when using TLSDESC
since that is where one may attempt to allocate a TLS block from
static storage for even those variables that are not IE. They're not
very strong arguments for the traditional TLS model though, since it
assumes that the static storage would be used sparingly and definitely
not by default. Hence, for now this would only theoretically affect
ARM architectures.
The impact is hence limited to statically linked binaries that dlopen
modules that in turn load libc.so, all that on arm hardware. It seems
like a small enough impact to justify fixing the larger problem that
currently affects everything everywhere.
This still does not solve the original problem completely. That is,
it is still possible to deadlock on the big rtld lock with a small
tweak to the test case attached to this patch. That problem is
however not a regression in 2.22 and hence could be tackled as a
separate project. The test case is picked up as is from Alex's patch.
This change has been tested to verify that it does not cause any
issues on x86_64.
ChangeLog:
[BZ #18457]
* nptl/Makefile (tests): New test case tst-join7.
(modules-names): New test case module tst-join7mod.
* nptl/tst-join7.c: New file.
* nptl/tst-join7mod.c: New file.
* Makeconfig (tls-model): Pass -ftls-model=initial-exec for
all translation units in libc.so, libpthread.so and rtld.
and also powerpc64 and powerpc64le. See the discussion in the thread
below for details. This change reverts the problematic bits leaving
the added test in place and marking XFAIL in anticipation of fixing
the bug in the near future.
https://sourceware.org/ml/libc-alpha/2015-07/msg00141.html
[BZ #18435]
* nptl/pthreadP.h (pthread_cleanup_push, pthread_cleanup_pop):
Revert commit ed225df3ad.
* nptl/Makefile (test-xfail-tst-once5): Define.
the initialization routine to exit by throwing an exception.
Such an execution, termed exceptional, requires call_once to
propagate the exception to its caller. A program may contain
any number of exceptional executions but only one returning
execution (which, if it exists, must be the last execution
with the same once flag).
On POSIX systems such as Linux, std::call_once is implemented
in terms of pthread_once. However, as discussed in libstdc++
bug 66146 - "call_once not C++11-compliant on ppc64le," GLIBC's
pthread_once hangs when the initialization function exits by
throwing an exception on at least arm and ppc64 (though
apparently not on x86_64). This effectively prevents call_once
from conforming to the C++ requirements since there doesn't
appear to be a thread-safe way to work around this problem in
libstdc++.
This patch changes pthread_once to handle gracefully init
functions that exit by throwing exceptions. It was successfully
tested on ppc64, ppc64le, and x86_64.
[BZ #18435]
* nptl/Makefile: Add tst-once5.cc.
* nptl/pthreadP.h (pthread_cleanup_push, pthread_cleanup_pop):
Remove macro redefinitions.
* nptl/tst-once5.cc: New test.
In commit 02657da2cf, .interp section
was removed from libpthread.so. This led to an error:
$ /lib64/libpthread.so.0
Native POSIX Threads Library by Ulrich Drepper et al
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Forced unwind support included.
Segmentation fault
(gdb) bt
#0 0x00000000000055a6 in _exit@plt ()
Unfortunately, there is no way to add a regression test for the bug
because .interp specifies the path to dynamic linker of the target
system.
[BZ #18479]
* nptl/pt-interp.c: New file.
* nptl/Makefile (libpthread-routines, libpthread-shared-only-routines):
Add pt-interp.
[$(build-shared) = yes] ($(objpfx)pt-interp.os): Depend on
$(common-objpfx)runtime-linker.h.
This adds wake-ups that would be missing if assuming that for a
non-writer-preferring rwlock, if one thread has acquired a rdlock and
does not release it, another thread will eventually acquire a rdlock too
despite concurrent write lock acquisition attempts. BZ 14958 is about
supporting this assumption. Strictly speaking, this isn't a valid
test case, but nonetheless worth supporting (see comment 7 of BZ 14958).
If we set up a rwlock to prefer writers (and disallow recursive rdlock
acquisitions), then readers will block for writers that are blocked to
acquire the lock (otherwise, readers could constantly enter and exit,
and the writer would never get the lock). However, the existing
implementation did not wake such readers when the writer timed out.
This patch adds the missing wake-up.
There's no similar case for writers being blocked on readers.
This patch removes the socket.S implementation for all ports and replace
it by a C implementation using socketcall. For ports that implement
the syscall directly, there is no change.
The patch idea is to simplify the socket function implementation that
uses the socketcall to be based on C implemetation instead of a pseudo
assembly implementation with arch specific parts. The patch then remove
the assembly implementatation for the ports which uses socketcall
(i386, microblaze, mips, powerpc, sparc, m68k, s390 and sh).
I have cross-build GLIBC for afore-mentioned ports and tested on both
i386 and ppc32 without regressions.
pthread_mutexattr_settype adds PTHREAD_MUTEX_NO_ELISION_NP to kind,
which is an internal flag that pthread_mutexattr_gettype shouldn't
expose, since pthread_mutexattr_settype wouldn't accept it.
This commit fixes semaphore destruction by either using 64b atomic
operations (where available), or by using two separate fields when only
32b atomic operations are available. In the latter case, we keep a
conservative estimate of whether there are any waiting threads in one
bit of the field that counts the number of available tokens, thus
allowing sem_post to atomically both add a token and determine whether
it needs to call futex_wake.
See:
https://sourceware.org/ml/libc-alpha/2014-12/msg00155.html
This patch changes _dl_allocate_tls_init to resize DTV if the current DTV
isn't big enough. Tested on X86-64, x32 and ia32.
[BZ #13862]
* elf/dl-tls.c: Include <atomic.h>.
(oom): Remove #ifdef SHARED/#endif.
(_dl_static_dtv, _dl_initial_dtv): Moved before ...
(_dl_resize_dtv): This. Extracted from _dl_update_slotinfo.
(_dl_allocate_tls_init): Resize DTV if the current DTV isn't
big enough.
(_dl_update_slotinfo): Call _dl_resize_dtv to resize DTV.
* nptl/Makefile (tests): Add tst-stack4.
(modules-names): Add tst-stack4mod.
($(objpfx)tst-stack4): New.
(tst-stack4mod.sos): Likewise.
($(objpfx)tst-stack4.out): Likewise.
($(tst-stack4mod.sos)): Likewise.
(clean): Likewise.
* nptl/tst-stack4.c: New file.
* nptl/tst-stack4mod.c: Likewise.
