This patch remove the PID cache and usage in current GLIBC code. Current
usage is mainly used a performance optimization to avoid the syscall,
however it adds some issues:
- The exposed clone syscall will try to set pid/tid to make the new
thread somewhat compatible with current GLIBC assumptions. This cause
a set of issue with new workloads and usecases (such as BZ#17214 and
[1]) as well for new internal usage of clone to optimize other algorithms
(such as clone plus CLONE_VM for posix_spawn, BZ#19957).
- The caching complexity also added some bugs in the past [2] [3] and
requires more effort of each port to handle such requirements (for
both clone and vfork implementation).
- Caching performance gain in mainly on getpid and some specific
code paths. The getpid performance leverage is questionable [4],
either by the idea of getpid being a hotspot as for the getpid
implementation itself (if it is indeed a justifiable hotspot a
vDSO symbol could let to a much more simpler solution).
Other usage is mainly for non usual code paths, such as pthread
cancellation signal and handling.
For thread creation (on stack allocation) the code simplification in fact
adds some performance gain due the no need of transverse the stack cache
and invalidate each element pid.
Other thread usages will require a direct getpid syscall, such as
cancellation/setxid signal, thread cancellation, thread fail path (at
create_thread), and thread signal (pthread_kill and pthread_sigqueue).
However these are hardly usual hotspots and I think adding a syscall is
justifiable.
It also simplifies both the clone and vfork arch-specific implementation.
And by review each fork implementation there are some discrepancies that
this patch also solves:
- microblaze clone/vfork does not set/reset the pid/tid field
- hppa uses the default vfork implementation that fallback to fork.
Since vfork is deprecated I do not think we should bother with it.
The patch also removes the TID caching in clone. My understanding for
such semantic is try provide some pthread usage after a user program
issue clone directly (as done by thread creation with CLONE_PARENT_SETTID
and pthread tid member). However, as stated before in multiple discussions
threads, GLIBC provides clone syscalls without further supporting all this
semantics.
I ran a full make check on x86_64, x32, i686, armhf, aarch64, and powerpc64le.
For sparc32, sparc64, and mips I ran the basic fork and vfork tests from
posix/ folder (on a qemu system). So it would require further testing
on alpha, hppa, ia64, m68k, nios2, s390, sh, and tile (I excluded microblaze
because it is already implementing the patch semantic regarding clone/vfork).
[1] https://codereview.chromium.org/800183004/
[2] https://sourceware.org/ml/libc-alpha/2006-07/msg00123.html
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=15368
[4] http://yarchive.net/comp/linux/getpid_caching.html
* sysdeps/nptl/fork.c (__libc_fork): Remove pid cache setting.
* nptl/allocatestack.c (allocate_stack): Likewise.
(__reclaim_stacks): Likewise.
(setxid_signal_thread): Obtain pid through syscall.
* nptl/nptl-init.c (sigcancel_handler): Likewise.
(sighandle_setxid): Likewise.
* nptl/pthread_cancel.c (pthread_cancel): Likewise.
* sysdeps/unix/sysv/linux/pthread_kill.c (__pthread_kill): Likewise.
* sysdeps/unix/sysv/linux/pthread_sigqueue.c (pthread_sigqueue):
Likewise.
* sysdeps/unix/sysv/linux/createthread.c (create_thread): Likewise.
* sysdeps/unix/sysv/linux/getpid.c: Remove file.
* nptl/descr.h (struct pthread): Change comment about pid value.
* nptl/pthread_getattr_np.c (pthread_getattr_np): Remove thread
pid assert.
* sysdeps/unix/sysv/linux/pthread-pids.h (__pthread_initialize_pids):
Do not set pid value.
* nptl_db/td_ta_thr_iter.c (iterate_thread_list): Remove thread
pid cache check.
* nptl_db/td_thr_validate.c (td_thr_validate): Likewise.
* sysdeps/aarch64/nptl/tcb-offsets.sym: Remove pid offset.
* sysdeps/alpha/nptl/tcb-offsets.sym: Likewise.
* sysdeps/arm/nptl/tcb-offsets.sym: Likewise.
* sysdeps/hppa/nptl/tcb-offsets.sym: Likewise.
* sysdeps/i386/nptl/tcb-offsets.sym: Likewise.
* sysdeps/ia64/nptl/tcb-offsets.sym: Likewise.
* sysdeps/m68k/nptl/tcb-offsets.sym: Likewise.
* sysdeps/microblaze/nptl/tcb-offsets.sym: Likewise.
* sysdeps/mips/nptl/tcb-offsets.sym: Likewise.
* sysdeps/nios2/nptl/tcb-offsets.sym: Likewise.
* sysdeps/powerpc/nptl/tcb-offsets.sym: Likewise.
* sysdeps/s390/nptl/tcb-offsets.sym: Likewise.
* sysdeps/sh/nptl/tcb-offsets.sym: Likewise.
* sysdeps/sparc/nptl/tcb-offsets.sym: Likewise.
* sysdeps/tile/nptl/tcb-offsets.sym: Likewise.
* sysdeps/x86_64/nptl/tcb-offsets.sym: Likewise.
* sysdeps/unix/sysv/linux/aarch64/clone.S: Remove pid and tid caching.
* sysdeps/unix/sysv/linux/alpha/clone.S: Likewise.
* sysdeps/unix/sysv/linux/arm/clone.S: Likewise.
* sysdeps/unix/sysv/linux/hppa/clone.S: Likewise.
* sysdeps/unix/sysv/linux/i386/clone.S: Likewise.
* sysdeps/unix/sysv/linux/ia64/clone2.S: Likewise.
* sysdeps/unix/sysv/linux/mips/clone.S: Likewise.
* sysdeps/unix/sysv/linux/nios2/clone.S: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/clone.S: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/sh/clone.S: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/clone.S: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/tile/clone.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/aarch64/vfork.S: Remove pid set and reset.
* sysdeps/unix/sysv/linux/alpha/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/arm/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/i386/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/ia64/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/m68k/clone.S: Likewise.
* sysdeps/unix/sysv/linux/m68k/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/mips/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/nios2/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/sh/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/tile/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/vfork.S: Likewise.
* sysdeps/unix/sysv/linux/tst-clone2.c (f): Remove direct pthread
struct access.
(clone_test): Remove function.
(do_test): Rewrite to take in consideration pid is not cached anymore.
There are a few .set mips* assembler directives used in the MIPS specific
sysdep code that force an instruction to be assembled for a specific ISA.
The reason for these is to allow an instruction to be encoded when it might
not be supported in the current ISA (when the code is run the Linux kernel
will trap and emulate any unsupported instructions). Unfortunately forcing
a specific ISA means that when assembling for a newer ISA, where the
instruction has a different encoding, the wrong encoding will be used.
* sysdeps/mips/bits/atomic.h [_MIPS_SIM == _ABIO32] (MIPS_PUSH_MIPS2):
Only use .set mips2 if the current ISA is below mips2.
* sysdeps/mips/sys/tas.h [_MIPS_SIM == _ABIO32] (_test_and_set):
Likewise.
* sysdeps/mips/nptl/tls.h (READ_THREAD_POINTER): Only use .set
mips32r2 if the current ISA is below mips32r2.
* sysdeps/mips/tls-macros.h (TLS_RDHWR): New define.
(TLS_IE): Updated to use the TLD_RDHWR macro.
(TLS_LE): Likewise.
* sysdeps/unix/mips/sysdep.h (__mips_isa_rev): Moved out of #ifdef
__ASSEMBLER__ condition.
Various architectures have files such as sysdeps/<arch>/shlib-versions
whose contents are in fact entirely Linux-specific, relating only to
the symbol / shared library versions for the port to Linux on that
architecture, when any future port to a different OS on that
architecture would use the symbol version of the glibc release it goes
in, as standard for new ports.
This patch moves such files under sysdeps/unix/sysv/linux/, merging in
the contents of sysdeps/<arch>/nptl/shlib-versions in the process.
The only bits not moved are those relating to libgcc_s versions, which
don't appear OS-specific in the same way that glibc's symbol versions
so. It deliberately does not change the regular expressions given for
matching configurations in each file; some match only Linux although
not Linux-specific, or match other OSes although Linux-specific. It
is with a view to at least the following further cleanups:
* Move architecture-specific content from the toplevel shlib-versions
and nptl/shlib-versions into sysdeps shlib-versions files, so
eliminating another difference between ex-ports and non-ex-ports
architectures.
* Likewise, for OS-specific content in shlib-versions files.
* At that point, the first field in shlib-versions files (the regular
expression matching a configuration triplet) should be redundant, so
eliminate that field and leave shlib-versions selection working
purely on a sysdeps basis (with limited use of %ifdef in
shlib-versions files when needed) rather than having its own
separate mechanism to select what configuration information is
relevant.
* Move the build of gnu/lib-names.h to a similar mechanism to that
used for gnu/stubs.h (each library build installing a version of the
header specifically for that build), so we can eliminate the
duplication of soname information in the makefiles and get it purely
from shlib-versions files again.
There may be other cleanups possible as well (in particular, I'm not
sure that all cases where the same "Earliest symbol set" information
is repeated for many different libraries actually should need to
repeat it rather than specifying it just once for DEFAULT for the
given configuration, and separately specifying any non-default choices
of soname).
Tested x86_64 that the installed shared libraries are unchanged by
this patch.
* sysdeps/aarch64/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/aarch64/shlib-versions: ... here.
* sysdeps/alpha/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/alpha/shlib-versions: ... here.
* sysdeps/arm/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/arm/shlib-versions: ... here.
* sysdeps/hppa/shlib-versions: Move all contents except for
libgcc_s entry to ...
* sysdeps/unix/sysv/linux/hppa/shlib-versions: ... here. Merge in
entry from ...
* sysdeps/hppa/nptl/shlib-versions: ... here. Remove file.
* sysdeps/ia64/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/ia64/shlib-versions: ... here. Merge in
entry from ...
* sysdeps/ia64/nptl/shlib-versions: ... here. Remove file.
* sysdeps/m68k/coldfire/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/m68k/coldfire/shlib-versions: ... here.
* sysdeps/microblaze/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/microblaze/shlib-versions: ... here.
* sysdeps/mips/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/mips/shlib-versions: ... here. Merge in
entry from ...
* sysdeps/mips/nptl/shlib-versions: ... here. Remove file.
* sysdeps/tile/shlib-versions: Move to ...
* sysdeps/unix/sysv/linux/tile/shlib-versions: ... here.
* sysdeps/unix/sysv/linux/x86_64/64/shlib-versions: Merge in entry
from ...
* sysdeps/x86_64/64/shlib-versions: ... here. Remove file.
* sysdeps/unix/sysv/linux/x86_64/x32/shlib-versions: Merge in
entry from ...
* sysdeps/x86_64/x32/shlib-versions: ... here. Remove file.
I've moved the MIPS port from ports to the main sysdeps hierarchy.
Beyond the README update, the move of the files was simply
git mv ports/sysdeps/mips sysdeps/mips
git mv ports/sysdeps/unix/mips sysdeps/unix/mips
git mv ports/sysdeps/unix/sysv/linux/mips sysdeps/unix/sysv/linux/mips
and in addition to the ChangeLog entries here, I put a note at the top
of ports/ChangeLog.mips similar to those in other files.
Tested that disassembly of installed shared libraries for mips is the
same before and after this patch (except for ld.so where paths in
assertions are involved, as for arm).
* sysdeps/mips: Move directory from ports/sysdeps/mips.
* sysdeps/unix/mips: Move directory from ports/sysdeps/unix/mips.
* sysdeps/unix/sysv/linux/mips: Move directory from
ports/sysdeps/unix/sysv/linux/mips.
* README: Update listing for mips-*-linux-gnu and
mips64-*-linux-gnu.
* sysdeps/mips: Move directory to ../sysdeps/mips.
* sysdeps/unix/mips: Move directory to ../sysdeps/unix/mips.
* sysdeps/unix/sysv/linux/mips: Move directory to
../sysdeps/unix/sysv/linux/mips.
