This patch implements a new posix_spawn{p} implementation for Linux. The main
difference is it uses the clone syscall directly with CLONE_VM and CLONE_VFORK
flags and a direct allocated stack. The new stack and start function solves
most the vfork limitation (possible parent clobber due stack spilling). The
remaning issue are related to signal handling:
1. That no signal handlers must run in child context, to avoid corrupt
parent's state.
2. Child must synchronize with parent to enforce stack deallocation and
to possible return execv issues.
The first one is solved by blocking all signals in child, even NPTL-internal
ones (SIGCANCEL and SIGSETXID). The second issue is done by a stack allocation
in parent and a synchronization with using a pipe or waitpid (in case or error).
The pipe has the advantage of allowing the child signal an exec error (checked
with new tst-spawn2 test).
There is an inherent race condition in pipe2 usage for architectures that do not
support the syscall directly. In such cases the a pipe plus fctnl is used
instead and it may lead to file descriptor leak in parent (as decribed by fcntl
documentation).
The child process stack is allocate with a mmap with MAP_STACK flag using
default architecture stack size. Although it is slower than use a stack buffer
from parent, it allows some slack for the compatibility code to run scripts
with no shebang (which may use a buffer with size depending of argument list
count).
Performance should be similar to the vfork default posix implementation and
way faster than fork path (vfork on mostly linux ports are basically
clone with CLONE_VM plus CLONE_VFORK). The only difference is the syscalls
required for the stack allocation/deallocation.
It fixes BZ#10354, BZ#14750, and BZ#18433.
Tested on i386, x86_64, powerpc64le, and aarch64.
[BZ #14750]
[BZ #10354]
[BZ #18433]
* include/sched.h (__clone): Add hidden prototype.
(__clone2): Likewise.
* include/unistd.h (__dup): Likewise.
* posix/Makefile (tests): Add tst-spawn2.
* posix/tst-spawn2.c: New file.
* sysdeps/posix/dup.c (__dup): Add hidden definition.
* sysdeps/unix/sysv/linux/aarch64/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/alpha/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/arm/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/hppa/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/i386/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/ia64/clone2.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/m68k/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/microblaze/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/mips/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/nios2/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S (__clone):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S (__clone):
Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/sh/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/tile/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/x86_64/clone.S (__clone): Likewise.
* sysdeps/unix/sysv/linux/nptl-signals.h
(____nptl_is_internal_signal): New function.
* sysdeps/unix/sysv/linux/spawni.c: New file.
This patch removes all the dynamic allocation on execvpe code and
instead use direct stack allocation. This is QoI approach to make
it possible use in scenarios where memory is shared with parent
(vfork or clone with CLONE_VM).
For default process spawn (script file without a shebang), stack
allocation is bounded by NAME_MAX plus PATH_MAX plus 1. Large
file arguments returns an error (ENAMETOOLONG). This differs than
current GLIBC pratice in general, but it used to limit stack
allocation for large inputs. Also, path in PATH environment variable
larger than PATH_MAX are ignored.
The shell direct execution exeception, where execve returns ENOEXEC,
might requires a large stack allocation due large input argument list.
Tested on i686, x86_64, powerpc64le, and aarch64.
* posix/execvpe.c (__execvpe): Remove dynamic allocation.
* posix/Makefile (tests): Add tst-execvpe{1,2,3,4,5,6}.
* posix/tst-execvp1.c (do_test): Use a macro to call execvp.
* posix/tst-execvp2.c (do_test): Likewise.
* posix/tst-execvp3.c (do_test): Likewise.
* posix/tst-execvp4.c (do_test): Likewise.
* posix/tst-execvpe1.c: New file.
* posix/tst-execvpe2.c: Likewise.
* posix/tst-execvpe3.c: Likewise.
* posix/tst-execvpe4.c: Likewise.
* posix/tst-execvpe5.c: Likewise.
* posix/tst-execvpe6.c: Likewise.
GLIBC execl{e,p} implementation might use malloc if the total number of
arguments exceed initial assumption size (1024). This might lead to
issues in two situations:
1. execl/execle is stated to be async-signal-safe by POSIX [1]. However
if execl is used in a signal handler with a large argument set (that
may call malloc internally) and if the resulting call fails it might
lead malloc in the program in a bad state.
2. If the functions are used in a vfork/clone(VFORK) situation it also
might issue malloc internal bad state.
This patch fixes it by using stack allocation instead. It also fixes
BZ#19534.
Tested on x86_64.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html
[BZ #19534]
* posix/execl.c (execl): Remove dynamic memory allocation.
* posix/execle.c (execle): Likewise.
* posix/execlp.c (execlp): Likewise.
Since we have loaded address of PREINIT_FUNCTION into %rax, we can
avoid extra branch to PLT slot.
[BZ #19745]
* sysdeps/x86_64/crti.S (_init): Replace PREINIT_FUNCTION@PLT
with *%rax in call.
Since __libc_start_main is called very early, lazy binding isn't relevant
here. Use indirect branch via GOT to avoid extra branch to PLT slot.
[BZ #19745]
* sysdeps/x86_64/start.S (_start): __libc_start_main@PLT
with *__libc_start_main@GOTPCREL(%rip) in call.
Chek Fast_Unaligned_Load, instead of Slow_BSF, and also check for
Fast_Copy_Backward to enable __memcpy_ssse3_back. Existing selection
order is updated with following selection order:
1. __memcpy_avx_unaligned if AVX_Fast_Unaligned_Load bit is set.
2. __memcpy_sse2_unaligned if Fast_Unaligned_Load bit is set.
3. __memcpy_sse2 if SSSE3 isn't available.
4. __memcpy_ssse3_back if Fast_Copy_Backward bit it set.
5. __memcpy_ssse3
[BZ #18880]
* sysdeps/x86_64/multiarch/memcpy.S: Check Fast_Unaligned_Load,
instead of Slow_BSF, and also check for Fast_Copy_Backward to
enable __memcpy_ssse3_back.
We should turn on bit_Prefer_MAP_32BIT_EXEC in EXTRA_LD_ENVVARS without
overriding other bits.
[BZ #19758]
* sysdeps/unix/sysv/linux/x86_64/64/dl-librecon.h
(EXTRA_LD_ENVVARS): Or bit_Prefer_MAP_32BIT_EXEC.
No need to compile x86_64 _mcount.S with -pg. We can just copy the
normal static object.
* gmon/Makefile (noprof): Add $(sysdep_noprof).
* sysdeps/x86_64/Makefile (sysdep_noprof): Add _mcount.
Since __mcount_internal and __sigjmp_save are internal to x86-64 libc.so:
3532: 0000000000104530 289 FUNC LOCAL DEFAULT 13 __mcount_internal
3391: 0000000000034170 38 FUNC LOCAL DEFAULT 13 __sigjmp_save
they can be called directly without PLT.
* sysdeps/x86_64/_mcount.S (C_LABEL(_mcount)): Call
__mcount_internal directly.
(C_LABEL(__fentry__)): Likewise.
* sysdeps/x86_64/setjmp.S __sigsetjmp): Call __sigjmp_save
directly.
In 1999 the project split "localedir" into "localedir" (path to compiled
locale archives) and "msgcatdir" (path to message catalogs). This
predates the 2002 change in the GNU Coding Standard to document the use
of "localedir" for the path to the message catalogs. It appears that
newlib, gcc, and several other projects also used "msgcatdir" at one
point or another in the past, and so it is in line with historical
precedent that glibc would also use "msgcatdir." However, given that the
GNU Coding Standard uses "localedir", we will switch to that for
consistency as a GNU project. Previous uses of --localdir didn't work
anyway (see bug 14259).
I am committing this patch in the understanding that nobody would object
to fixing #14259 as part of aligning our variable usage to the GNU
Coding Standard.
Given that previous "localedir" uses were converted to "complocaledir"
by [1], we can now convert "msgcatdir" to "localedir" and complete the
transition. With an addition to config.make.in we also fix bug 14259 and
allow users to specify the locale dependent data directory with
"--localedir" at configure time. There is still no way to control at
configure time the location of the *compiled* locale directory.
Tested on x86_64 with no regressions.
Tested using "--localedir" to specify alternate locale dependent data
directory and verified with "make install DESTDIR=/tmp/glibc".
[1] 90fe682d30
Now we require Linux 3.2 or later kernel headers everywhere, the
configure test for <linux/fanotify.h> is obsolete; this patch removes
it.
Tested for x86_64.
* sysdeps/unix/sysv/linux/configure.ac (linux/fanotify.h): Do not
test for header.
* sysdeps/unix/sysv/linux/configure: Regenerated.
* config.h.in (HAVE_LINUX_FANOTIFY_H): Remove #undef.
* sysdeps/unix/sysv/linux/tst-fanotify.c [!HAVE_LINUX_FANOTIFY_H]:
Remove conditional code.
[HAVE_LINUX_FANOTIFY_H]: Make code unconditional.
In <https://sourceware.org/ml/libc-alpha/2016-01/msg00885.html> I
proposed a minimum Linux kernel version of 3.2 for glibc 2.24, since
Linux 2.6.32 has reached EOL.
In the discussion in February, some concerns were expressed about
compatibility with OpenVZ containers. It's not clear that these are
real issues, given OpenVZ backporting kernel features and faking the
kernel version for guest software, as discussed in
<https://sourceware.org/ml/libc-alpha/2016-02/msg00278.html>. It's
also not clear that supporting running GNU/Linux distributions from
late 2016 (at the earliest) on a kernel series from 2009 is a sensible
expectation. However, as an interim step, this patch increases the
requirement everywhere except x86 / x86_64 (since the controversy was
only about those architectures); the special caveats and settings can
easily be removed later when we're ready to increase the requirements
on x86 / x86_64 (and if someone would like to raise the issue on LWN
as suggested in the previous discussion, that would be welcome). 3.2
kernel headers are required everywhere by this patch.
(x32 already requires 3.4 or later, so is unaffected by this patch.)
As usual for such a change, this patch only changes the configure
scripts and associated documentation. The intent is to follow up with
removal of dead __LINUX_KERNEL_VERSION conditionals. Each __ASSUME_*
or other macro that becomes dead can then be removed independently.
Tested for x86_64 and x86.
* sysdeps/unix/sysv/linux/configure.ac (LIBC_LINUX_VERSION):
Define to 3.2.0.
(arch_minimum_kernel): Likewise.
* sysdeps/unix/sysv/linux/configure: Regenerated.
* sysdeps/unix/sysv/linux/i386/configure.ac (arch_minimum_kernel):
Define to 2.6.32.
* sysdeps/unix/sysv/linux/i386/configure: Regenerated.
* sysdeps/unix/sysv/linux/x86_64/64/configure.ac
(arch_minimum_kernel): Define to 2.6.32.
* sysdeps/unix/sysv/linux/x86_64/64/configure: Regenerated.
* README: Document Linux 3.2 requirement.
* manual/install.texi (Linux): Document Linux 3.2 headers
requirement.
* INSTALL: Regenerated.
HAVE_BUILTIN_EXPECT macro was removed by commit glibc-2.14-280-g3ce1f29,
but then its use was unintentionally reintroduced during merge with GNU
gettext 0.19.3 by commit glibc-2.20-324-g6d24885, effectively disabling
all optimization based on __builtin_expect. As intl files are also part
of GNU gettext, HAVE_BUILTIN_EXPECT macro cannot be removed, so define
it unconditionally in config.h.in instead.
[BZ #19512]
* config.h.in (HAVE_BUILTIN_EXPECT): New macro.
The build of posix/tst-dir.c fails due to undefined DIAG_* macros.
The usage of the macros were introduced in recent commit
7584a3f96d
"Deprecate readdir_r, readdir64_r [BZ #19056]".
This patch adds the missing header libc-internal.h.
Due to GCC bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58066
__tls_get_addr may be called with 8-byte stack alignment. Although
this bug has been fixed in GCC 4.9.4, 5.3 and 6, we can't assume
that stack will be always aligned at 16 bytes. Since SSE optimized
memory/string functions with aligned SSE register load and store are
used in the dynamic linker, we must set DL_RUNTIME_UNALIGNED_VEC_SIZE
to 8 so that _dl_runtime_resolve_sse will align the stack before
calling _dl_fixup:
Dump of assembler code for function _dl_runtime_resolve_sse:
0x00007ffff7deea90 <+0>: push %rbx
0x00007ffff7deea91 <+1>: mov %rsp,%rbx
0x00007ffff7deea94 <+4>: and $0xfffffffffffffff0,%rsp
^^^^^^^^^^^ Align stack to 16 bytes
0x00007ffff7deea98 <+8>: sub $0x100,%rsp
0x00007ffff7deea9f <+15>: mov %rax,0xc0(%rsp)
0x00007ffff7deeaa7 <+23>: mov %rcx,0xc8(%rsp)
0x00007ffff7deeaaf <+31>: mov %rdx,0xd0(%rsp)
0x00007ffff7deeab7 <+39>: mov %rsi,0xd8(%rsp)
0x00007ffff7deeabf <+47>: mov %rdi,0xe0(%rsp)
0x00007ffff7deeac7 <+55>: mov %r8,0xe8(%rsp)
0x00007ffff7deeacf <+63>: mov %r9,0xf0(%rsp)
0x00007ffff7deead7 <+71>: movaps %xmm0,(%rsp)
0x00007ffff7deeadb <+75>: movaps %xmm1,0x10(%rsp)
0x00007ffff7deeae0 <+80>: movaps %xmm2,0x20(%rsp)
0x00007ffff7deeae5 <+85>: movaps %xmm3,0x30(%rsp)
0x00007ffff7deeaea <+90>: movaps %xmm4,0x40(%rsp)
0x00007ffff7deeaef <+95>: movaps %xmm5,0x50(%rsp)
0x00007ffff7deeaf4 <+100>: movaps %xmm6,0x60(%rsp)
0x00007ffff7deeaf9 <+105>: movaps %xmm7,0x70(%rsp)
[BZ #19679]
* sysdeps/x86_64/dl-trampoline.S (DL_RUNIME_UNALIGNED_VEC_SIZE):
Renamed to ...
(DL_RUNTIME_UNALIGNED_VEC_SIZE): This. Set to 8.
(DL_RUNIME_RESOLVE_REALIGN_STACK): Renamed to ...
(DL_RUNTIME_RESOLVE_REALIGN_STACK): This. Updated.
(DL_RUNIME_RESOLVE_REALIGN_STACK): Renamed to ...
(DL_RUNTIME_RESOLVE_REALIGN_STACK): This.
* sysdeps/x86_64/dl-trampoline.h
(DL_RUNIME_RESOLVE_REALIGN_STACK): Renamed to ...
(DL_RUNTIME_RESOLVE_REALIGN_STACK): This.
The following new 386 and X86_64 were added to binutils. They are
non-dynamic relocations, so don't need direct handling in glibc.
But other programs, like elfutils, use the glibc elf.h definitions
for the names and numbers when inspecting ET_REL files.
R_386_GOT32X was proposed in
https://groups.google.com/forum/#!topic/ia32-abi/GbJJskkid4I
X86_64_GOTPCRELX and R_X86_64_REX_GOTPCRELX were proposed in
https://groups.google.com/forum/#!topic/x86-64-abi/n9AWHogmVY0
There also used to be R_X86_64_PC32_BND and R_X86_64_PLT32_BND
but those already got deprecated in
https://groups.google.com/d/msg/x86-64-abi/-hdQyMixt8Y/XFDOvioG85cJ
* elf/elf.h (R_386_GOT32X): New.
(R_386_NUM): Update.
(R_X86_64_GOTPCRELX: New.
(R_X86_64_REX_GOTPCRELX): New.
(R_X86_64_NUM): Update.
The vast majority of timeouts I've seen w/glibc tests are due to:
- slow system (e.g. <1 GHz cpu)
- loaded system (e.g. lots of parallelism)
Even then, I've seen timeouts on system I don't generally consider
slow, or even loaded, and considering TIMEOUT is set to <=10 in ~60
tests (and <=20 in ~75 tests), it seems I'm not alone. I've just
gotten in the habit of doing `export TIMEOUTFACTOR=10` on all my
setups.
In the edge case where there is a bug in the test and the timeout is
hit, I think we all agree that's either a problem with the test or a
real bug in the library somewhere. In either case, the incident rate
should be low, so catering to that seems like the wrong trade-off.
Other developers too usually set large timeout factors. Increase the
default to 20 seconds to match reality.
From the bug:
Netherlands Antilles was dissolved, and "AN" is not a part of ISO 3166
anymore. According to setlocale(3), "territory is an ISO 3166 country
code". We now have pap_AW and pap_CW.
Reported-by: Chris Leonard <cjlhomeaddress@gmail.com>
This updates a bunch of locales based on CLDR v28 data:
ar_SS: int_prefix: changing 249 to 211
bn_BD: int_prefix: changing 88 to 880
dz_BT: int_prefix: changing 66 to 975
en_HK: int_prefix: changing to 852
en_PH: int_prefix: changing to 63
en_SG: int_prefix: changing to 65
es_DO: int_prefix: changing 1809 to 1
es_PA: int_prefix: changing 502 to 507
es_PR: int_prefix: changing 1787 to 1
km_KH: int_prefix: changing 856 to 855
mt_MT: int_prefix: changing to 356
ne_NP: int_prefix: changing 91 to 977
pap_AW: int_prefix: changing 599 to 297
the_NP: int_prefix: changing 91 to 977
tk_TM: int_prefix: changing to 993
uz_UZ: int_prefix: changing 27 to 998
zh_SG: int_prefix: changing to 65
I've also checked these against https://countrycode.org/.
Note: the Dominican Republic (DO) and Puerto Rico (PR) updates are
correct: they both use +1. Historically, DO had one area code of
809 and PR of 787 which is why they were listed as such, but they
have both expanded into 829 and 989 respectively, so using the four
digit value is def incorrect now.
The ldbl-128ibm implementation of nextafterl / nexttowardl returns -0
in FE_DOWNWARD mode when taking the next value below the least
positive subnormal, when it should return +0. This patch fixes it to
check explicitly for this case.
Tested for powerpc.
[BZ #19678]
* sysdeps/ieee754/ldbl-128ibm/s_nextafterl.c (__nextafterl):
Ensure +0.0 is returned when taking the next value below the least
positive value.
