Neither the <dlfcn.h> entry points, nor lazy symbol resolution, nor
initial shared library load-up, are cancellation points, so ld.so
should exclusively use I/O primitives that are not cancellable. We
currently achieve this by having the cancellation hooks compile as
no-ops when IS_IN(rtld); this patch changes to using exclusively
_nocancel primitives in the source code instead, which makes the
intent clearer and significantly reduces the amount of code compiled
under IS_IN(rtld) as well as IS_IN(libc) -- in particular,
elf/Makefile no longer thinks we require a copy of unwind.c in
rtld-libc.a. (The older mechanism is preserved as a backstop.)
The bulk of the change is splitting up the files that define the
_nocancel I/O functions, so they don't also define the variants that
*are* cancellation points; after which, the existing logic for picking
out the bits of libc that need to be recompiled as part of ld.so Just
Works. I did this for all of the _nocancel functions, not just the
ones used by ld.so, for consistency.
fcntl was a little tricky because it's only a cancellation point for
certain opcodes (F_SETLKW(64), which can block), and the existing
__fcntl_nocancel wasn't applying the FCNTL_ADJUST_CMD hook, which
strikes me as asking for trouble, especially as the only nontrivial
definition of FCNTL_ADJUST_CMD (for powerpc64) changes F_*LK* opcodes.
To fix this, fcntl_common moves to fcntl_nocancel.c along with
__fcntl_nocancel, and changes its name to the extern (but hidden)
symbol __fcntl_nocancel_adjusted, so that regular fcntl can continue
calling it. __fcntl_nocancel now applies FCNTL_ADJUST_CMD; so that
both both fcntl.c and fcntl_nocancel.c can see it, the only nontrivial
definition moves from sysdeps/u/s/l/powerpc/powerpc64/fcntl.c to
.../powerpc64/sysdep.h and becomes entirely a macro, instead of a macro
that calls an inline function.
The nptl version of libpthread also changes a little, because its
"compat-routines" formerly included files that defined all the
_nocancel functions it uses; instead of continuing to duplicate them,
I exported the relevant ones from libc.so as GLIBC_PRIVATE. Since the
Linux fcntl.c calls a function defined by fcntl_nocancel.c, it can no
longer be used from libpthread.so; instead, introduce a custom
forwarder, pt-fcntl.c, and export __libc_fcntl from libc.so as
GLIBC_PRIVATE. The nios2-linux ABI doesn't include a copy of vfork()
in libpthread, and it was handling that by manipulating
libpthread-routines in .../linux/nios2/Makefile; it is cleaner to do
what other such ports do, and have a pt-vfork.S that defines no symbols.
Right now, it appears that Hurd does not implement _nocancel I/O, so
sysdeps/generic/not-cancel.h will forward everything back to the
regular functions. This changed the names of some of the functions
that sysdeps/mach/hurd/dl-sysdep.c needs to interpose.
* elf/dl-load.c, elf/dl-misc.c, elf/dl-profile.c, elf/rtld.c
* sysdeps/unix/sysv/linux/dl-sysdep.c
Include not-cancel.h. Use __close_nocancel instead of __close,
__open64_nocancel instead of __open, __read_nocancel instead of
__libc_read, and __write_nocancel instead of __libc_write.
* csu/check_fds.c (check_one_fd)
* sysdeps/posix/fdopendir.c (__fdopendir)
* sysdeps/posix/opendir.c (__alloc_dir): Use __fcntl_nocancel
instead of __fcntl and/or __libc_fcntl.
* sysdeps/unix/sysv/linux/pthread_setname.c (pthread_setname_np)
* sysdeps/unix/sysv/linux/pthread_getname.c (pthread_getname_np)
* sysdeps/unix/sysv/linux/i386/smp.h (is_smp_system):
Use __open64_nocancel instead of __open_nocancel.
* sysdeps/unix/sysv/linux/not-cancel.h: Move all of the
hidden_proto declarations to the end and issue them if either
IS_IN(libc) or IS_IN(rtld).
* sysdeps/unix/sysv/linux/Makefile [subdir=io] (sysdep_routines):
Add close_nocancel, fcntl_nocancel, nanosleep_nocancel,
open_nocancel, open64_nocancel, openat_nocancel, pause_nocancel,
read_nocancel, waitpid_nocancel, write_nocancel.
* io/Versions [GLIBC_PRIVATE]: Add __libc_fcntl,
__fcntl_nocancel, __open64_nocancel, __write_nocancel.
* posix/Versions: Add __nanosleep_nocancel, __pause_nocancel.
* nptl/pt-fcntl.c: New file.
* nptl/Makefile (pthread-compat-wrappers): Remove fcntl.
(libpthread-routines): Add pt-fcntl.
* include/fcntl.h (__fcntl_nocancel_adjusted): New function.
(__libc_fcntl): Remove attribute_hidden.
* sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Call
__fcntl_nocancel_adjusted, not fcntl_common.
(__fcntl_nocancel): Move to new file fcntl_nocancel.c.
(fcntl_common): Rename to __fcntl_nocancel_adjusted; also move
to fcntl_nocancel.c.
* sysdeps/unix/sysv/linux/fcntl_nocancel.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/fcntl.c: Remove file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h:
Define FCNTL_ADJUST_CMD here, as a self-contained macro.
* sysdeps/unix/sysv/linux/close.c: Move __close_nocancel to...
* sysdeps/unix/sysv/linux/close_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/nanosleep.c: Move __nanosleep_nocancel to...
* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/open.c: Move __open_nocancel to...
* sysdeps/unix/sysv/linux/open_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/open64.c: Move __open64_nocancel to...
* sysdeps/unix/sysv/linux/open64_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/openat.c: Move __openat_nocancel to...
* sysdeps/unix/sysv/linux/openat_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/openat64.c: Move __openat64_nocancel to...
* sysdeps/unix/sysv/linux/openat64_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/pause.c: Move __pause_nocancel to...
* sysdeps/unix/sysv/linux/pause_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/read.c: Move __read_nocancel to...
* sysdeps/unix/sysv/linux/read_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/waitpid.c: Move __waitpid_nocancel to...
* sysdeps/unix/sysv/linux/waitpid_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/write.c: Move __write_nocancel to...
* sysdeps/unix/sysv/linux/write_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/nios2/Makefile: Don't override
libpthread-routines.
* sysdeps/unix/sysv/linux/nios2/pt-vfork.S: New file which
defines nothing.
* sysdeps/mach/hurd/dl-sysdep.c: Define __read instead of
__libc_read, and __write instead of __libc_write. Define
__open64 in addition to __open.
This patch consolidates all the non cancellable openat{64} calls to use
the __openat{64}_nocancel identifier. For non cancellable targets it will
be just a macro to call the default respective symbol while on Linux
will be a internal one.
Checked on x86_64-linux-gnu, x86_64-linux-gnu-x32, and i686-linux-gnu.
* sysdeps/generic/not-cancel.h (openat_not_cancel): Remove macro.
(openat_not_cancel_3): Likewise.
(openat64_not_cancel_3): Likewise).
(openat_not_cancel_3): Likewise).
* sysdeps/unix/sysv/linux/not-cancel.h (openat_not_cancel): Remove
macro.
(openat_not_cancel_3): Likewise.
(openat64_not_cancel): Likewise.
(openat64_not_cancel_3): Likewise.
* sysdeps/unix/sysv/linux/openat.c (__openat_nocancel): New function.
* sysdeps/unix/sysv/linux/openat64.c (__openat64_nocancel): Likewise.
* io/ftw.c (open_dir_stream): Replace openat{64}_not_cancel{_3} with
__open{64}_nocancel.
* sysdeps/mach/hurd/opendir.c (__opendirat): Likewise.
* sysdeps/posix/getcwd.c (__getcwd): Likewise.
* sysdeps/posix/opendir.c (__opendirat): Likewise.
This patch consolidates the open Linux syscall implementation on
sysdeps/unix/sysv/linux/open{64}.c. The changes are:
1. Remove wordsize-64 openat{64}.
2. For architetures that define __OFF_T_MATCHES_OFF64_T openat64
will be default one with alias to required symbols. Otherwise
openat64 will pass the required O_LARGEFILE flag on syscall.
Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32,
arch64-linux-gnu, arm-linux-gnueabihf, and powerpc64le-linux-gnu.
* sysdeps/unix/sysv/linux/openat.c (__libc_openat): Build only
for !__OFF_T_MATCHES_OFF64_T.
* sysdeps/unix/sysv/linux/openat64.c (__libc_openat64): New
implementation based on open64.
* sysdeps/unix/sysv/linux/wordsize-64/openat.c: Remove file.
* sysdeps/unix/sysv/linux/wordsize-64/openat64.c: Likewise.
Both open and openat load their last argument 'mode' lazily, using
va_arg() only if O_CREAT is found in oflag. This is wrong, mode is also
necessary if O_TMPFILE is in oflag.
By chance on x86_64, the problem wasn't evident when using O_TMPFILE
with open, as the 3rd argument of open, even when not loaded with
va_arg, is left untouched in RDX, where the syscall expects it.
However, openat was not so lucky, and O_TMPFILE couldn't be used: mode
is the 4th argument, in RCX, but the syscall expects its 4th argument in
a different register than the glibc wrapper, in R10.
Introduce a macro __OPEN_NEEDS_MODE (oflag) to test if either O_CREAT or
O_TMPFILE is set in oflag.
Tested on Linux x86_64.
[BZ #17523]
* io/fcntl.h (__OPEN_NEEDS_MODE): New macro.
* io/bits/fcntl2.h (open): Use it.
(openat): Likewise.
* io/open.c (__libc_open): Likewise.
* io/open64.c (__libc_open64): Likewise.
* io/open64_2.c (__open64_2): Likewise.
* io/open_2.c (__open_2): Likewise.
* io/openat.c (__openat): Likewise.
* io/openat64.c (__openat64): Likewise.
* io/openat64_2.c (__openat64_2): Likewise.
* io/openat_2.c (__openat_2): Likewise.
* sysdeps/mach/hurd/open.c (__libc_open): Likewise.
* sysdeps/mach/hurd/openat.c (__openat): Likewise.
* sysdeps/posix/open64.c (__libc_open64): Likewise.
* sysdeps/unix/sysv/linux/dl-openat64.c (openat64): Likewise.
* sysdeps/unix/sysv/linux/generic/open.c (__libc_open): Likewise.
(__open_nocancel): Likewise.
* sysdeps/unix/sysv/linux/generic/open64.c (__libc_open64): Likewise.
* sysdeps/unix/sysv/linux/open64.c (__libc_open64): Likewise.
* sysdeps/unix/sysv/linux/openat.c (__OPENAT): Likewise.
This patch cleans up for __ASSUME_ATFCTS now always being true for the
supported Linux kernel versions by removing conditional code in
sysdeps/unix/sysv/linux. Several fchownat.c files that were only
present because of differences in the fallback syscalls used
(depending on the architecture-specific names of chown-related
syscalls for 32-bit uids) are removed. Files that looks like they
could be replaced by syscalls.list entries have the standard "Consider
moving to syscalls.list." comment (see bug 14138) added. Conditionals
on the relevant __NR_* syscall numbers being defined are also removed,
since my analysis indicated that the relevant syscalls are always
defined for all relevant kernel versions using any affected file.
Much of the removed fallback code had unbounded stack allocations, so
this reduces the number of cases to consider for anyone reviewing uses
of alloca and VLAs in glibc.
There remain tests of __ASSUME_ATFCTS in io/openat.c (to determine
whether to define __have_atfcts) and sysdeps/posix/getcwd.c (which
also uses __have_atfcts); thus, the definition of __ASSUME_ATFCTS
remains in kernel-features.h. The logical condition relevant there is
whether openat64_not_cancel_3 is known to work. Hurd doesn't use this
version of getcwd at all, so the conditionals in getcwd.c are always
true in glibc. However, this code is also used in gnulib. So the
best way to deal with the conditionals there may be for gnulib people
to deal with merging all relevant changes in both directions between
the glibc and gnulib versions of this file, at the end of which the
openat conditionals should be in whatever form is best for gnulib, and
hardcoded in the _LIBC case to having openat supported.
Tested by comparing before-and-after disassembly of installed
(stripped) shared libraries, on x86_64 and x86. On x86 the patch made
no change to the disassembly; on x86_64, the only changes were in
readlinkat, where formerly the return value from the readlinkat
syscall was stored in an int variable before being converted to
ssize_t for the return, and now the return value is returned directly
without truncation to int. I think it's clearly correct not to
truncate the return value (although I also think the truncation would
not have been a user-visible bug because the kernel would never have
returned a value it could have affected).
* include/fcntl.h (__atfct_seterrno): Remove prototype.
(__atfct_seterrno_2): Likewise.
* sysdeps/unix/sysv/linux/alpha/dl-fxstatat64.c: Do not include
<kernel-features.h>.
(__ASSUME_ATFCTS): Do not undefine and redefine.
* sysdeps/unix/sysv/linux/alpha/fxstatat.c [__ASSUME_ATFCTS]
(__have_atfcts): Remove conditional definition.
(__fxstatat([__NR_fstatat64]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code and code
unreachable if [__ASSUME_ATFCTS].
* sysdeps/unix/sysv/linux/dl-fxstatat64.c (__ASSUME_ATFCTS): Do
not undefine and redefine.
* sysdeps/unix/sysv/linux/faccessat.c: Do not include
<kernel-features.h>.
(faccessat) [__NR_faccessat]: Make code unconditional.
(faccessat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fchmodat.c: Do not include
<kernel-features.h>.
(fchmodat) [__NR_fchmodat]: Make code unconditional.
(fchmodat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fchownat.c: Do not include
<kernel-features.h>.
(fchownat) [__NR_fchownat]: Make code unconditional.
(fchownat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/futimesat.c: Do not include
<kernel-features.h>.
(futimesat) [__NR_futimesat]: Make code unconditional.
(futimesat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fxstatat.c: Do not include
<kernel-features.h>.
(__fxstatat) [__NR_newfstatat]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fxstatat64.c: Do not include
<kernel-features.h>.
(__fxstatat64) [__NR_fstatat64]: Make code unconditional.
(__fxstatat64) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/i386/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/i386/fxstatat.c: Do not include
<kernel-features.h>.
(__fxstatat) [__NR_fstatat64]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/linkat.c: Do not include
<kernel-features.h>.
(linkat) [__NR_linkat]: Make code unconditional.
(linkat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/m68k/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/mips/mips64/fxstatat64.c: Do not include
<kernel-features.h>.
(__fxstatat64) [__NR_newfstatat]: Make code unconditional.
(__fxstatat64) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/mkdirat.c: Do not include
<kernel-features.h>.
(mkdirat) [__NR_mkdirat]: Make code unconditional.
(mkdirat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/openat.c: Do not include
<kernel-features.h>.
[!__ASSUME_ATFCTS] (__atfct_seterrno): Remove function.
[!__ASSUME_ATFCTS] (__have_atfcts): Remove variable.
(OPENAT_NOT_CANCEL) [__NR_openat]: Make code unconditional.
(OPENAT_NOT_CANCEL) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/powerpc/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/readlinkat.c: Do not include
<kernel-features.h>.
(readlinkat) [__NR_readlinkat]: Make code unconditional.
(readlinkat) [!__ASSUME_ATFCTS]: Remove conditional code. Return
result of INLINE_SYSCALL directly, not via int variable.
* sysdeps/unix/sysv/linux/renameat.c: Do not include
<kernel-features.h>.
[!__ASSUME_ATFCTS] (__atfct_seterrno_2): Remove function.
(renameat) [__NR_renameat]: Make code unconditional.
(renameat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/s390/s390-32/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/sh/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/sparc/sparc32/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/sparc/sparc64/dl-fxstatat64.c
(__ASSUME_ATFCTS): Do not undefine and redefine.
* sysdeps/unix/sysv/linux/symlinkat.c: Do not include
<kernel-features.h>.
(symlinkat) [__NR_symlinkat]: Make code unconditional.
(symlinkat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/unlinkat.c: Do not include
<kernel-features.h>.
(unlinkat) [__NR_unlinkat]: Make code unconditional.
(unlinkat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/wordsize-64/dl-fxstatat64.c
(__ASSUME_ATFCTS): Do not undefine and redefine.
* sysdeps/unix/sysv/linux/wordsize-64/fxstatat.c: Do not include
<kernel-features.h>.
(__fxstatat) [__NR_newfstatat]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/xmknodat.c: Do not include
<kernel-features.h>.
(__xmknodat) [__NR_mknodat]: Make code unconditional.
(__xmknodat) [!__ASSUME_ATFCTS]: Remove conditional code.
* io/openat.c (__openat): Define instead of openat.
Use libc_hidden_def. Define openat with weak_alias.
* io/openat64.c (__openat64): Define instead of openat64.
Use libc_hidden_def. Define openat64 with weak_alias.
* sysdeps/unix/sysv/linux/openat.c: Likewise.
* libio/genops.c: Include <sched.h> for __sched_yield decl.
fchownat, and unlinkat.
(static-only-routines): Add fstatat and fstatat64.
(tests): Add tst-unlinkat, tst-fstatat, tst-futimesat, tst-renameat,
and tst-fchownat.
* io/Versions [GLIBC_2.4]: Add fchownat, __fxstatat, __fxstatat64,
and unlinkat.
* io/fcntl.h: Define AT_FDCWD, AT_SYMLINK_NOFOLLOW, and AT_REMOVEDIR.
* io/fstatat.c: New file.
* io/fstatat64.c: New file.
* io/sys/stat.h: Declare fstatat, fstatat64, __fxstatat, __fxstatat64
and define fstatat and fstatat64 inline functions.
* libio/stdio.h: Declare renameat.
* misc/Makefile (routines): Add futimesat.
* misc/Versions [GLIBC_2.4]: Add futimesat.
* posix/unistd.h: Declare fchownat and unlinkat.
* stdio-common/Makefile (routines): Add renameat.
* stdio-common/Versions [GLIBC_2.4]: Add renameat.
* sysdeps/generic/fchownat.c: New file.
* sysdeps/generic/futimesat.c: New file.
* sysdeps/generic/fxstatat.c: New file.
* sysdeps/generic/fxstatat64.c: New file.
* sysdeps/generic/renameat.c: New file.
* sysdeps/generic/unlinkat.c: New file.
* sysdeps/unix/sysv/linux/fchownat.c: New file.
* sysdeps/unix/sysv/linux/futimesat.c: New file.
* sysdeps/unix/sysv/linux/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/fxstatat64.c: New file.
* sysdeps/unix/sysv/linux/renameat.c: New file.
* sysdeps/unix/sysv/linux/unlinkat.c: New file.
* sysdeps/unix/sysv/linux/alpha/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/i386/fchownat.c: New file.
* sysdeps/unix/sysv/linux/i386/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/m68k/fchownat.c: New file.
* sysdeps/unix/sysv/linux/m68k/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/powerpc/fchownat.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/s390/s390-32/fchownat.c: New file.
* sysdeps/unix/sysv/linux/s390/s390-32/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/sh/fchownat.c: New file.
* sysdeps/unix/sysv/linux/sh/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/fchownat.c: New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/wordsize-64/fxstatat.c: New file.
* sysdeps/unix/sysv/linux/wordsize-64/fxstatat64.c: New file.
* time/sys/time.h: Declare futimesat.
* include/fcntl.h: Declare __atfct_seterrno.
* sysdeps/unix/sysv/linux/openat.c (__atfct_seterrno): New function.
Split out from openat code and called.
(do_test): Allow openat to fail with ENOSYS. Handle AT_FDCWD.
* sysdeps/generic/openat.c: Handle AT_FDCWD.
* sysdeps/generic/openat64.c: Likewise.
* io/tst-unlinkat.c: New file.
* io/tst-fstatat.c: New file.
* io/tst-futimesat.c: New file.
* io/tst-renameat.c: New file.
* io/tst-fchownat.c: New file.
* io/tst-openat.c: Don't fail if openat is not implemented.
* sysdeps/unix/sysv/linux/powerpc/chown.c: Don't provide backward
compatibility code if new kernel is guaranteed.
* time/sys/time.h: Add a few nonnull attributes.
* io/Makefile (routines): Add openat and openat64.
(tests): Add tst-openat.
* io/Versions: Add openat and openat64 for GLIBC_2.4.
* sysdeps/generic/openat.c: New file.
* sysdeps/generic/openat64.c: New file.
* sysdeps/unix/sysv/linux/openat.c: New file.
* sysdeps/unix/sysv/linux/openat64.c: New file.
* io/tst-openat.c: New file.
* sysdeps/unix/sysv/linux/device-nrs.h: Add info for /dev/full.
