getaddrinfo doesn't look for any RESOLVER defines for conditional
compilation. Therefore, remove the unnecessary -DRESOLVER build flag in
getaddrinfo's CFLAGS.
Checked on x86_64 for code generation changes; none found.
Returning a pidfd allows a process to keep a race-free handle for a
child process, otherwise, the caller will need to either use pidfd_open
(which still might be subject to TOCTOU) or keep the old racy interface
base on pid_t.
To correct use pifd_spawn, the kernel must support not only returning
the pidfd with clone/clone3 but also waitid (P_PIDFD) (added on Linux
5.4). If kernel does not support the waitid, pidfd return ENOSYS.
It avoids the need to racy workarounds, such as reading the procfs
fdinfo to get the pid to use along with other wait interfaces.
These interfaces are similar to the posix_spawn and posix_spawnp, with
the only difference being it returns a process file descriptor (int)
instead of a process ID (pid_t). Their prototypes are:
int pidfd_spawn (int *restrict pidfd,
const char *restrict file,
const posix_spawn_file_actions_t *restrict facts,
const posix_spawnattr_t *restrict attrp,
char *const argv[restrict],
char *const envp[restrict])
int pidfd_spawnp (int *restrict pidfd,
const char *restrict path,
const posix_spawn_file_actions_t *restrict facts,
const posix_spawnattr_t *restrict attrp,
char *const argv[restrict_arr],
char *const envp[restrict_arr]);
A new symbol is used instead of a posix_spawn extension to avoid
possible issues with language bindings that might track the return
argument lifetime. Although on Linux pid_t and int are interchangeable,
POSIX only states that pid_t should be a signed integer.
Both symbols reuse the posix_spawn posix_spawn_file_actions_t and
posix_spawnattr_t, to void rehash posix_spawn API or add a new one. It
also means that both interfaces support the same attribute and file
actions, and a new flag or file action on posix_spawn is also added
automatically for pidfd_spawn.
Also, using posix_spawn plumbing allows the reusing of most of the
current testing with some changes:
- waitid is used instead of waitpid since it is a more generic
interface.
- tst-posix_spawn-setsid.c is adapted to take into consideration that
the caller can check for session id directly. The test now spawns
itself and writes the session id as a file instead.
- tst-spawn3.c need to know where pidfd_spawn is used so it keeps an
extra file description unused.
Checked on x86_64-linux-gnu on Linux 4.15 (no CLONE_PIDFD or waitid
support), Linux 5.4 (full support), and Linux 6.2.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
These functions allow to posix_spawn and posix_spawnp to use
CLONE_INTO_CGROUP with clone3, allowing the child process to
be created in a different cgroup version 2. These are GNU
extensions that are available only for Linux, and also only
for the architectures that implement clone3 wrapper
(HAVE_CLONE3_WRAPPER).
To create a process on a different cgroupv2, one can use the:
posix_spawnattr_t attr;
posix_spawnattr_init (&attr);
posix_spawnattr_setflags (&attr, POSIX_SPAWN_SETCGROUP);
posix_spawnattr_setcgroup_np (&attr, cgroup);
posix_spawn (...)
Similar to other posix_spawn flags, POSIX_SPAWN_SETCGROUP control
whether the cgroup file descriptor will be used or not with
clone3.
There is no fallback if either clone3 does not support the flag
or if the architecture does not provide the clone3 wrapper, in
this case posix_spawn returns EOPNOTSUPP.
Checked on x86_64-linux-gnu.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
This change is similar to what was done for bits/wchar2.h.
Routines declaration are moved into a dedicated bits/unistd-decl.h file
which is then included into the bits/unistd.h file.
This will allow to adapt the files so that PLT entries are not created when
_FORTIFY_SOURCE is enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Since the _FORTIFY_SOURCE feature uses some routines of Glibc, they need to
be excluded from the fortification.
On top of that:
- some tests explicitly verify that some level of fortification works
appropriately, we therefore shouldn't modify the level set for them.
- some objects need to be build with optimization disabled, which
prevents _FORTIFY_SOURCE to be used for them.
Assembler files that implement architecture specific versions of the
fortified routines were not excluded from _FORTIFY_SOURCE as there is no
C header included that would impact their behavior.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Fix LOCALE list formatting.
Sort all reflowed text using scripts/sort-makefile-lines.py.
No code generation changes observed in binary artifacts.
No regressions on x86_64 and i686.
The clone3 flag resets all signal handlers of the child not set to
SIG_IGN to SIG_DFL. It allows to skip most of the sigaction calls
to setup child signal handling, where previously a posix_spawn
had to issue 2 times NSIG sigaction calls (one to obtain the current
disposition and another to set either SIG_DFL or SIG_IGN).
With POSIX_SPAWN_SETSIGDEF the child will setup the signal for the case
where the disposition is SIG_IGN.
The code must handle the fallback where clone3 is not available. This is
done by splitting __clone_internal_fallback from __clone_internal.
Checked on x86_64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
clang emits an warning when a double alias redirection is used, to warn
the the original symbol will be used even when weak definition is
overridden. However, this is a common pattern for weak_alias, where
multiple alias are set to same symbol.
Reviewed-by: Fangrui Song <maskray@google.com>
It was added on Linux 5.10 (ecb8ac8b1f146915aa6b96449b66dd48984caacc)
with the same functionality as madvise but using a pidfd of the target
process.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
It was added on Linux 5.4 (3695eae5fee0605f316fbaad0b9e3de791d7dfaf)
to extend waitid to wait on pidfd.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
The posix_spawnattr_tcsetpgrp_np works on a file descriptor (the
controlling terminal), so it would make more sense to actually fit
it on the file actions API.
Also, POSIX_SPAWN_TCSETPGROUP is not really required since it is
implicit by the presence of tcsetpgrp file action.
The posix/tst-spawn6.c is also fixed when TTY can is not present.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Currently there is no proper way to set the controlling terminal through
posix_spawn in race free manner [1]. This forces shell implementations
to keep using fork+exec when launching background process groups,
even when using posix_spawn yields better performance.
This patch adds a new GNU extension so the creating process can
configure the created process terminal group. This is done with a new
flag, POSIX_SPAWN_TCSETPGROUP, along with two new attribute functions:
posix_spawnattr_tcsetpgrp_np, and posix_spawnattr_tcgetpgrp_np.
The function sets a new attribute, spawn-tcgroupfd, that references to
the controlling terminal.
The controlling terminal is set after the spawn-pgroup attribute, and
uses the spawn-tcgroupfd along with current creating process group
(so it is composable with POSIX_SPAWN_SETPGROUP).
To create a process and set the controlling terminal, one can use the
following sequence:
posix_spawnattr_t attr;
posix_spawnattr_init (&attr);
posix_spawnattr_setflags (&attr, POSIX_SPAWN_TCSETPGROUP);
posix_spawnattr_tcsetpgrp_np (&attr, tcfd);
If the idea is also to create a new process groups:
posix_spawnattr_t attr;
posix_spawnattr_init (&attr);
posix_spawnattr_setflags (&attr, POSIX_SPAWN_TCSETPGROUP
| POSIX_SPAWN_SETPGROUP);
posix_spawnattr_tcsetpgrp_np (&attr, tcfd);
posix_spawnattr_setpgroup (&attr, 0);
The controlling terminal file descriptor is ignored if the new flag is
not set.
This interface is slight different than the one provided by QNX [2],
which only provides the POSIX_SPAWN_TCSETPGROUP flag. The QNX
documentation does not specify how the controlling terminal is obtained
nor how it iteracts with POSIX_SPAWN_SETPGROUP. Since a glibc
implementation is library based, it is more straightforward and avoid
requires additional file descriptor operations to request the caller
to setup the controlling terminal file descriptor (and it also allows
a bit less error handling by posix_spawn).
Checked on x86_64-linux-gnu and i686-linux-gnu.
[1] https://github.com/ksh93/ksh/issues/79
[2] https://www.qnx.com/developers/docs/7.0.0/index.html#com.qnx.doc.neutrino.lib_ref/topic/p/posix_spawn.html
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
When fnmatch processes a bracket expression, and eventually finds it to be
unterminated, it should rescan it, treating the starting bracket as a
normal character. That didn't happen when a matching character was found
while scanning the bracket expression.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
This patch simplifies the memory allocation code and uses the sched
routines instead of reimplement it. This still uses a stack
allocation buffer, so it can be used on malloc initialization code.
Linux currently supports at maximum of 4096 cpus for most architectures:
$ find -iname Kconfig | xargs git grep -A10 -w NR_CPUS | grep -w range
arch/alpha/Kconfig- range 2 32
arch/arc/Kconfig- range 2 4096
arch/arm/Kconfig- range 2 16 if DEBUG_KMAP_LOCAL
arch/arm/Kconfig- range 2 32 if !DEBUG_KMAP_LOCAL
arch/arm64/Kconfig- range 2 4096
arch/csky/Kconfig- range 2 32
arch/hexagon/Kconfig- range 2 6 if SMP
arch/ia64/Kconfig- range 2 4096
arch/mips/Kconfig- range 2 256
arch/openrisc/Kconfig- range 2 32
arch/parisc/Kconfig- range 2 32
arch/riscv/Kconfig- range 2 32
arch/s390/Kconfig- range 2 512
arch/sh/Kconfig- range 2 32
arch/sparc/Kconfig- range 2 32 if SPARC32
arch/sparc/Kconfig- range 2 4096 if SPARC64
arch/um/Kconfig- range 1 1
arch/x86/Kconfig-# [NR_CPUS_RANGE_BEGIN ... NR_CPUS_RANGE_END] range.
arch/x86/Kconfig- range NR_CPUS_RANGE_BEGIN NR_CPUS_RANGE_END
arch/xtensa/Kconfig- range 2 32
With x86 supporting 8192:
arch/x86/Kconfig
976 config NR_CPUS_RANGE_END
977 int
978 depends on X86_64
979 default 8192 if SMP && CPUMASK_OFFSTACK
980 default 512 if SMP && !CPUMASK_OFFSTACK
981 default 1 if !SMP
So using a maximum of 32k cpu should cover all cases (and I would
expect once we start to have many more CPUs that Linux would provide
a more straightforward way to query for such information).
A test is added to check if sched_getaffinity can successfully return
with large buffers.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
We add a new C.UTF-8 locale. This locale is not builtin to glibc, but
is provided as a distinct locale. The locale provides full support for
UTF-8 and this includes full code point sorting via STRCMP-based
collation (strcmp or wcscmp).
The collation uses a new keyword 'codepoint_collation' which drops all
collation rules and generates an empty zero rules collation to enable
STRCMP usage in collation. This ensures that we get full code point
sorting for C.UTF-8 with a minimal 1406 bytes of overhead (LC_COLLATE
structure information and ASCII collating tables).
The new locale is added to SUPPORTED. Minimal test data for specific
code points (minus those not supported by collate-test) is provided in
C.UTF-8.in, and this verifies code point sorting is working reasonably
across the range. The locale was tested manually with the full set of
code points without failure.
The locale is harmonized with locales already shipping in various
downstream distributions. A new tst-iconv9 test is added which verifies
the C.UTF-8 locale is generally usable.
Testing for fnmatch, regexec, and recomp is provided by extending
bug-regex1, bugregex19, bug-regex4, bug-regex6, transbug, tst-fnmatch,
tst-regcomp-truncated, and tst-regex to use C.UTF-8.
Tested on x86_64 or i686 without regression.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Split the mcheck implementation into the debugging hooks and API so
that the API can be replicated in libc and libc_malloc_debug.so. The
libc APIs always result in failure.
The mcheck implementation has also been moved entirely into
libc_malloc_debug.so and with it, all of the hook initialization code
can now be moved into the debug library. Now the initialization can
be done independently of libc internals.
With this patch, libc_malloc_debug.so can no longer be used with older
libcs, which is not its goal anyway. tst-vfork3 breaks due to this
since it spawns shell scripts, which in turn execute using the system
glibc. Move the test to tests-container so that only the built glibc
is used.
This move also fixes bugs in the mcheck version of memalign and
realloc, thus allowing removal of the tests from tests-mcheck
exclusion list.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so. With this, the hooks now no
longer have any effect on the core library.
libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.
Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.
The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.
Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
This patch adds a way to close a range of file descriptors on
posix_spawn as a new file action. The API is similar to the one
provided by Solaris 11 [1], where the file action causes the all open
file descriptors greater than or equal to input on to be closed when
the new process is spawned.
The function posix_spawn_file_actions_addclosefrom_np is safe to be
implemented by iterating over /proc/self/fd, since the Linux spawni.c
helper process does not use CLONE_FILES, so its has own file descriptor
table and any failure (in /proc operation) aborts the process creation
and returns an error to the caller.
I am aware that this file action might be redundant to the current
approach of POSIX in promoting O_CLOEXEC in more interfaces. However
O_CLOEXEC is still not the default and for some specific usages, the
caller needs to close all possible file descriptors to avoid them
leaking. Some examples are CPython (discussed in BZ#10353) and OpenJDK
jspawnhelper [2] (where OpenJDK spawns a helper process to exactly
closes all file descriptors). Most likely any environment which calls
functions that might open file descriptor under the hood and aim to use
posix_spawn might face the same requirement.
Checked on x86_64-linux-gnu and i686-linux-gnu on kernel 5.11 and 4.15.
[1] https://docs.oracle.com/cd/E36784_01/html/E36874/posix-spawn-file-actions-addclosefrom-np-3c.html
[2] https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/childproc.c#L82
Austin Group issue 62 [1] dropped the async-signal-safe requirement
for fork and provided a async-signal-safe _Fork replacement that
does not run the atfork handlers. It will be included in the next
POSIX standard.
It allow to close a long standing issue to make fork AS-safe (BZ#4737).
As indicated on the bug, besides the internal lock for the atfork
handlers itself; there is no guarantee that the handlers itself will
not introduce more AS-safe issues.
The idea is synchronize fork with the required internal locks to allow
children in multithread processes to use mostly of standard function
(even though POSIX states only AS-safe function should be used). On
signal handles, _Fork should be used intead and only AS-safe functions
should be used.
For testing, the new tst-_Fork only check basic usage. I also added
a new tst-mallocfork3 which uses the same strategy to check for
deadlock of tst-mallocfork2 but using threads instead of subprocesses
(and it does deadlock if it replaces _Fork with fork).
[1] https://austingroupbugs.net/view.php?id=62
The Linux nptl implementation is used as base for generic fork
implementation to handle the internal locks and mutexes. The
system specific bits are moved a new internal _Fork symbol.
(This new implementation will be used to provide a async-signal-safe
_Fork now that POSIX has clarified that fork might not be
async-signal-safe [1]).
For Hurd it means that the __nss_database_fork_prepare_parent and
__nss_database_fork_subprocess will be run in a slight different
order.
[1] https://austingroupbugs.net/view.php?id=62
The glob might pass a different stat struct for gl_stat and gl_lstat
when GLOB_ALTDIRFUNC is used. This requires add a new 64-bit time
version that also uses 64-bit time stat functions.
Checked on i686-linux-gnu and x86_64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
It operates similar to execve and it is is already used to implement
fexecve without requiring /proc to be mounted. However, different
than fexecve, if the syscall is not supported by the kernel an error
is returned instead of trying a fallback.
Checked on x86_64-linux-gnu and powerpc64le-linux-gnu.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The tst-wait4 is moved to common file and used for wait3
tests.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Both htl and nptl uses a different data structure to implement atfork
handlers. The nptl one was refactored by 27761a1042 to use a dynarray
which simplifies the code.
This patch moves the nptl one to be the generic implementation and
replace Hurd linked one. Different than previous NPTL, Hurd also uses
a global lock, so performance should be similar.
Checked on x86_64-linux-gnu, i686-linux-gnu, and with a build for
i686-gnu.
compat_symbol_reference now works for non-internal tests, too.
Also stop building and running the tests on those architectures
that lack the test symbol versions.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
compat_symbol_reference is now available for regular tests as well.
Also avoid building and running the tests in case the pre-2.27
symbol version of glob is not available. This avoids a spurious
UNSUPPORTED result.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This change adds new test to assess sched_rr_get_interval's
functionality.
To be more specific - following use cases are checked:
- If the read interval is correct
- If the proper ABI is used - to check if adjacent data is not
overwritten
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Previously, glibc would pick an arbitrary tmpfs file system from
/proc/mounts if /dev/shm was not available. This could lead to
an unsuitable file system being picked for the backing storage for
shm_open, sem_open, and related functions.
This patch introduces a new function, __shm_get_name, which builds
the file name under the appropriate (now hard-coded) directory. It is
called from the various shm_* and sem_* function. Unlike the
SHM_GET_NAME macro it replaces, the callers handle the return values
and errno updates. shm-directory.c is moved directly into the posix
subdirectory because it can be implemented directly using POSIX
functionality. It resides in libc because it is needed by both
librt and nptl/htl.
In the sem_open implementation, tmpfname is initialized directly
from a string constant. This happens to remove one alloca call.
Checked on x86_64-linux-gnu.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
Starting with commit 29fddfc7df, the
tests posix/bug-ga2 and resolv/tst-leaks2 are test-container tests.
If test-container.c returns with EXIT_UNSUPPORTED, the tests with
mtrace() are not executed and the mtrace files do not exist.
Therefore the "mtrace-analysis-part" of those tests are marked
UNSUPPORTED if the mtrace files are missing.
Reported-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
The tests posix/bug-ga2-mem and resolv/mtrace-tst-leaks2 are failing on
fedora 33 as mtrace reports memory leaks.
The /etc/nsswitch.conf differs between
Fedora 32: hosts: files dns myhostname
Fedora 33: hosts: files resolve [!UNAVAIL=return] myhostname dns
Therefore /lib64/libnss_resolve.so.2 (from systemd) and the dependencies
libgcc_s.so.1 and libpthread.so.0 are loaded.
Usually all malloc'ed resources from getaddrinfo / gethostbyname are freed
and the libraries are dlclose'd in nss/nsswitch.c:libc_freeres_fn (free_mem).
Unfortunately, /lib64/libnss_resolve.so.2 is marked with DF_1_NODELETE.
As this library is not unmapped, you'll see "Memory not freed".
Therefore those tests are now only relying on libnss_files.so by making
them test-container tests and providing the required configuration files.
By moving the tests to tests-container, those are now running with
"make check". Therefore the mtrace part of the tests are also moved
from "make xcheck" to "make check".
bug-ga2.c is now using test-driver.c in order to support WAIT_FOR_DEBUGGER
environment variable.
The variable idx contains the index into the extra array, whereas wextra
points into the extra array at this index, containing the length of the
following collating sequence in the wide character representation.
Once wordexp switches to posix_spawn, testing for command execution
based on fork handlers will not work anymore. Therefore, move these
subtests into a new test, posix/tst-wordexp-nocmd, which uses a
different form of command execution detection, based on PID
namespaces.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The stub implementations are turned into compat symbols.
Linux actually has two reserved system call numbers (for getpmsg
and putpmsg), but these system calls have never been implemented,
and there are no plans to implement them, so this patch replaces
the wrappers with the generic stubs.
According to <https://bugzilla.redhat.com/show_bug.cgi?id=436349>,
the presence of the XSI STREAMS declarations is a minor portability
hazard because they are not actually implemented.
This commit does not change the TIRPC support code in
sunrpc/rpc_svcout.c. It uses additional XTI functionality and
therefore never worked with glibc.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This commit adds gettid to <unistd.h> on Linux, and not to the
kernel-independent GNU API.
gettid is now supportable on Linux because too many things assume a
1:1 mapping between libpthread threads and kernel threads.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This fixes the same bug in fnmatch that was fixed by commit 7e2f0d2d77 for
regexp matching. As a side effect it also removes the use of an unbound
VLA.
Along with posix_spawn_file_actions_addchdir,
posix_spawn_file_actions_addfchdir is the subject of a change proposal
for POSIX: <http://austingroupbugs.net/view.php?id=1208>
These files were both auto-generated and shipped in the source tree.
We can assume that sed is available and always generate the files
during the build.
