Commit Graph

1622 Commits

Author SHA1 Message Date
Noah Goldstein
535e935a28 Replace {u}int_fast{16|32} with {u}int32_t
On 32-bit machines this has no affect. On 64-bit machines
{u}int_fast{16|32} are set as {u}int64_t which is often not
ideal. Particularly x86_64 this change both saves code size and
may save instruction cost.

Full xcheck passes on x86_64.
2022-04-13 21:23:04 -05:00
Adhemerval Zanella
f902892aaf posix: Remove unused variable on tst-_Fork.c
Checked on x86_64-linux-gnu and i686-linux-gnu.
2022-03-31 08:57:12 -03:00
Adhemerval Zanella
a9d3576572 posix: Fix tst-spawn6 terminal handling (BZ #28853)
The test changes the current foreground process group, which might
break testing depending of how the make check is issued.  For instance:

  nohup make -j1 test t=posix/tst-spawn6 | less

Will set 'make' and 'less' to be in the foreground process group in
the current session.  When tst-spawn6 new child takes over it becomes
the foreground process and 'less' is stopped and backgrounded which
interrupts the 'make check' command.

To fix it a pseudo-terminal is allocated, the test starts in new
session (so there is no controlling terminal associated), and the
pseudo-terminal is set as the controlling one (similar to what
login_tty does).

Checked on x86_64-linux-gnu.

Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2022-02-03 08:04:08 -03:00
Adhemerval Zanella
6289d28d3c posix: Replace posix_spawnattr_tc{get,set}pgrp_np with posix_spawn_file_actions_addtcsetpgrp_np
The posix_spawnattr_tcsetpgrp_np works on a file descriptor (the
controlling terminal), so it would make more sense to actually fit
it on the file actions API.

Also, POSIX_SPAWN_TCSETPGROUP is not really required since it is
implicit by the presence of tcsetpgrp file action.

The posix/tst-spawn6.c is also fixed when TTY can is not present.

Checked on x86_64-linux-gnu and i686-linux-gnu.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2022-02-02 08:34:16 -03:00
Adhemerval Zanella
342cc934a3 posix: Add terminal control setting support for posix_spawn
Currently there is no proper way to set the controlling terminal through
posix_spawn in race free manner [1].  This forces shell implementations
to keep using fork+exec when launching background process groups,
even when using posix_spawn yields better performance.

This patch adds a new GNU extension so the creating process can
configure the created process terminal group.  This is done with a new
flag, POSIX_SPAWN_TCSETPGROUP, along with two new attribute functions:
posix_spawnattr_tcsetpgrp_np, and posix_spawnattr_tcgetpgrp_np.
The function sets a new attribute, spawn-tcgroupfd, that references to
the controlling terminal.

The controlling terminal is set after the spawn-pgroup attribute, and
uses the spawn-tcgroupfd along with current creating process group
(so it is composable with POSIX_SPAWN_SETPGROUP).

To create a process and set the controlling terminal, one can use the
following sequence:

    posix_spawnattr_t attr;
    posix_spawnattr_init (&attr);
    posix_spawnattr_setflags (&attr, POSIX_SPAWN_TCSETPGROUP);
    posix_spawnattr_tcsetpgrp_np (&attr, tcfd);

If the idea is also to create a new process groups:

    posix_spawnattr_t attr;
    posix_spawnattr_init (&attr);
    posix_spawnattr_setflags (&attr, POSIX_SPAWN_TCSETPGROUP
				     | POSIX_SPAWN_SETPGROUP);
    posix_spawnattr_tcsetpgrp_np (&attr, tcfd);
    posix_spawnattr_setpgroup (&attr, 0);

The controlling terminal file descriptor is ignored if the new flag is
not set.

This interface is slight different than the one provided by QNX [2],
which only provides the POSIX_SPAWN_TCSETPGROUP flag.  The QNX
documentation does not specify how the controlling terminal is obtained
nor how it iteracts with POSIX_SPAWN_SETPGROUP.  Since a glibc
implementation is library based, it is more straightforward and avoid
requires additional file descriptor operations to request the caller
to setup the controlling terminal file descriptor (and it also allows
a bit less error handling by posix_spawn).

Checked on x86_64-linux-gnu and i686-linux-gnu.

[1] https://github.com/ksh93/ksh/issues/79
[2] https://www.qnx.com/developers/docs/7.0.0/index.html#com.qnx.doc.neutrino.lib_ref/topic/p/posix_spawn.html

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2022-01-25 14:07:53 -03:00
Andreas Schwab
8442f0d966 Fix handling of unterminated bracket expressions in fnmatch (bug 28792)
When fnmatch processes a bracket expression, and eventually finds it to be
unterminated, it should rescan it, treating the starting bracket as a
normal character.  That didn't happen when a matching character was found
while scanning the bracket expression.
2022-01-24 17:13:33 +01:00
Siddhesh Poyarekar
fcfc908681 debug: Synchronize feature guards in fortified functions [BZ #28746]
Some functions (e.g. stpcpy, pread64, etc.) had moved to POSIX in the
main headers as they got incorporated into the standard, but their
fortified variants remained under __USE_GNU.  As a result, these
functions did not get fortified when _GNU_SOURCE was not defined.

Add test wrappers that check all functions tested in tst-chk0 at all
levels with _GNU_SOURCE undefined and then use the failures to (1)
exclude checks for _GNU_SOURCE functions in these tests and (2) Fix
feature macro guards in the fortified function headers so that they're
the same as the ones in the main headers.

This fixes BZ #28746.

Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-01-12 23:34:48 +05:30
Paul Eggert
634b5ebac6 Update copyright dates not handled by scripts/update-copyrights.
I've updated copyright dates in glibc for 2022.  This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files.  As well as the usual annual
updates, mainly dates in --version output (minus csu/version.c which
previously had to be handled manually but is now successfully updated
by update-copyrights), there is a small change to the copyright notice
in NEWS which should let NEWS get updated automatically next year.

Please remember to include 2022 in the dates for any new files added
in future (which means updating any existing uncommitted patches you
have that add new files to use the new copyright dates in them).
2022-01-01 11:42:26 -08:00
Paul Eggert
581c785bf3 Update copyright dates with scripts/update-copyrights
I used these shell commands:

../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")

and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.

I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah.  I don't
know why I run into these diagnostics whereas others evidently do not.

remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
2022-01-01 11:40:24 -08:00
Paul Eggert
c52ef24829 regex: fix buffer read overrun in search [BZ#28470]
Problem reported by Benno Schulenberg in:
https://lists.gnu.org/r/bug-gnulib/2021-10/msg00035.html
* posix/regexec.c (re_search_internal): Use better bounds check.
2021-11-24 14:16:09 -08:00
Adhemerval Zanella
456b3c08b6 io: Refactor close_range and closefrom
Now that Hurd implementis both close_range and closefrom (f2c996597d),
we can make close_range() a base ABI, and make the default closefrom()
implementation on top of close_range().

The generic closefrom() implementation based on __getdtablesize() is
moved to generic close_range().  On Linux it will be overriden by
the auto-generation syscall while on Hurd it will be a system specific
implementation.

The closefrom() now calls close_range() and __closefrom_fallback().
Since on Hurd close_range() does not fail, __closefrom_fallback() is an
empty static inline function set by__ASSUME_CLOSE_RANGE.

The __ASSUME_CLOSE_RANGE also allows optimize Linux
__closefrom_fallback() implementation when --enable-kernel=5.9 or
higher is used.

Finally the Linux specific tst-close_range.c is moved to io and
enabled as default.  The Linuxism and CLOSE_RANGE_UNSHARE are
guarded so it can be built for Hurd (I have not actually test it).

Checked on x86_64-linux-gnu, i686-linux-gnu, and with a i686-gnu
build.
2021-11-24 09:09:37 -03:00
Fangrui Song
fdcd177fd3 regex: Unnest nested functions in regcomp.c
This refactor moves four functions out of a nested scope and converts
them into static always_inline functions. collseqwc, table_size,
symb_table, extra are now initialized to zero because they are passed as
function arguments.

On x86-64, .text is 16 byte larger likely due to the 4 stores.
This is nothing compared to the amount of work that regcomp has to do
looking up the collation weights, or other functions.

If the non-buildable `sysdeps/generic/dl-machine.h` doesn't count,
this patch removes the last `auto inline` usage from glibc.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2021-11-02 10:07:59 -07:00
Adhemerval Zanella
4e32c8f568 posix: Remove alloca usage for internal fnmatch implementation
This patch replaces the internal fnmatch pattern list generation
to use a dynamic array.

Checked on x86_64-linux-gnu.
2021-10-21 10:30:31 -03:00
Siddhesh Poyarekar
a643f60c53 Make sure that the fortified function conditionals are constant
In _FORTIFY_SOURCE=3, the size expression may be non-constant,
resulting in branches in the inline functions remaining intact and
causing a tiny overhead.  Clang (and in future, gcc) make sure that
the -1 case is always safe, i.e. any comparison of the generated
expression with (size_t)-1 is always false so that bit is taken care
of.  The rest is avoidable since we want the _chk variant whenever we
have a size expression and it's not -1.

Rework the conditionals in a uniform way to clearly indicate two
conditions at compile time:

- Either the size is unknown (-1) or we know at compile time that the
  operation length is less than the object size.  We can call the
  original function in this case.  It could be that either the length,
  object size or both are non-constant, but the compiler, through
  range analysis, is able to fold the *comparison* to a constant.

- The size and length are known and the compiler can see at compile
  time that operation length > object size.  This is valid grounds for
  a warning at compile time, followed by emitting the _chk variant.

For everything else, emit the _chk variant.

This simplifies most of the fortified function implementations and at
the same time, ensures that only one call from _chk or the regular
function is emitted.

Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-10-20 18:12:41 +05:30
Siddhesh Poyarekar
e938c02748 Don't add access size hints to fortifiable functions
In the context of a function definition, the size hints imply that the
size of an object pointed to by one parameter is another parameter.
This doesn't make sense for the fortified versions of the functions
since that's the bit it's trying to validate.

This is harmless with __builtin_object_size since it has fairly simple
semantics when it comes to objects passed as function parameters.
With __builtin_dynamic_object_size we could (as my patchset for gcc[1]
already does) use the access attribute to determine the object size in
the general case but it misleads the fortified functions.

Basically the problem occurs when access attributes are present on
regular functions that have inline fortified definitions to generate
_chk variants; the attributes get inherited by these definitions,
causing problems when analyzing them.  For example with poll(fds, nfds,
timeout), nfds is hinted using the __attr_access as being the size of
fds.

Now, when analyzing the inline function definition in bits/poll2.h, the
compiler sees that nfds is the size of fds and tries to use that
information in the function body.  In _FORTIFY_SOURCE=3 case, where the
object size could be a non-constant expression, this information results
in the conclusion that nfds is the size of fds, which defeats the
purpose of the implementation because we're trying to check here if nfds
does indeed represent the size of fds.  Hence for this case, it is best
to not have the access attribute.

With the attributes gone, the expression evaluation should get delayed
until the function is actually inlined into its destinations.

Disable the access attribute for fortified function inline functions
when building at _FORTIFY_SOURCE=3 to make this work better.  The
access attributes remain for the _chk variants since they can be used
by the compiler to warn when the caller is passing invalid arguments.

[1] https://gcc.gnu.org/pipermail/gcc-patches/2021-October/581125.html

Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2021-10-20 08:33:31 +05:30
Szabolcs Nagy
83b5323261 elf: Avoid deadlock between pthread_create and ctors [BZ #28357]
The fix for bug 19329 caused a regression such that pthread_create can
deadlock when concurrent ctors from dlopen are waiting for it to finish.
Use a new GL(dl_load_tls_lock) in pthread_create that is not taken
around ctors in dlopen.

The new lock is also used in __tls_get_addr instead of GL(dl_load_lock).

The new lock is held in _dl_open_worker and _dl_close_worker around
most of the logic before/after the init/fini routines.  When init/fini
routines are running then TLS is in a consistent, usable state.
In _dl_open_worker the new lock requires catching and reraising dlopen
failures that happen in the critical section.

The new lock is reinitialized in a fork child, to keep the existing
behaviour and it is kept recursive in case malloc interposition or TLS
access from signal handlers can retake it.  It is not obvious if this
is necessary or helps, but avoids changing the preexisting behaviour.

The new lock may be more appropriate for dl_iterate_phdr too than
GL(dl_load_write_lock), since TLS state of an incompletely loaded
module may be accessed.  If the new lock can replace the old one,
that can be a separate change.

Fixes bug 28357.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-10-04 15:07:05 +01:00
Adhemerval Zanella
33099d72e4 linux: Simplify get_nprocs
This patch simplifies the memory allocation code and uses the sched
routines instead of reimplement it.  This still uses a stack
allocation buffer, so it can be used on malloc initialization code.

Linux currently supports at maximum of 4096 cpus for most architectures:

$ find -iname Kconfig | xargs git grep -A10 -w NR_CPUS | grep -w range
arch/alpha/Kconfig-	range 2 32
arch/arc/Kconfig-	range 2 4096
arch/arm/Kconfig-	range 2 16 if DEBUG_KMAP_LOCAL
arch/arm/Kconfig-	range 2 32 if !DEBUG_KMAP_LOCAL
arch/arm64/Kconfig-	range 2 4096
arch/csky/Kconfig-	range 2 32
arch/hexagon/Kconfig-	range 2 6 if SMP
arch/ia64/Kconfig-	range 2 4096
arch/mips/Kconfig-	range 2 256
arch/openrisc/Kconfig-	range 2 32
arch/parisc/Kconfig-	range 2 32
arch/riscv/Kconfig-	range 2 32
arch/s390/Kconfig-	range 2 512
arch/sh/Kconfig-	range 2 32
arch/sparc/Kconfig-	range 2 32 if SPARC32
arch/sparc/Kconfig-	range 2 4096 if SPARC64
arch/um/Kconfig-	range 1 1
arch/x86/Kconfig-# [NR_CPUS_RANGE_BEGIN ... NR_CPUS_RANGE_END] range.
arch/x86/Kconfig-	range NR_CPUS_RANGE_BEGIN NR_CPUS_RANGE_END
arch/xtensa/Kconfig-	range 2 32

With x86 supporting 8192:

arch/x86/Kconfig
 976 config NR_CPUS_RANGE_END
 977         int
 978         depends on X86_64
 979         default 8192 if  SMP && CPUMASK_OFFSTACK
 980         default  512 if  SMP && !CPUMASK_OFFSTACK
 981         default    1 if !SMP

So using a maximum of 32k cpu should cover all cases (and I would
expect once we start to have many more CPUs that Linux would provide
a more straightforward way to query for such information).

A test is added to check if sched_getaffinity can successfully return
with large buffers.

Checked on x86_64-linux-gnu and i686-linux-gnu.

Reviewed-by: Florian Weimer <fweimer@redhat.com>
2021-09-27 09:18:12 -03:00
Paul Eggert
0b5ca7c3e5 regex: copy back from Gnulib
Copy regex-related files back from Gnulib, to fix a problem with
static checking of regex calls noted by Martin Sebor.  This merges the
following changes:

* New macro __attribute_nonnull__ in misc/sys/cdefs.h, for use later
when copying other files back from Gnulib.

* Use __GNULIB_CDEFS instead of __GLIBC__ when deciding
whether to include bits/wordsize.h etc.

* Avoid duplicate entries in epsilon closure table.

* New regex.h macro _REGEX_NELTS to let regexec say that its pmatch
arg should contain nmatch elts.  Use that for regexec, instead of
__attr_access (which is incorrect).

* New regex.h macro _Attr_access_ which is like __attr_access except
portable to non-glibc platforms.

* Add some DEBUG_ASSERTs to pacify gcc -fanalyzer and to catch
recently-fixed performance bugs if they recur.

* Add Gnulib-specific stuff to port the dynarray- and lock-using parts
of regex code to non-glibc platforms.

* Fix glibc bug 11053.

* Avoid some undefined behavior when popping an empty fail stack.
2021-09-21 08:00:44 -07:00
Aurelien Jarno
63a788f48a posix: Fix attribute access mode on getcwd [BZ #27476]
There is a GNU extension that allows to call getcwd(NULL, >0). It is
described in the documentation, but also directly in the unistd.h
header, just above the declaration.

Therefore the attribute access mode added in commit 06febd8c67
is not correct. Drop it.
2021-09-16 22:32:53 +02:00
Carlos O'Donell
466f2be6c0 Add generic C.UTF-8 locale (Bug 17318)
We add a new C.UTF-8 locale. This locale is not builtin to glibc, but
is provided as a distinct locale. The locale provides full support for
UTF-8 and this includes full code point sorting via STRCMP-based
collation (strcmp or wcscmp).

The collation uses a new keyword 'codepoint_collation' which drops all
collation rules and generates an empty zero rules collation to enable
STRCMP usage in collation. This ensures that we get full code point
sorting for C.UTF-8 with a minimal 1406 bytes of overhead (LC_COLLATE
structure information and ASCII collating tables).

The new locale is added to SUPPORTED. Minimal test data for specific
code points (minus those not supported by collate-test) is provided in
C.UTF-8.in, and this verifies code point sorting is working reasonably
across the range. The locale was tested manually with the full set of
code points without failure.

The locale is harmonized with locales already shipping in various
downstream distributions. A new tst-iconv9 test is added which verifies
the C.UTF-8 locale is generally usable.

Testing for fnmatch, regexec, and recomp is provided by extending
bug-regex1, bugregex19, bug-regex4, bug-regex6, transbug, tst-fnmatch,
tst-regcomp-truncated, and tst-regex to use C.UTF-8.

Tested on x86_64 or i686 without regression.

Reviewed-by: Florian Weimer <fweimer@redhat.com>
2021-09-06 11:30:28 -04:00
Siddhesh Poyarekar
30891f35fa Remove "Contributed by" lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date.  Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.

Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions.  These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.

The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively.  These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:

https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dc
https://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2021-09-03 22:06:44 +05:30
DJ Delorie
69623c0db0 posix: remove some iso-8859-encoded characters
With the increasing adoption of UTF-8, modern editors may (will?)
replace iso-8859-encoded characters in the range 0x80..0xff with
their UTF-8 equivalent, as will mailers and other tools.  This breaks
our testsuite and corrupts patches.

So, this patch starts replacing these problematic characters with
\OCTal sequences instead (adding support for those in tst-fnmatch.c)
or with plain ASCII characters (PTESTS).

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2021-09-01 14:29:59 -04:00
Adhemerval Zanella
6b20880b22 Use support_open_dev_null_range io/tst-closefrom, misc/tst-close_range, and posix/tst-spawn5 (BZ #28260)
It ensures a continuous range of file descriptor and avoid hitting
the RLIMIT_NOFILE.

Checked on x86_64-linux-gnu.
2021-08-26 17:13:47 -03:00
Samuel Thibault
9a7ab0769b hurd: Fix glob lstat compatibility
84f7ce8447 ("posix: Add glob64 with 64-bit time_t support") replaced
GLOB_NO_LSTAT with defining GLOB_LSTAT and GLOB_LSTAT64, but the posix
and gnu versions of the change were missing in the commit.
2021-07-22 20:31:52 +02:00
Siddhesh Poyarekar
c142eb253f mcheck: Wean away from malloc hooks [BZ #23489]
Split the mcheck implementation into the debugging hooks and API so
that the API can be replicated in libc and libc_malloc_debug.so.  The
libc APIs always result in failure.

The mcheck implementation has also been moved entirely into
libc_malloc_debug.so and with it, all of the hook initialization code
can now be moved into the debug library.  Now the initialization can
be done independently of libc internals.

With this patch, libc_malloc_debug.so can no longer be used with older
libcs, which is not its goal anyway.  tst-vfork3 breaks due to this
since it spawns shell scripts, which in turn execute using the system
glibc.  Move the test to tests-container so that only the built glibc
is used.

This move also fixes bugs in the mcheck version of memalign and
realloc, thus allowing removal of the tests from tests-mcheck
exclusion list.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 18:38:02 +05:30
Siddhesh Poyarekar
2d2d9f2b48 Move malloc hooks into a compat DSO
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so.  With this, the hooks now no
longer have any effect on the core library.

libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.

Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.

The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.

Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 18:37:59 +05:30
Adhemerval Zanella
9ed752af8d posix: Ignore non opened files on tst-spawn5
The make program  might open a pipe for its job server, which triggers
an invalid check on the spawned process.  This patch now passes the
lowest file descriptor as ithe first argument, so only the range
that was actually opened is checked.

Checked on x86_64-linux-gnu and i686-linux-gnu and centos7 (which
triggers the issue).
2021-07-13 14:09:03 -03:00
H.J. Lu
0ec97597c8 Properly run tst-spawn5 directly [BZ #28067]
Change tst-spawn5.c to handle tst-spawn5 without optional path to ld.so,
--library-path nor the library path when glibc is configured with
--enable-hardcoded-path-in-tests.  This fixes BZ #28067.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-07-09 06:37:51 -07:00
Adhemerval Zanella
882d6e17bc posix: Add posix_spawn_file_actions_addclosefrom_np
This patch adds a way to close a range of file descriptors on
posix_spawn as a new file action.  The API is similar to the one
provided by Solaris 11 [1], where the file action causes the all open
file descriptors greater than or equal to input on to be closed when
the new process is spawned.

The function posix_spawn_file_actions_addclosefrom_np is safe to be
implemented by iterating over /proc/self/fd, since the Linux spawni.c
helper process does not use CLONE_FILES, so its has own file descriptor
table and any failure (in /proc operation) aborts the process creation
and returns an error to the caller.

I am aware that this file action might be redundant to the current
approach of POSIX in promoting O_CLOEXEC in more interfaces. However
O_CLOEXEC is still not the default and for some specific usages, the
caller needs to close all possible file descriptors to avoid them
leaking.  Some examples are CPython (discussed in BZ#10353) and OpenJDK
jspawnhelper [2] (where OpenJDK spawns a helper process to exactly
closes all file descriptors).  Most likely any environment which calls
functions that might open file descriptor under the hood and aim to use
posix_spawn might face the same requirement.

Checked on x86_64-linux-gnu and i686-linux-gnu on kernel 5.11 and 4.15.

[1] https://docs.oracle.com/cd/E36784_01/html/E36874/posix-spawn-file-actions-addclosefrom-np-3c.html
[2] https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/childproc.c#L82
2021-07-08 14:08:15 -03:00
Adhemerval Zanella
607449506f io: Add closefrom [BZ #10353]
The function closes all open file descriptors greater than or equal to
input argument.  Negative values are clamped to 0, i.e, it will close
all file descriptors.

As indicated by the bug report, this is a common symbol provided by
different systems (Solaris, OpenBSD, NetBSD, FreeBSD) and, although
its has inherent issues with not taking in consideration internal libc
file descriptors (such as syslog), this is also a common feature used
in multiple projects [1][2][3][4][5].

The Linux fallback implementation iterates over /proc and close all
file descriptors sequentially.  Although it was raised the questioning
whether getdents on /proc/self/fd might return disjointed entries
when file descriptor are closed; it does not seems the case on my
testing on multiple kernel (v4.18, v5.4, v5.9) and the same strategy
is used on different projects [1][2][3][5].

Also, the interface is set a fail-safe meaning that a failure in the
fallback results in a process abort.

Checked on x86_64-linux-gnu and i686-linux-gnu on kernel 5.11 and 4.15.

[1] 5238e95759/src/basic/fd-util.c (L217)
[2] ddf4b77e11/src/lxc/start.c (L236)
[3] 9e4f2f3a6b/Modules/_posixsubprocess.c (L220)
[4] 5f47c0613e/src/libstd/sys/unix/process2.rs (L303-L308)
[5] https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/childproc.c#L82
2021-07-08 14:08:14 -03:00
Stefan Liebler
ba436665b1 Fix extra PLT reference in libc.so due to __glob64_time64 if build with gcc 7.5 on 32bit.
Starting with recent commit 84f7ce8447
"posix: Add glob64 with 64-bit time_t support", elf/check-localplt
fails due to extra PLT reference __glob64_time64 in __glob64_time64
itself.

This is observable with gcc 7.5 on x86_64 with -m32 or s390x with
-m31.  E.g. if build with gcc 10, gcc is generating a call to
__glob64_time64.localalias.

This patch is adding a hidden version of __glob64_time64 in the
same way as for __globfree64_time64.
2021-07-01 16:46:59 +02:00
Adhemerval Zanella
c32c868ab8 posix: Add _Fork [BZ #4737]
Austin Group issue 62 [1] dropped the async-signal-safe requirement
for fork and provided a async-signal-safe _Fork replacement that
does not run the atfork handlers.  It will be included in the next
POSIX standard.

It allow to close a long standing issue to make fork AS-safe (BZ#4737).
As indicated on the bug, besides the internal lock for the atfork
handlers itself; there is no guarantee that the handlers itself will
not introduce more AS-safe issues.

The idea is synchronize fork with the required internal locks to allow
children in multithread processes to use mostly of standard function
(even though POSIX states only AS-safe function should be used).  On
signal handles, _Fork should be used intead and only AS-safe functions
should be used.

For testing, the new tst-_Fork only check basic usage.  I also added
a new tst-mallocfork3 which uses the same strategy to check for
deadlock of tst-mallocfork2 but using threads instead of subprocesses
(and it does deadlock if it replaces _Fork with fork).

[1] https://austingroupbugs.net/view.php?id=62
2021-06-28 15:55:56 -03:00
Andreas Schwab
5adda61f62 wordexp: handle overflow in positional parameter number (bug 28011)
Use strtoul instead of atoi so that overflow can be detected.
2021-06-27 19:35:42 +02:00
Adhemerval Zanella
d0c4083386 posix: Do not clobber errno by atfork handlers
Checked on x86_64-linux-gnu.
2021-06-24 10:04:13 -03:00
Adhemerval Zanella
9a75654037 posix: Consolidate fork implementation
The Linux nptl implementation is used as base for generic fork
implementation to handle the internal locks and mutexes.  The
system specific bits are moved a new internal _Fork symbol.

(This new implementation will be used to provide a async-signal-safe
_Fork now that POSIX has clarified that fork might not be
async-signal-safe [1]).

For Hurd it means that the __nss_database_fork_prepare_parent and
__nss_database_fork_subprocess will be run in a slight different
order.

[1] https://austingroupbugs.net/view.php?id=62
2021-06-24 10:02:06 -03:00
Adhemerval Zanella
088d3291ef y2038: Add test coverage
It is enabled through a new rule, tests-y2038, which is built only
when the ABI supports the comapt 64-bit time_t (defined by the
header time64-compat.h, which also enables the creation of the
symbol Version for Linux).  It means the tests are not built
for ABI which already provide default 64-bit time_t.

The new rule already adds the required LFS and 64-bit time_t
compiler flags.

The current coverage is:

  * libc:
    - adjtime                       tst-adjtime-time64
    - adjtimex                      tst-adjtimex-time64
    - clock_adjtime                 tst-clock_adjtime-time64
    - clock_getres                  tst-clock-time64, tst-cpuclock1-time64
    - clock_gettime                 tst-clock-time64, tst-clock2-time64,
				    tst-cpuclock1-time64
    - clock_nanosleep               tst-clock_nanosleep-time64,
				    tst-cpuclock1-time64
    - clock_settime                 tst-clock2-time64
    - cnd_timedwait                 tst-cnd-timedwait-time64
    - ctime                         tst-ctime-time64
    - ctime_r                       tst-ctime-time64
    - difftime                      tst-difftime-time64
    - fstat                         tst-stat-time64
    - fstatat                       tst-stat-time64
    - futimens                      tst-futimens-time64
    - futimes                       tst-futimes-time64
    - futimesat                     tst-futimesat-time64
    - fts_*                         tst-fts-time64
    - getitimer                     tst-itimer-timer64
    - getrusage
    - gettimeofday                  tst-clock_nanosleep-time64
    - glob / globfree               tst-gnuglob64-time64
    - gmtime                        tst-gmtime-time64
    - gmtime_r                      tst-gmtime-time64
    - lstat                         tst-stat-time64
    - localtime                     tst-y2039-time64
    - localtime_t                   tst-y2039-time64
    - lutimes                       tst-lutimes-time64
    - mktime                        tst-mktime4-time64
    - mq_timedreceive               tst-mqueue{1248}-time64
    - mq_timedsend                  tst-mqueue{1248}-time64
    - msgctl                        test-sysvmsg-time64
    - mtx_timedlock                 tst-mtx-timedlock-time64
    - nanosleep                     tst-cpuclock{12}-time64,
				    tst-mqueue8-time64, tst-clock-time64
    - nftw / ftw                    ftwtest-time64
    - ntp_adjtime                   tst-ntp_adjtime-time64
    - ntp_gettime                   tst-ntp_gettime-time64
    - ntp_gettimex                  tst-ntp_gettimex-time64
    - ppoll                         tst-ppoll-time64
    - pselect                       tst-pselect-time64
    - pthread_clockjoin_np          tst-join14-time64
    - pthread_cond_clockwait        tst-cond11-time64
    - pthread_cond_timedwait        tst-abstime-time64
    - pthread_mutex_clocklock       tst-abstime-time64
    - pthread_mutex_timedlock       tst-abstime-time64
    - pthread_rwlock_clockrdlock    tst-abstime-time64, tst-rwlock14-time64
    - pthread_rwlock_clockwrlock    tst-abstime-time64, tst-rwlock14-time64
    - pthread_rwlock_timedrdlock    tst-abstime-time64, tst-rwlock14-time64
    - pthread_rwlock_timedwrlock    tst-abstime-time64, tst-rwlock14-time64
    - pthread_timedjoin_np          tst-join14-time64
    - recvmmsg                      tst-cancel4_2-time64
    - sched_rr_get_interval         tst-sched_rr_get_interval-time64
    - select                        tst-select-time64
    - sem_clockwait                 tst-sem5-time64
    - sem_timedwait                 tst-sem5-time64
    - semctl                        test-sysvsem-time64
    - semtimedop                    test-sysvsem-time64
    - setitimer                     tst-mqueue2-time64, tst-itimer-timer64
    - settimeofday                  tst-settimeofday-time64
    - shmctl                        test-sysvshm-time64
    - sigtimedwait                  tst-sigtimedwait-time64
    - stat                          tst-stat-time64
    - thrd_sleep                    tst-thrd-sleep-time64
    - time                          tst-mqueue{1248}-time64
    - timegm                        tst-timegm-time64
    - timer_gettime                 tst-timer4-time64
    - timer_settime                 tst-timer4-time64
    - timerfd_gettime               tst-timerfd-time64
    - timerfd_settime               tst-timerfd-time64
    - timespec_get                  tst-timespec_get-time64
    - timespec_getres               tst-timespec_getres-time64
    - utime                         tst-utime-time64
    - utimensat                     tst-utimensat-time64
    - utimes                        tst-utimes-time64
    - wait3                         tst-wait3-time64
    - wait4                         tst-wait4-time64

  * librt:
    - aio_suspend                   tst-aio6-time64
    - mq_timedreceive               tst-mqueue{1248}-time64
    - mq_timedsend                  tst-mqueue{1248}-time64
    - timer_gettime                 tst-timer4-time64
    - timer_settime                 tst-timer4-time64

  * libanl:
    - gai_suspend

Reviewed-by: Lukasz Majewski <lukma@denx.de>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-06-15 10:42:11 -03:00
Adhemerval Zanella
84f7ce8447 posix: Add glob64 with 64-bit time_t support
The glob might pass a different stat struct for gl_stat and gl_lstat
when GLOB_ALTDIRFUNC is used.  This requires add a new 64-bit time
version that also uses 64-bit time stat functions.

Checked on i686-linux-gnu and x86_64-linux-gnu.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-06-15 10:42:11 -03:00
Adhemerval Zanella
47f24c21ee y2038: Add support for 64-bit time on legacy ABIs
A new build flag, _TIME_BITS, enables the usage of the newer 64-bit
time symbols for legacy ABI (where 32-bit time_t is default).  The 64
bit time support is only enabled if LFS (_FILE_OFFSET_BITS=64) is
also used.

Different than LFS support, the y2038 symbols are added only for the
required ABIs (armhf, csky, hppa, i386, m68k, microblaze, mips32,
mips64-n32, nios2, powerpc32, sparc32, s390-32, and sh).  The ABIs with
64-bit time support are unchanged, both for symbol and types
redirection.

On Linux the full 64-bit time support requires a minimum of kernel
version v5.1.  Otherwise, the 32-bit fallbacks are used and might
results in error with overflow return code (EOVERFLOW).

The i686-gnu does not yet support 64-bit time.

This patch exports following rediretions to support 64-bit time:

  * libc:
    adjtime
    adjtimex
    clock_adjtime
    clock_getres
    clock_gettime
    clock_nanosleep
    clock_settime
    cnd_timedwait
    ctime
    ctime_r
    difftime
    fstat
    fstatat
    futimens
    futimes
    futimesat
    getitimer
    getrusage
    gettimeofday
    gmtime
    gmtime_r
    localtime
    localtime_r
    lstat_time
    lutimes
    mktime
    msgctl
    mtx_timedlock
    nanosleep
    nanosleep
    ntp_gettime
    ntp_gettimex
    ppoll
    pselec
    pselect
    pthread_clockjoin_np
    pthread_cond_clockwait
    pthread_cond_timedwait
    pthread_mutex_clocklock
    pthread_mutex_timedlock
    pthread_rwlock_clockrdlock
    pthread_rwlock_clockwrlock
    pthread_rwlock_timedrdlock
    pthread_rwlock_timedwrlock
    pthread_timedjoin_np
    recvmmsg
    sched_rr_get_interval
    select
    sem_clockwait
    semctl
    semtimedop
    sem_timedwait
    setitimer
    settimeofday
    shmctl
    sigtimedwait
    stat
    thrd_sleep
    time
    timegm
    timerfd_gettime
    timerfd_settime
    timespec_get
    utime
    utimensat
    utimes
    utimes
    wait3
    wait4

  * librt:
    aio_suspend
    mq_timedreceive
    mq_timedsend
    timer_gettime
    timer_settime

  * libanl:
    gai_suspend

Reviewed-by: Lukasz Majewski <lukma@denx.de>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-06-15 10:42:11 -03:00
Florian Weimer
186cd80b1e Add missing symbols to Version files
Some symbols have explicit versioned_symbol or compat_symbol markers
in the sources, but no corresponding entry in the Versions files.
This presently works because the local: * directive is only applied
to the base version.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-06-02 07:32:19 +02:00
Stefan Liebler
2457175e8b Fix stringop-overflow warning in bug-regex19.c.
Starting with commit
26492c0a14
"Annotate additional APIs with GCC attribute access.",
gcc emits this warning on s390x:
In function 'do_one_test',
    inlined from 'do_mb_tests' at bug-regex19.c:385:11:
bug-regex19.c:271:9: error: 're_search' specified size 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
  271 |   res = re_search (&regbuf, test->string, strlen (test->string),
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  272 |      test->start, strlen (test->string) - test->start, NULL);
      |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../include/regex.h:2,
                 from bug-regex19.c:22:
bug-regex19.c: In function 'do_mb_tests':
../posix/regex.h:554:17: note: in a call to function 're_search' declared with attribute 'read_only (2, 3)'
  554 | extern regoff_t re_search (struct re_pattern_buffer *__buffer,
      |                 ^~~~~~~~~
...

The function do_one_test is inlined into do_mb_tests on s390x (at least with
gcc 10).  If do_one_test is marked with __attribute__ ((noinline)), there are
no warnings on s390x. If do_one_test is marked with
__attribute__ ((always_inline)), there are the same warnings on x86_64.

test->string points to a variable length array on stack of do_mb_tests
and the content is generated based on the passed test struct.

According to Martin Sebor, this is a false positive caused by the same bug as
the one in nss/makedb.c.  It's fixed in GCC 11 and will also be available in
the next GCC 10.4 release.
2021-05-18 10:07:30 +02:00
Adhemerval Zanella
903bc7dcc2 linux: Use sched_getaffinity for __get_nprocs (BZ #27645)
Both the sysfs and procfs parsing (through GET_NPROCS_PARSER) are
removed in favor the syscall.  The initial scratch buffer should
fit to most of the common usage (1024 bytes with maps to 8192 CPUs).

Checked on x86_64-linux-gnu and aarch64-linux-gnu.
2021-05-07 13:54:09 -03:00
Adhemerval Zanella
db373e4c57 Remove architecture specific sched_cpucount optimizations
And replace the generic algorithm with the Brian Kernighan's one.
GCC optimize it with popcnt if the architecture supports, so there
is no need to add the extra POPCNT define to enable it.

This is really a micro-optimization that only adds complexity:
recent ABIs already support it (x86-64-v2 or power64le) and it
simplifies the code for internal usage, since i686 does not allow an
internal iFUNC call.

Checked on x86_64-linux-gnu, aarch64-linux-gnu, and
powerpc64le-linux-gnu.
2021-05-07 13:35:29 -03:00
Martin Sebor
26492c0a14 Annotate additional APIs with GCC attribute access.
This change continues the improvements to compile-time out of bounds
checking by decorating more APIs with either attribute access, or by
explicitly providing the array bound in APIs such as tmpnam() that
expect arrays of some minimum size as arguments.  (The latter feature
is new in GCC 11.)

The only effects of the attribute and/or the array bound is to check
and diagnose calls to the functions that fail to provide a sufficient
number of elements, and the definitions of the functions that access
elements outside the specified bounds.  (There is no interplay with
_FORTIFY_SOURCE here yet.)

Tested with GCC 7 through 11 on x86_64-linux.
2021-05-06 11:01:05 -06:00
Florian Weimer
0b7d48d106 nptl: Move sem_close, sem_open into libc
The symbols were moved using move-symbol-to-libc.py.

Both functions are moved at the same time because they depend
on internal functions in sysdeps/pthread/sem_routines.c, which
are moved in this commit as well.  Additional hidden prototypes
are required to avoid check-localplt failures.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-05-05 17:19:38 +02:00
Florian Weimer
2c71177309 posix: Fix Hurd build failure in tst-execveat
This avoids a -Werror compilation failure due to unused local
variables.
2021-05-04 16:03:07 +02:00
Alexandra Hájková
19d83270fc linux: Add execveat system call wrapper
It operates similar to execve and it is is already used to implement
fexecve without requiring /proc to be mounted.  However, different
than fexecve, if the syscall is not supported by the kernel an error
is returned instead of trying a fallback.

Checked on x86_64-linux-gnu and powerpc64le-linux-gnu.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-05-03 16:46:06 -03:00
Andreas Schwab
b59c698981 Fix argv overrun in getconf (bug 27761)
Correct argument counter accounting when processing the -v option with the
argument directly attached.
2021-04-21 15:11:26 +02:00
Adhemerval Zanella
48ec055ddc posix: Add wait3 tests
The tst-wait4 is moved to common file and used for wait3
tests.

Checked on x86_64-linux-gnu and i686-linux-gnu.

Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2021-04-15 11:32:40 -03:00
Adhemerval Zanella
2b47727c68 posix: Consolidate register-atfork
Both htl and nptl uses a different data structure to implement atfork
handlers.  The nptl one was refactored by 27761a1042 to use a dynarray
which simplifies the code.

This patch moves the nptl one to be the generic implementation and
replace Hurd linked one.  Different than previous NPTL, Hurd also uses
a global lock, so performance should be similar.

Checked on x86_64-linux-gnu, i686-linux-gnu, and with a build for
i686-gnu.
2021-03-12 10:19:22 -03:00
Florian Weimer
63c317fe31 posix: tst-spawn4-compat can be a regular test
compat_symbol_reference now works for non-internal tests, too.
Also stop building and running the tests on those architectures
that lack the test symbol versions.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-03-09 21:07:24 +01:00