Improve performance of recursive IO locks by adding a fast path for
the single-threaded case. To reduce the number of memory accesses for
locking/unlocking, only increment the recursion counter if the lock
is already taken.
On Neoverse V1, a microbenchmark with many small freads improved by
2.9x. Multithreaded performance improved by 2%.
Reviewed-by: Cristian Rodríguez <crrodriguez@opensuse.org>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
The _chk variants of all of the printf functions become much simpler.
This is the last thing that we needed _IO_acquire_lock_clear_flags2
for, so it can go as well. I took the opportunity to make the headers
included and the names of all local variables consistent across all the
affected files.
Since we ultimately want to get rid of __no_long_double as well, it
must be possible to get all of the nontrivial effects of the _chk
functions by calling the _internal functions with appropriate flags.
For most of the __(v)xprintf_chk functions, this is covered by
PRINTF_FORTIFY plus some up-front argument checks that can be
duplicated. However, __(v)sprintf_chk installs a custom jump table so
that it can crash instead of overflowing the output buffer. This
functionality is moved to __vsprintf_internal, which now has a
'maxlen' argument like __vsnprintf_internal; to get the unsafe
behavior of ordinary (v)sprintf, pass -1 for that argument.
obstack_printf_chk and obstack_vprintf_chk are no longer in the same
file.
As a side-effect of the unification of both fortified and non-fortified
vdprintf initialization, this patch fixes bug 11319 for __dprintf_chk
and __vdprintf_chk, which was previously fixed only for dprintf and
vdprintf by the commit
commit 7ca890b88e
Author: Ulrich Drepper <drepper@redhat.com>
Date: Wed Feb 24 16:07:57 2010 -0800
Fix reporting of I/O errors in *dprintf functions.
This patch adds a test case to avoid regressions.
Tested for powerpc and powerpc64le.
This patch mechanically removes all remaining uses, and the
definitions, of the following libio name aliases:
name replaced with
---- -------------
_IO_FILE FILE
_IO_fpos_t __fpos_t
_IO_fpos64_t __fpos64_t
_IO_size_t size_t
_IO_ssize_t ssize_t or __ssize_t
_IO_off_t off_t
_IO_off64_t off64_t
_IO_pid_t pid_t
_IO_uid_t uid_t
_IO_wint_t wint_t
_IO_va_list va_list or __gnuc_va_list
_IO_BUFSIZ BUFSIZ
_IO_cookie_io_functions_t cookie_io_functions_t
__io_read_fn cookie_read_function_t
__io_write_fn cookie_write_function_t
__io_seek_fn cookie_seek_function_t
__io_close_fn cookie_close_function_t
I used __fpos_t and __fpos64_t instead of fpos_t and fpos64_t because
the definitions of fpos_t and fpos64_t depend on the largefile mode.
I used __ssize_t and __gnuc_va_list in a handful of headers where
namespace cleanliness might be relevant even though they're
internal-use-only. In all other cases, I used the public-namespace
name.
There are a tiny handful of places where I left a use of 'struct _IO_FILE'
alone, because it was being used together with 'struct _IO_FILE_plus'
or 'struct _IO_FILE_complete' in the same arithmetic expression.
Because this patch was almost entirely done with search and replace, I
may have introduced indentation botches. I did proofread the diff,
but I may have missed something.
The ChangeLog below calls out all of the places where this was not a
pure search-and-replace change.
Installed stripped libraries and executables are unchanged by this patch,
except that some assertions in vfscanf.c change line numbers.
* libio/libio.h (_IO_FILE): Delete; all uses changed to FILE.
(_IO_fpos_t): Delete; all uses changed to __fpos_t.
(_IO_fpos64_t): Delete; all uses changed to __fpos64_t.
(_IO_size_t): Delete; all uses changed to size_t.
(_IO_ssize_t): Delete; all uses changed to ssize_t or __ssize_t.
(_IO_off_t): Delete; all uses changed to off_t.
(_IO_off64_t): Delete; all uses changed to off64_t.
(_IO_pid_t): Delete; all uses changed to pid_t.
(_IO_uid_t): Delete; all uses changed to uid_t.
(_IO_wint_t): Delete; all uses changed to wint_t.
(_IO_va_list): Delete; all uses changed to va_list or __gnuc_va_list.
(_IO_BUFSIZ): Delete; all uses changed to BUFSIZ.
(_IO_cookie_io_functions_t): Delete; all uses changed to
cookie_io_functions_t.
(__io_read_fn): Delete; all uses changed to cookie_read_function_t.
(__io_write_fn): Delete; all uses changed to cookie_write_function_t.
(__io_seek_fn): Delete; all uses changed to cookie_seek_function_t.
(__io_close_fn): Delete: all uses changed to cookie_close_function_t.
* libio/iofopncook.c: Remove unnecessary forward declarations.
* libio/iolibio.h: Correct outdated commentary.
* malloc/malloc.c (__malloc_stats): Remove unnecessary casts.
* stdio-common/fxprintf.c (__fxprintf_nocancel):
Remove unnecessary casts.
* stdio-common/getline.c: Use _IO_getdelim directly.
Don't redefine ssize_t.
* stdio-common/printf_fp.c, stdio_common/printf_fphex.c
* stdio-common/printf_size.c: Don't redefine size_t or FILE.
Remove outdated comments.
* stdio-common/vfscanf.c: Don't redefine va_list.
_IO_MTSAFE_IO controls whether stdio is *built* with support for
multithreading. In the distant past it might also have worked as a
feature selection macro, allowing library *users* to select
thread-safe or lock-free stdio at application build time, I haven't
done the archaeology. Nowadays, defining _IO_MTSAFE_IO while using
the installed headers, or in _ISOMAC mode, will cause libio.h to throw
syntax errors.
This patch removes _IO_MTSAFE_IO from the public headers
(specifically, from libio/libio.h). The most important thing it
controlled in there was whether libio.h defines _IO_lock_t itself or
expects stdio-lock.h to have done it, and we do still need a
inter-header communication macro for that, because stdio-lock.h can
only define _IO_lock_t as a typedef. I've invented
_IO_lock_t_defined, which is defined by both versions of stdio-lock.h.
_IO_MTSAFE_IO also controlled the definitions of a handful of macros
that _might_ count as part of the public libio.h interface. They are
now unconditionally given their non-_IO_MTSAFE_IO definition in
libio/libio.h, and include/libio.h redefines them with the
_IO_MTSAFE_IO definition. This should minimize the odds of breaking
old software that actually uses those macros.
I suspect that this entire mechanism is vestigial, and that glibc
won't build anymore if you *don't* define _IO_MTSAFE_IO, but that's
another patchset. The bulk of libio.h is internal-use-only stuff that
no longer makes sense to expose (libstdc++ gave up on making a FILE
the same object as a C++ filebuf *decades* ago) but that, too, is
another patchset.
* libio/libio.h: Condition dummy definition of _IO_lock_t on
_IO_lock_t_defined, not _IO_MTSAFE_IO. Unconditionally use the
non-_IO_MTSAFE_IO definitions for _IO_peekc, _IO_flockfile,
_IO_funlockfile, and _IO_ftrylockfile. Only define
_IO_cleanup_region_start and _IO_cleanup_region_end if not
already defined.
* include/libio.h: If _IO_MTSAFE_IO is defined, redefine
_IO_peekc, _IO_flockfile, _IO_funlockfile, and _IO_ftrylockfile
appropriately.
* sysdeps/generic/stdio-lock.h, sysdeps/nptl/stdio-lock.h:
Define _IO_lock_t_defined after defining _IO_lock_t.
It was noted in
<https://sourceware.org/ml/libc-alpha/2012-09/msg00305.html> that the
bits/*.h naming scheme should only be used for installed headers.
This patch renames bits/stdio-lock.h to plain stdio-lock.h to follow
that convention.
Tested for x86_64 (testsuite, and that installed stripped shared
libraries are unchanged by the patch).
[BZ #14912]
* bits/stdio-lock.h: Move to ...
* sysdeps/generic/stdio-lock.h: ...here.
(_BITS_STDIO_LOCK_H): Rename macro to _STDIO_LOCK_H.
* sysdeps/nptl/bits/stdio-lock.h: Move to ...
* sysdeps/nptl/stdio-lock.h: ...here.
(_BITS_STDIO_LOCK_H): Rename macro to _STDIO_LOCK_H.
* include/libio.h: Include <stdio-lock.h> instead of
<bits/stdio-lock.h>.
* sysdeps/nptl/fork.c: Likewise.
* sysdeps/pthread/flockfile.c: Likewise.
* sysdeps/pthread/ftrylockfile.c: Likewise.
* sysdeps/pthread/funlockfile.c: Likewise.
