l_audit ends up as an internal array with _rtld_global, and GCC 10
warns about this.
This commit does not change the layout of _rtld_global, so it is
suitable for backporting. Future changes could allocate more of the
audit state dynamically and remove it from always-allocated data
structures, to optimize the common case of inactive auditing.
Change-Id: Ic911100730f9124d4ea977ead8e13cee64b84d45
To improve GCC 10 compatibility, it is necessary to remove the l_audit
zero-length array from the end of struct link_map. In preparation of
that, this commit introduces an accessor function for the audit state,
so that it is possible to change the representation of the audit state
without adjusting the code that accesses it.
Tested on x86_64-linux-gnu. Built on i686-gnu.
Change-Id: Id815673c29950fc011ae5301d7cde12624f658df
libio can only deal with gconv conversions which consist of a single
step. Not using __gconv_info simplifies the data structures somewhat.
This eliminates a new GCC 10 warning about subscribing an inner
zero-length array.
Tested on x86_64-linux-gnu with mainline GCC. Built with
build-many-glibcs.py, also with mainline GCC. Due to GCC PR 92039,
there are failures left on 32-bit architectures with float128 support.
Change-Id: I8b4c489b619a53154712ff32e1b6f13bb92d4203
Add a script for visualizing the JSON output generated by existing
glibc string microbenchmarks.
Overview:
plot_strings.py is capable of plotting benchmark results in the
following formats, which are controlled with the -p or --plot argument:
1. absolute timings (-p time): plot the timings as they are in the
input benchmark results file.
2. relative timings (-p rel): plot relative timing difference with
respect to a chosen ifunc (controlled with -b argument).
3. performance relative to max (-p max): for each varied parameter
value, plot 1/timing as the percentage of the maximum value out of
the plotted ifuncs.
4. throughput (-p thru): plot varied parameter value over timing
For all types of graphs, there is an option to explicitly specify
the subset of ifuncs to plot using the --ifuncs parameter.
For plot types 1. and 4. one can hide/expose exact benchmark figures
using the --values flag.
When plotting relative timing differences between ifuncs, the first
ifunc listed in the input JSON file is the baseline, unless the
baseline implementation is explicitly chosen with the --baseline
parameter. For the ease of reading, the script marks the statistically
insignificant range on the graphs. The default is +-5% but this
value can be controlled with the --threshold parameter.
To accommodate for the heterogeneity in benchmark results files,
one can control i.e the x-axis scale, the resolution (dpi) of the
generated figures or the key to access the varied parameter value
in the JSON file. The corresponding options are --logarithmic,
--resolution or --key. The --key parameter ensures that plot_strings.py
works with all files which pass JSON schema validation. The schema
can be chosen with the --schema parameter.
If a window manager is available, one can enable interactive
figure display using the --display flag.
Finally, one can use the --grid flag to enable grid lines in the
generated figures.
Implementation:
plot_strings.py traverses the JSON tree until a 'results' array
is found and generates a separate figure for each such array.
The figure is then saved to a file in one of the available formats
(controlled with the --extension parameter).
As the tree is traversed, the recursive function tracks the metadata
about the test being run, so that each figure has a unique and
meaningful title and filename.
While plot_strings.py works with existing benchmarks, provisions
have been made to allow adding more structure and metadata to these
benchmarks. Currently, many benchmarks produce multiple timing values
for the same value of the varied parameter (typically 'length').
Mutiple data points for the same parameter usually mean that some other
parameter was varied as well, for example, if memmove's src and dst
buffers overlap or not (see bench-memmove-walk.c and
bench-memmove-walk.out).
Unfortunately, this information is not exposed in the benchmark output
file, so plot_strings.py has to resort to computing the geometric mean
of these multiple values. In the process, useful information about the
benchmark configuration is lost. Also, averaging the timings for
different alignments can hide useful characterstics of the benchmarked
ifuncs.
Testing:
plot_strings.py has been tested on all existing string microbenchmarks
which produce results in JSON format. The script was tested on both
Windows 10 and Ubuntu 16.04.2 LTS. It runs on both python 2 and 3
(2.7.12 and 3.5.12 tested).
Useful commands:
1. Plot timings for all ifuncs in bench-strlen.out:
$ ./plot_strings.py bench-strlen.out
2. Display help:
$ ./plot_strings.py -h
3. Plot throughput for __memset_avx512_unaligned_erms and
__memset_avx512_unaligned. Save the generated figure in pdf format to
'results/'. Use logarithmic x-axis scale, show grid lines and expose
the performance numbers:
$ ./plot_strings.py bench.out -o results/ -lgv -e pdf -p thru \
-i __memset_avx512_unaligned_erms __memset_avx512_unaligned
4. Plot relative timings for all ifuncs in bench.out with __generic_memset
as baseline. Display percentage difference threshold of +-10%:
$ ./plot_strings.py bench.out -p rel -b __generic_memset -t 10
Discussion:
1. I would like to propose relaxing the benchout_strings.schema.json
to allow specifying either a 'results' array with 'timings' (as before)
or a 'variants' array. See below example:
{
"timing_type": "hp_timing",
"functions": {
"memcpy": {
"bench-variant": "default",
"ifuncs": ["generic_memcpy", "__memcpy_thunderx"],
"variants": [
{
"name": "powers of 2",
"variants": [
{
"name": "both aligned",
"results": [
{
"length": 1,
"align1": 0,
"align2": 0,
"timings": [x, y]
},
{
"length": 2,
"align1": 0,
"align2": 0,
"timings": [x, y]
},
...
{
"length": 65536,
"align1": 0,
"align2": 0,
"timings": [x, y]
}]
},
{
"name": "dst misaligned",
"results": [
{
"length": 1,
"align1": 0,
"align2": 0,
"timings": [x, y]
},
{
"length": 2,
"align1": 0,
"align2": 1,
"timings": [x, y]
},
...
'variants' array consists of objects such that each object has a 'name'
attribute to describe the configuration of a particular test in the
benchmark. This can be a description, for example, of how the parameter
was varied or what was the buffer alignment tested. The 'name' attribute
is then followed by another 'variants' array or a 'results' array.
The nesting of variants allows arbitrary grouping of benchmark timings,
while allowing description of these groups. Using recusion, it is
possible to proceduraly create titles and filenames for the figures being
generated.
PTHREAD_STACK_MIN comes from <limits.h>, so include it explicitly.
However, it is not defined on Hurd, so compensate for that as well.
Built on x86_64-linux-gnu, i686-linux-gnu, i686-gnu.
Change-Id: Ifacc888ef86731c2639721b0932ae59583bd6b3e
Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>
This reduces the possible error scenarios considerably because
no longer can file seek fail, leaving the file descriptor in an
inconsistent state and out of sync with the cache.
As a result, it is possible to avoid setting file_offset to -1
to make an error persistent. Instead, subsequent calls will retry
the operation and report any errors returned by the kernel.
This change also avoids reading the file from the start if pututline
is called multiple times, to work around lock acquisition failures
due to timeouts.
Change-Id: If21ea0c162c38830a89331ea93cddec14c0974de
Only one of the currently defined flags is incompatible with versioned
symbol lookups, so it makes sense to check for that flag and not its
complement.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Gabriel F. T. Gomes <gabrielftg@linux.ibm.com>
Change-Id: I3384349cef90cfd91862ebc34a4053f0c0a99404
Increase the upper bound on medium cases from 96 to 128 bytes.
Now, up to 128 bytes are copied unrolled.
Increase the upper bound on small cases from 16 to 32 bytes so that
copies of 17-32 bytes are not impacted by the larger medium case.
Benchmarking:
The attached figures show relative timing difference with respect
to 'memcpy_generic', which is the existing implementation.
'memcpy_med_128' denotes the the version of memcpy_generic with
only the medium case enlarged. The 'memcpy_med_128_small_32' numbers
are for the version of memcpy_generic submitted in this patch, which
has both medium and small cases enlarged. The figures were generated
using the script from:
https://www.sourceware.org/ml/libc-alpha/2019-10/msg00563.html
Depending on the platform, the performance improvement in the
bench-memcpy-random.c benchmark ranges from 6% to 20% between
the original and final version of memcpy.S
Tested against GLIBC testsuite and randomized tests.
This simplifies internal_getut_nolock and fixes a regression,
introduced in commit be6b16d975
("login: Acquire write lock early in pututline [BZ #24882]")
in pututxline because __utmp_equal can only compare process-related
utmp entries.
Fixes: be6b16d975
Change-Id: Ib8a85002f7f87ee41590846d16d7e52bdb82f5a5
GCC 10 will warn about subscribing inner length zero arrays. Use a GCC
extension in csu/libc-tls.c to allocate space for the static_slotinfo
variable. Adjust nptl_db so that the type description machinery does
not attempt to determine the size of the flexible array member slotinfo.
Change-Id: I51be146a7857186a4ede0bb40b332509487bdde8
This patch fixes the time64 support (added by 2e44b10b42) where it
misses the remaining argument updated if __NR_clock_nanosleep
returns EINTR.
Checked on i686-linux-gnu on 4.15 kernel (no time64 support) and
on 5.3 kernel (with time64 support).
Reviewed-by: Alistair Francis <alistair23@gmail.com>
C2X adds the asctime_r, ctime_r, gmtime_r and localtime_r functions.
This patch duly adds __GLIBC_USE (ISOC2X) to the conditions under
which <time.h> declares them.
Tested for x86_64.
This patch provides new __ppoll64 explicit 64 bit function for handling polling
events (with struct timespec specified timeout) for a set of file descriptors.
Moreover, a 32 bit version - __ppoll has been refactored to internally use
__ppoll64.
The __ppoll is now supposed to be used on systems still supporting 32 bit time
(__TIMESIZE != 64) - hence the necessary conversion to 64 bit struct
__timespec64.
The new ppoll_time64 syscall available from Linux 5.1+ has been used, when
applicable.
The Linux kernel checks if passed tv_nsec value overflows, so there is no need
to repeat it in the glibc.
When ppoll syscall on systems supporting 32 bit time ABI is used, the check is
performed if passed data (which may have 64 bit tv_sec) fits into 32 bit range.
Build tests:
- The code has been tested on x86_64/x86 (native compilation):
make PARALLELMFLAGS="-j8" && make check PARALLELMFLAGS="-j8" && \\
make xcheck PARALLELMFLAGS="-j8"
- The glibc has been build tested (make PARALLELMFLAGS="-j8") for
x86 (i386), x86_64-x32, and armv7
Run-time tests:
- Run specific tests on ARM/x86 32bit systems (qemu):
https://github.com/lmajewski/meta-y2038 and run tests:
https://github.com/lmajewski/y2038-tests/commits/master
- Use of cross-test-ssh.sh for ARM (armv7):
make PARALLELMFLAGS="-j8" test-wrapper='./cross-test-ssh.sh root@192.168.7.2' xcheck
Linux kernel, headers and minimal kernel version for glibc build test
matrix:
- Linux v5.1 (with ppoll_time64) and glibc build with v5.1 as
minimal kernel version (--enable-kernel="5.1.0")
The __ASSUME_TIME64_SYSCALLS flag defined.
- Linux v5.1 and default minimal kernel version
The __ASSUME_TIME64_SYSCALLS not defined, but kernel supports ppoll_time64
syscall.
- Linux v4.19 (no ppoll_time64 support) with default minimal kernel version for
contemporary glibc
This kernel doesn't support ppoll_time64 syscall, so the fallback to ppoll is
tested.
Above tests were performed with Y2038 redirection applied as well as without
(so the __TIMESIZE != 64 execution path is checked as well).
No regressions were observed.
And related tests. These tests create a thread for each core, so
they may fail due to address space limitations with the default
stack size.
Change-Id: Ieef44a7731f58d3b7d6638cce4ccd31126647551
If the regex has more subexpressions than the number of elements allocated
in the regmatch_t array passed to regexec then proceed_next_node may
access the regmatch_t array outside its bounds.
No testcase added because even without this bug it would then crash in
pop_fail_stack which is bug 11053.
The clock_nanosleep syscall is not supported on newer 32-bit platforms (such
as RV32). To fix this issue let's use clock_nanosleep_time64 if it is
avaliable.
It just contains duplicated defitions provided by other generic
nptl headers.
Checked with run-built-tests=no against hppa-linux-gnu.
Change-Id: I95f55d5b7b7ae528c81cd2394d57ce92398189bf
It has been reported that due to lack of fairness in POSIX file
locking, the current reader-to-writer lock upgrade can result in
lack of forward progress. Acquiring the write lock directly
hopefully avoids this issue if there are only writers.
This also fixes bug 24882 due to the cache revalidation in
__libc_pututline.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Change-Id: I57e31ae30719e609a53505a0924dda101d46372e
Adds the __libpthread_version_placeholder symbol with the same version
of nanosleep/__nanosleep that was removed by 79a547b162 and that
is not provided by other symbols.
Since l_whence is the second member of struct flock, it is written
twice. The double-assignment is technically undefined behavior due to
the lack of a sequence point.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Change-Id: I2baf9e70690e723c61051b25ccbd510aec15976c
They cause a check-localplt failure after commit f9a7554009.
Fixes: f9a7554009
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Change-Id: I37bc20f3449b9e358f32879ed231720c969965b4
In the function handle_input_flag, the end-loop condition is not
correct, because when the loop variable i equals 16
(num_input_flag_types), then input_flags[16] will be out of bounds.
(This issue is only relevant with invalid input files to
gen-auto-libm-tests.)
The generic version is straightforward. For Hurd, its nanosleep
implementation is moved to clock_nanosleep with adjustments from
generic unix implementation.
The generic clock_nanosleep unix version is also removed since
it calls nanosleep.
Checked on x86_64-linux-gnu and powerpc64le-linux-gnu.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Checked on x86_64-linux-gnu and powerpc64le-linux-gnu. I also checked
the libpthread.so .gnu.version_d entries for every ABI affected and
all of them contains the required versions (including for architectures
which exports __nanosleep with a different version).
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The s390 gcc bug https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77918
"S390: Floating point comparisons don't raise invalid for unordered operands."
is fixed with gcc 10. Thus we conditionally set FIX_COMPARE_INVALID
to 0 or 1.
Nothing defines CALL_PSELECT6 in the current tree, so remove it.
Tested with:
- make PARALLELMFLAGS="-j8" && make xcheck PARALLELMFLAGS="-j8" (x86_64)
- scripts/build-many-glibcs.py
The `test' make target passes a trailing slash in the subdir argument. This
does not play well with elf/rtld-Rules which looks for `elf' without any
trailing slash, and therefore doesn't find a match when running an elf test
individually. This commit removes the trailing slash from the invocation.
Reviewed-by: DJ Delorie <dj@redhat.com>
The hppa architecture requires strict alignment for loads and stores.
As a result, the minimum stack alignment that will work is 8 bytes.
This patch adjusts __clone() to align the stack argument passed to it.
It also adjusts slightly some formatting.
This fixes the nptl/tst-tls1 test.
This patch provides new __futimens64 explicit 64 bit function for
setting access and modification time of file (by using its file descriptor).
Moreover, a 32 bit version - __futimens has been refactored to internally use
__futimens64.
The __futimens is now supposed to be used on systems still supporting
32 bit time (__TIMESIZE != 64) - hence the necessary conversions to 64 bit
struct __timespec64.
When pointer to struct __timespec64 is NULL - the file access and modification
time is set to the current one (by the kernel) and no conversions from struct
timespec to __timespec64 are performed.
The __futimens64 reuses __utimensat64_helper defined for __utimensat64.
The test procedure for __futimens64 is the same as for __utimensat64 conversion
patch.
This patch provides new __utimensat64 explicit 64 bit function for
setting access and modification time of a file. Moreover, a 32 bit version
- __utimensat has been refactored to internally use __utimensat64.
The __utimensat is now supposed to be used on systems still supporting
32 bit time (__TIMESIZE != 64) - hence the necessary conversions to 64 bit
struct __timespec64.
When pointer to struct __timespec64 is NULL - the file access and modification
time is set to the current one and no conversions from struct timespec to
__timespec64 are performed.
The new utimensat_time64 syscall available from Linux 5.1+ has been used,
when applicable.
The new helper function - __utimensat64_helper - has been introduced to
facilitate code re-usage on function providing futimens syscall handling.
The Linux kernel checks if passed tv_nsec value overflows, so there is no
need to repeat it in glibc.
When utimensat syscall on systems supporting 32 bit time ABI is used,
the check is performed if passed data (which may have 64 bit tv_sec) fits
into 32 bit range.
Build tests:
- The code has been tested on x86_64/x86 (native compilation):
make PARALLELMFLAGS="-j8" && make xcheck PARALLELMFLAGS="-j8"
- The glibc has been build tested (make PARALLELMFLAGS="-j8") for
x86 (i386), x86_64-x32, and armv7
Run-time tests:
- Run specific tests on ARM/x86 32bit systems (qemu):
https://github.com/lmajewski/meta-y2038 and run tests:
https://github.com/lmajewski/y2038-tests/commits/master
- Use of cross-test-ssh.sh for ARM (armv7):
make PARALLELMFLAGS="-j8" test-wrapper='./cross-test-ssh.sh root@192.168.7.2' xcheck
Linux kernel, headers and minimal kernel version for glibc build test
matrix:
- Linux v5.1 (with utimensat_time64) and glibc build with v5.1 as
minimal kernel version (--enable-kernel="5.1.0")
The __ASSUME_TIME64_SYSCALLS flag defined.
- Linux v5.1 and default minimal kernel version
The __ASSUME_TIME64_SYSCALLS not defined, but kernel supports utimensat_time64
syscall.
- Linux v4.19 (no utimensat_time64 support) with default minimal kernel
version for contemporary glibc
This kernel doesn't support utimensat_time64 syscall, so the fallback
to utimensat is tested.
The above tests were performed with Y2038 redirection applied as well as
without (so the __TIMESIZE != 64 execution path is checked as well).
No regressions were observed.
Passing NULL as the timeout parameter to pthread_timedjoin_np has resulted
in it behaving like pthread_join for a long time. Since that is now the
documented behaviour, we ought to test that both it and the new
pthread_clockjoin_np support it.
Checked on x86_64-linux-gnu.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Introduce pthread_clockjoin_np as a version of pthread_timedjoin_np that
accepts a clockid_t parameter to indicate which clock the timeout should be
measured against. This mirrors the recently-added POSIX-proposed "clock"
wait functions.
Checked on x86_64-linux-gnu.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch adds the generic futex_lock_pi and futex_unlock_pi to wrap
around the syscall machinery required to issue the syscall calls. It
simplifies a bit the futex code required to implement PI mutexes.
No function changes, checked on x86_64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
