Use "=" instead of ":=" to allow sysdeps Makefiles to add more benches
to bench and benchset. This fixes BZ #28970.
Reviewed-by: Sunil K Pandey <skpgkp2@gmail.com>
Commit ac759b1fbf added attribute
"overlap" to bench-memmove-walk, whose value is a string. This change
makes compare_strings.py fail since benchout_strings.schema.json
requires the values of attributes to be number.
This patch relaxes such constraint.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Small sizes (<= 64) represent large portion of memset usages with zero
value. Add sizes (<= 64) to bench-bzero-walk.c to cover small sizes.
Reviewed-by: Sunil K Pandey <skpgkp2@gmail.com>
memset with zero as the value to set is by far the majority value (99%+
for Python3 and GCC). Add bench-memset-zero-large.c,
bench-memset-zero-walk.c and bench-memset-zero.c to measure memset
implementations for zeroing.
Reviewed-by: Sunil K Pandey <skpgkp2@gmail.com>
Zero is a relevant size for some workloads (roughly 5% of uses for
GCC) so we should be testing it's performance as well.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Add more small and medium sized tests for strcmp and strncmp.
As well for strcmp add option for more direct control of
alignment. Previously alignment was being pushed to the end of the
page. While this is the most difficult case to implement, it is far
from the common case and so shouldn't be the only benchmark.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
This commit adds a new partial overlap benchmark. This is generally
the most interesting performance case for memmove and was missing.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
This commit adds more benchmarks for the common memcpy/memmove
benchmarks. The most signifcant cases are the half page offsets. The
current versions leaves dst and src near page aligned which leads to
false 4k aliasing on x86_64. This can add noise due to false
dependencies from one run to the next. As well, this seems like more
of an edge case that common case so it shouldn't be the only thing
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Based on random input arguments. About 85% tuples have exponents
of the two arguments close together (+-1 range).
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Instead of inputs based on the algorithm implementation details.
About 85% tuples have exponents of the two arguments close
together (+-1 range).
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Add acosf function to bench-math and copy acosf-inputs to benchtests.
Motivation for this patch is to prepare for upcoming libmvec new
functions. Float and double version of libmvec functions stays
together.
acosf-inputs file generated from acos-inputs file using following
scaling formula:
f = d * (FLT_MAX/DBL_MAX)
Where d is input(double) and f is output(float). If scaled float value
is duplicate in new input file, nextafterf() function used to find next
float value, ensuring no duplicates.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Improve the random memcpy benchmark. Double the number of tests and increase
the size of the memory region to test between 32KB and 1024KB. This improves
accuracy on modern cores. Clean up formatting of the frequency array.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
No bug. This commit adds __memcmpeq benchmarks. The benchmarks just
use the existing ones in memcmp. This will be useful for testing
implementations of __memcmpeq that do not just alias memcmp.
No bug.
This commit adds new medium size cases for lengths in [512, 1024). As
well it increase the iters to INNER_LOOP_ITERS_LARGE for more reliable
results.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
Building benchmarks as static executables:
=========================================
To build benchmarks as static executables, on the build system, run:
$ make STATIC-BENCHTESTS=yes bench-build
You can copy benchmark executables to another machine and run them
without copying the source nor build directories.
No bug. Remove reallocation of bufs between implementation tests. Move
initialization outside of foreach implementation test loop. Increase
iteration count.
Generally before this commit was seeing a great deal of variability
between runs. The goal of this commit is to make the results more
reliable.
Benchtests build and bench-memcmp succeeding.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
This patch fixed validate_benchout.py two exceptions,
1) AttributeError
if benchout_strings.schema.json is specified, and
2) json.decoder.JSONDecodeError
if benchout file is not JSON.
$ ~/glibc/benchtests/scripts/validate_benchout.py bench-memset.out \
~/glibc/benchtests/scripts/benchout_strings.schema.json
Traceback (most recent call last):
File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in <module>
sys.exit(main(sys.argv[1:]))
File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main
bench.parse_bench(args[0], args[1])
File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 139, in parse_bench
do_for_all_timings(bench, lambda b, f, v:
File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 107, in do_for_all_timings
if 'timings' not in bench['functions'][func][k].keys():
AttributeError: 'str' object has no attribute 'keys'
$ ~/glibc/benchtests/scripts/validate_benchout.py bench-math-inlines.out \
~/glibc/benchtests/scripts/benchout_strings.schema.json
Traceback (most recent call last):
File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in <module>
sys.exit(main(sys.argv[1:]))
File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main
bench.parse_bench(args[0], args[1])
File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 137, in parse_bench
bench = json.load(benchfile)
File "/usr/lib/python3.6/json/__init__.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 342, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 17 (char 16)
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
This patch removed redundant "#include <assert.h>" from
bench-memset-large.c and bench-memset-walk.c.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
This patch enables scripts/plot_strings.py to read a benchmark result
file from stdin.
To keep backward compatibility, that is to keep accepting multiple of
benchmark result files in argument, blank argument doesn't mean stdin,
but '-' does.
Therefore nargs parameter of ArgumentParser.add_argument() method is
not changed to '?', but keep '+'.
ex:
$ jq '.' bench-memset.out | plot_strings.py -
$ jq '.' bench-memset.out | plot_strings.py - bench-memset-large.out
$ plot_strings.py bench-memset.out bench-memset-large.out
error ex:
$ jq '.' bench-memset.out | plot_strings.py
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
They provide TLS_GD/TLS_LD/TLS_IE/TLS_IE macros for TLS testing. Now
that we have migrated to __thread and tls_model attributes, these macros
are unused and the tls-macros.h files can retire.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
These comments refer to slow paths that were removed in
glibc 2.34 or earlier. The corresponding "names" that yield
separate workload traces for "make bench" are thus obsolete.
We are however keeping the corresponding inputs.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
The benchmark and tests must fail in case of allocation failure in the
implementation array. Also annotate the x* allocators in support.h so
that the compiler has more information about them.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
This patch fixed mprotect system call failure on AArch64.
This failure happened on not only A64FX but also ThunderX2.
Also this patch updated a JSON key from "max-size" to "length" so that
'plot_strings.py' can process 'bench-memcpy-random.out'
This patch adds support for testing memcpy with both dst > src and dst
< src. Since memcpy is implemented as memmove which has seperate
control flows for certain sizes depending on dst > src it seems like
1) information that should be provided in the benchtest output and a
variable that can be controlled for the benchmarks.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
No bug. This commit adds some additional performance test cases to
bench-memcmp.c and test-memcmp.c. The new benchtests include some
medium range sizes, as well as small sizes near page cross. The new
correctness tests correspond with the new benchtests though add some
additional cases for checking the page cross logic.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Convert the output of benchtests/bench-rawmemchr to JSON like other string
benchmarks. This makes the output more parseable and allows usage of
compare_strings.py, for example.
Reviewed-by: Lucas A. M. Magalhaes <lamm@linux.ibm.com>
These workload traces cover the whole "long double" range.
This patch was prepared with the help of Adhemerval Zanella.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
No bug. This commit adds some additional cases for bench-memchr.c
including testing medium sizes and testing short length with both an
inbound match and out of bound match.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
No bug. This commit adds tests cases and benchmarks for page cross and
for memset to the end of the page without crossing. As well in
test-memset.c this commit adds sentinel on start/end of tstbuf to test
for overwrites
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
The benchtests json allows {function {variant}} categorization of
results whereas the pthread-locks tests had {function {variant
{subvariant}}}, which broke validation. Fix that by serializing the
subvariants as variant-subvariant. Also update the schema to
recognize the new benchmark attributes after fixing the naming
conventions.
No Bug. This commit expanding the range of tests / benchmarks for
memmove and memcpy. The test expansion is mostly in the vein of
increasing the maximum size, increasing the number of unique
alignments tested, and testing both source < destination and vice
versa. The benchmark expansaion is just to increase the number of
unique alignments. test-memcpy, test-memccpy, test-mempcpy,
test-memmove, and tst-memmove-overflow all pass.
Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
This patch adds workload traces for all double format functions where such
files are missing. For each function, a set of 1000 random values is
generated at random using SageMath, such that the output values are
meaningful (for example avoiding too large inputs for exp10 where the
output would be +Inf). More details about the generated values are
given at the beginning of each file.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
This patch updates json "bench-variant" attribute of "bench-memset.c"
to "default" so that the script "benchtests/scripts/plot_strings.py"
can generate a file "memset_time_default_linear.png".
Without this patch, the script "benchtests/scripts/plot_strings.py"
generates a file "memset_time__linear.png" which has inconsistent form
with "memcpy_time_default_linear.png" and
"memmove_time_default_linear.png".
Since commit 2682695e5c, `make bench-build' with `--enable-static-pie'
fails due to bench-timing-type being incorrectly built with MODULE_NAME
set to `libc'. This commit sets MODULE_NAME to nonlib, thus fixing the
build failure.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
GNU ld and gold have supported --print-output-format since 2011. glibc
requires binutils>=2.25 (2015), so if LD is GNU ld or gold, we can
assume the option is supported.
lld is by default a cross linker supporting multiple targets. It auto
detects the file format and does not need OUTPUT_FORMAT. It does not
support --print-output-format.
By parsing objdump -f, we can support all the three linkers.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
Performance benchmarks for various posix locks: mutex, rwlock,
spinlock, condvar, and semaphore. Each test is performed with
an empty loop body or with a computationally "interesting" (i.e.
difficult to optimize away, and used just to allow lock code to
be "hidden" in the filler's CPU cycles).
