glibc/benchtests
Will Newton 44558701ff benchtests: Switch string benchmarks to use bench-timing.h.
Switch the string benchmarks to using bench-timing.h instead
of hp-timing.h directly. This allows the string benchmarks to
be run usefully on architectures such as ARM that do not have
support for hp-timing.h.

In order to do this the tests have been changed from timing each
individual call and picking the lowest execution time recorded to
timing a number of calls and taking the mean execution time.

ChangeLog:

2013-09-04   Will Newton  <will.newton@linaro.org>

	* benchtests/bench-timing.h (TIMING_PRINT_MEAN): New macro.
	* benchtests/bench-string.h: Include bench-timing.h instead
	of including hp-timing.h directly. (INNER_LOOP_ITERS): New
	define. (HP_TIMING_BEST): Delete macro. (test_init): Remove
	call to HP_TIMING_DIFF_INIT.
	* benchtests/bench-memccpy.c: Use bench-timing.h macros
	instead of hp-timing.h macros.
	* benchtests/bench-memchr.c: Likewise.
	* benchtests/bench-memcmp.c: Likewise.
	* benchtests/bench-memcpy.c: Likewise.
	* benchtests/bench-memmem.c: Likewise.
	* benchtests/bench-memmove.c: Likewise.
	* benchtests/bench-memset.c: Likewise.
	* benchtests/bench-rawmemchr.c: Likewise.
	* benchtests/bench-strcasecmp.c: Likewise.
	* benchtests/bench-strcasestr.c: Likewise.
	* benchtests/bench-strcat.c: Likewise.
	* benchtests/bench-strchr.c: Likewise.
	* benchtests/bench-strcmp.c: Likewise.
	* benchtests/bench-strcpy.c: Likewise.
	* benchtests/bench-strcpy_chk.c: Likewise.
	* benchtests/bench-strlen.c: Likewise.
	* benchtests/bench-strncasecmp.c: Likewise.
	* benchtests/bench-strncat.c: Likewise.
	* benchtests/bench-strncmp.c: Likewise.
	* benchtests/bench-strncpy.c: Likewise.
	* benchtests/bench-strnlen.c: Likewise.
	* benchtests/bench-strpbrk.c: Likewise.
	* benchtests/bench-strrchr.c: Likewise.
	* benchtests/bench-strspn.c: Likewise.
	* benchtests/bench-strstr.c: Likewise.
2013-09-04 15:40:12 +01:00
..
acos-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
acosh-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
asin-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
asinh-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
atan-inputs Allow multiple input domains to be run in the same benchmark program 2013-04-30 14:17:57 +05:30
atanh-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
bench-bcopy-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-bcopy.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-bzero-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-bzero.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memccpy-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memccpy.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-memchr-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memchr.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-memcmp-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memcmp.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-memcpy-ifunc.c Begin porting string performance tests to benchtests 2013-06-11 15:08:13 +05:30
bench-memcpy.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-memmem-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memmem.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-memmove-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memmove.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-mempcpy-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-mempcpy.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memset-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-memset.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-modf.c Prevent optimizing out of benchmark function call 2013-05-17 19:10:33 +05:30
bench-rawmemchr-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-rawmemchr.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-skeleton.c Use HP_TIMING for benchmarks if available 2013-05-13 13:44:32 +05:30
bench-stpcpy_chk-ifunc.c Port remaining string benchmarks 2013-06-11 20:51:55 +05:30
bench-stpcpy_chk.c Port remaining string benchmarks 2013-06-11 20:51:55 +05:30
bench-stpcpy-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-stpcpy.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-stpncpy-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-stpncpy.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcasecmp-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcasecmp.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strcasestr-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcasestr.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strcat-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcat.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strchr-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strchr.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strchrnul-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strchrnul.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcmp-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcmp.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strcpy_chk-ifunc.c Port remaining string benchmarks 2013-06-11 20:51:55 +05:30
bench-strcpy_chk.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strcpy-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcpy.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strcspn-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strcspn.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-string.h benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strlen-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strlen.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strncasecmp-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strncasecmp.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strncat-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strncat.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strncmp-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strncmp.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strncpy-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strncpy.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strnlen-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strnlen.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strpbrk-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strpbrk.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strrchr-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strrchr.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strspn-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strspn.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-strstr-ifunc.c Copy over string performance tests into benchtests 2013-06-11 15:08:13 +05:30
bench-strstr.c benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
bench-timing.h benchtests: Switch string benchmarks to use bench-timing.h. 2013-09-04 15:40:12 +01:00
cos-inputs Allow multiple input domains to be run in the same benchmark program 2013-04-30 14:17:57 +05:30
cosh-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
exp-inputs Allow multiple input domains to be run in the same benchmark program 2013-04-30 14:17:57 +05:30
log-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
Makefile benchtests/Makefile: Use LDLIBS instead of LDFLAGS. 2013-09-04 15:38:41 +01:00
pow-inputs Allow multiple input domains to be run in the same benchmark program 2013-04-30 14:17:57 +05:30
README Begin porting string performance tests to benchtests 2013-06-11 15:08:13 +05:30
rint-inputs PowerPC: remove branch prediction from rint implementation 2013-04-01 06:36:51 -05:00
sin-inputs Allow multiple input domains to be run in the same benchmark program 2013-04-30 14:17:57 +05:30
sinh-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30
tan-inputs Allow multiple input domains to be run in the same benchmark program 2013-04-30 14:17:57 +05:30
tanh-inputs Add benchmark inputs for math functions 2013-05-22 11:07:33 +05:30

Using the glibc microbenchmark suite
====================================

The glibc microbenchmark suite automatically generates code for specified
functions, builds and calls them repeatedly for given inputs to give some
basic performance properties of the function.

Running the benchmark:
=====================

The benchmark can be executed by invoking make as follows:

  $ make bench

This runs each function for 10 seconds and appends its output to
benchtests/bench.out.  To ensure that the tests are rebuilt, one could run:

  $ make bench-clean

The duration of each test can be configured setting the BENCH_DURATION variable
in the call to make.  One should run `make bench-clean' before changing
BENCH_DURATION.

  $ make BENCH_DURATION=1 bench

The benchmark suite does function call measurements using architecture-specific
high precision timing instructions whenever available.  When such support is
not available, it uses clock_gettime (CLOCK_PROCESS_CPUTIME_ID).  One can force
the benchmark to use clock_gettime by invoking make as follows:

  $ make USE_CLOCK_GETTIME=1 bench

Again, one must run `make bench-clean' before changing the measurement method.

Adding a function to benchtests:
===============================

If the name of the function is `foo', then the following procedure should allow
one to add `foo' to the bench tests:

- Append the function name to the bench variable in the Makefile.

- Define foo-ARGLIST as a colon separated list of types of the input
  arguments.  Use `void' if function does not take any inputs.  Put in quotes
  if the input argument is a pointer, e.g.:

     malloc-ARGLIST: "void *"

- Define foo-RET as the type the function returns.  Skip if the function
  returns void.  One could even skip foo-ARGLIST if the function does not
  take any inputs AND the function returns void.

- Make a file called `foo-inputs` with one input value per line, an input
  being a comma separated list of arguments to be passed into the function.
  See pow-inputs for an example.

  The script that parses the -inputs file treats lines beginning with a single
  `#' as comments.  Lines beginning with two hashes `##' are treated specially
  as `directives'.

Multiple execution units per function:
=====================================

Some functions have distinct performance characteristics for different input
domains and it may be necessary to measure those separately.  For example, some
math functions perform computations at different levels of precision (64-bit vs
240-bit vs 768-bit) and mixing them does not give a very useful picture of the
performance of these functions.  One could separate inputs for these domains in
the same file by using the `name' directive that looks something like this:

  ##name: 240bit

See the pow-inputs file for an example of what such a partitioned input file
would look like.

Benchmark Sets:
==============

In addition to standard benchmarking of functions, one may also generate
custom outputs for a set of functions.  This is currently used by string
function benchmarks where the aim is to compare performance between
implementations at various alignments and for various sizes.

To add a benchset for `foo':

- Add `foo' to the benchset variable.
- Write your bench-foo.c that prints out the measurements to stdout.
- On execution, a bench-foo.out is created in $(objpfx) with the contents of
  stdout.