The benchtests json allows {function {variant}} categorization of
results whereas the pthread-locks tests had {function {variant
{subvariant}}}, which broke validation. Fix that by serializing the
subvariants as variant-subvariant. Also update the schema to
recognize the new benchmark attributes after fixing the naming
conventions.
Add the workload test properties (max-throughput, latency, etc.) to
the schema to prevent benchmark output validation from failing.
* benchtests/scripts/benchout.schema.json (properties): Add
new properties.
When executing bench-math the benchmark output is invalid with this
error msg:
Invalid benchmark output: 'workload-spec2006.wrf' does not match any of
the regexes: '^[_a-zA-Z0-9]*$¹ or Invalid benchmark output: Additional
properties are not allowed ('workload-spec2006.wrf' was unexpected)
The error was seen when running the test:
workload-spec2006.wrf, 'stack=1024,guard=1' and 'stack=1024,guard=2'.
The problem is that the current regex's do not accept the hyphen, dot, equal
and comma in the output.
This patch changes the regex in benchout.schema.json to accept symbols in
benchmark tests names.
ChangeLog:
* benchtests/scripts/benchout.schema.json: Fix regex to accept a
wider range of tests names.
Signed-off-by: Victor Rodriguez <victor.rodriguez.bahena@intel.com>
Reviewed-By: Siddhesh Poyarekar <siddhesh@sourceware.org>
Benchmark workload-spec2006.wrf does not produce max, min or mean
results but instead produce throughput. This is represented in
benchtests/bench-skeleton.c. This patch adjust benchout.schema.json to consider
bench.out from bench-math benchmarks as valid
ChangeLog:
* benchtests/scripts/benchout.schema.json: Add throughput as accepted
result from property and remove "max", min" and "mean" from required
properties based on benchtests/bench-skeleton.c.
Signed-off-by: Victor Rodriguez <victor.rodriguez.bahena@intel.com>
Reviewed-By: Siddhesh Poyarekar <siddhesh@sourceware.org>
