Commit Graph

3 Commits

Author SHA1 Message Date
machenbach@chromium.org
522beebb1a Allow benchmarks to provide the standard deviation.
Some benchmarks include their own runner which provides an
overall average and a standard deviation. This enables
extraction of that value similar to the other measurements.

These benchmarks should only be run once. If a benchmarks
specifies multiple runs and provides a standard deviation,
a warning will be issued that makes the build fail on the
buildbot side.

TEST=python -m unittest run_benchmarks_test
BUG=393947
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/395633012

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22424 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-16 08:53:46 +00:00
machenbach@chromium.org
e4bde5ca25 Let benchmark runner exit with proper return codes.
BUG=374740
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/313603002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21648 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-06-03 14:58:52 +00:00
machenbach@chromium.org
509c15c799 Add new benchmark suite runner.
TEST=./tools/run_benchmarks.py benchmarks/v8.json
TEST=cd tools/unittests; python -m unittest run_benchmarks_test

Does not support custom results processors yet. Will implement that in a future CL.

BUG=374740
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/293023006

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21570 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 13:05:17 +00:00