Commit Graph

16 Commits

Author SHA1 Message Date
machenbach
e925e0b69f [test] Let perf runner interleave try executions.
This allows running two executables interleaved to reduce
flakiness of tryjobs. The executables must reside in
different out directories.

BUG=chromium:507213
LOG=n
NOTRY=true
TEST=python -m unittest run_perf_test

Review URL: https://codereview.chromium.org/1215273003

Cr-Commit-Position: refs/heads/master@{#29572}
2015-07-10 13:02:25 +00:00
machenbach
af1eb2258c Make perf test runner more robust.
Convert all results to floats and log errors on failing
conversions. Otherwise these results bubble up into
buildbot.

TBR=tandrii@chromium.org
BUG=chromium:374740
LOG=n
NOTRY=true

Review URL: https://codereview.chromium.org/811483008

Cr-Commit-Position: refs/heads/master@{#25913}
2014-12-21 10:25:32 +00:00
machenbach
6a3460129c Add Android platform to performance runner.
BUG=chromium:374740
LOG=n
TEST=python -m unittest run_perf_test
NOTRY=true

Review URL: https://codereview.chromium.org/768143002

Cr-Commit-Position: refs/heads/master@{#25614}
2014-12-02 14:24:00 +00:00
machenbach
7c92531255 Make resources config ready for android test runner.
This switches off the resources feature. It will be solely
used for configuring files to be copied to android devices
in a follow up CL.

TBR=svenpanne@chromium.org
NOTRY=true

Review URL: https://codereview.chromium.org/764013002

Cr-Commit-Position: refs/heads/master@{#25563}
2014-11-28 11:35:44 +00:00
machenbach
e42fda5ebb Add test flags feature to perf runner.
TBR=svenpanne@chromium.org
NOTRY=true

Review URL: https://codereview.chromium.org/722023006

Cr-Commit-Position: refs/heads/master@{#25559}
2014-11-28 10:34:35 +00:00
machenbach@chromium.org
bb117b4dde Make benchmark runner understand chromium perf output.
BUG=406405
LOG=n
TEST=python -m unittest run_benchmarks_test
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/498163002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24509 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:12:38 +00:00
machenbach@chromium.org
5826c82058 Make timeout configurable in performance test runner.
BUG=374740
LOG=n
TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/569213002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23945 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-15 13:00:32 +00:00
machenbach@chromium.org
d909ebd5aa Remove benchmark runner.
This has been renamed to run_perf.

BUG=374740
LOG=n
TBR=dslomov@chromium.org

Review URL: https://codereview.chromium.org/564373003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23942 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-15 11:47:47 +00:00
machenbach@chromium.org
b77cb4a793 Revert "Refactoring: Make gtest testsuite the default."
This reverts commit cb8f08658767e7e3182e5deab0473f264f6ce92f.

TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/532673002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23584 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:17:26 +00:00
machenbach@chromium.org
a050734c35 Refactoring: Make gtest testsuite the default.
BUG=v8:3489
R=bmeurer@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/526133003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:11:13 +00:00
machenbach@chromium.org
68ec08240c Add test driver with the notion of perf tests.
BUG=374740
LOG=n
R=dslomov@chromium.org

Review URL: https://codereview.chromium.org/526953005

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23569 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-01 14:27:44 +00:00
machenbach@chromium.org
8aa7bb8a10 Let benchmark runner add summary traces.
BUG=374740
LOG=n
R=jarin@chromium.org

Review URL: https://codereview.chromium.org/496183003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23493 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-28 14:42:24 +00:00
machenbach@chromium.org
8545f713c6 Teach benchmark runner to understand generic traces.
BUG=406405
LOG=n
TEST=python -m unittest run_benchmarks_test
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/502473002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23329 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-24 14:32:44 +00:00
machenbach@chromium.org
522beebb1a Allow benchmarks to provide the standard deviation.
Some benchmarks include their own runner which provides an
overall average and a standard deviation. This enables
extraction of that value similar to the other measurements.

These benchmarks should only be run once. If a benchmarks
specifies multiple runs and provides a standard deviation,
a warning will be issued that makes the build fail on the
buildbot side.

TEST=python -m unittest run_benchmarks_test
BUG=393947
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/395633012

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22424 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-16 08:53:46 +00:00
machenbach@chromium.org
e4bde5ca25 Let benchmark runner exit with proper return codes.
BUG=374740
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/313603002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21648 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-06-03 14:58:52 +00:00
machenbach@chromium.org
509c15c799 Add new benchmark suite runner.
TEST=./tools/run_benchmarks.py benchmarks/v8.json
TEST=cd tools/unittests; python -m unittest run_benchmarks_test

Does not support custom results processors yet. Will implement that in a future CL.

BUG=374740
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/293023006

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21570 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 13:05:17 +00:00