Commit Graph

17 Commits

Author SHA1 Message Date
Michael Achenbach
b593f1a56e [test] Implement results processor for perf runner.
This adds the possibility to specify a python script for post-processing stdout.

This also adds some system tests for testing the new feature.

NOTRY=true

Change-Id: I0383afb3e23513629508feeb639ed2dfce56b54a
Reviewed-on: https://chromium-review.googlesource.com/443449
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43257}
2017-02-16 15:43:19 +00:00
Michael Achenbach
34663b428c [test] Fix unittests for perf runner.
This was omitted in:
https://codereview.chromium.org/2615623003

TBR=clemensh@chromium.org,mtrofin@chromium.org
NOTRY=true

Change-Id: I78449fe72e27976b95a9557e0bd8f986ed8caa64
Reviewed-on: https://chromium-review.googlesource.com/443526
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43232}
2017-02-16 08:48:33 +00:00
machenbach
4a58b46e55 Revert of [tools] add --pretty switch to run_perf.py (patchset #8 id:140001 of https://codereview.chromium.org/1681283004/ )
Reason for revert:
[Sheriff] Still fails on android.

Original issue's description:
> reland [tools] add --pretty switch to run_perf.py
>
> This CL improves running our internal benchmarks locally by adding the
> --pretty option to tools/run_perf.py. With the flag enabled we print
> the run-time of each benchmark directly and avoid the json output at
> the end.
>
> NOTRY=true
>
> Committed: https://crrev.com/b543c40d78c3aa8283a7d9fccb6c9e26a9a81f90
> Cr-Commit-Position: refs/heads/master@{#34023}

TBR=cbruni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1700953002

Cr-Commit-Position: refs/heads/master@{#34033}
2016-02-16 12:56:43 +00:00
cbruni
b543c40d78 reland [tools] add --pretty switch to run_perf.py
This CL improves running our internal benchmarks locally by adding the
--pretty option to tools/run_perf.py. With the flag enabled we print
the run-time of each benchmark directly and avoid the json output at
the end.

NOTRY=true

Review URL: https://codereview.chromium.org/1681283004

Cr-Commit-Position: refs/heads/master@{#34023}
2016-02-16 09:51:54 +00:00
machenbach
2f8e33526b Revert of [tools] add --pretty switch to run_perf.py (patchset #7 id:120001 of https://codereview.chromium.org/1681283004/ )
Reason for revert:
[Sheriff] Breaks android runs. Will look later why.

Original issue's description:
> [tools] add --pretty switch to run_perf.py
>
> This CL improves running our internal benchmarks locally by adding the
> --pretty option to tools/run_perf.py. With the flag enabled we print
> the run-time of each benchmark directly and avoid the json output at
> the end.
>
> NOTRY=true
>
> Committed: https://crrev.com/83f69507ab1b9380b56758b747d4f3fabc849e49
> Cr-Commit-Position: refs/heads/master@{#33981}

TBR=cbruni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1696293002

Cr-Commit-Position: refs/heads/master@{#34017}
2016-02-16 08:19:32 +00:00
cbruni
83f69507ab [tools] add --pretty switch to run_perf.py
This CL improves running our internal benchmarks locally by adding the
--pretty option to tools/run_perf.py. With the flag enabled we print
the run-time of each benchmark directly and avoid the json output at
the end.

NOTRY=true

Review URL: https://codereview.chromium.org/1681283004

Cr-Commit-Position: refs/heads/master@{#33981}
2016-02-15 09:23:26 +00:00
machenbach
18d2c58cfd [test] More robust perf runner with profiler option.
NOTRY=true

Review URL: https://codereview.chromium.org/1341213002

Cr-Commit-Position: refs/heads/master@{#30731}
2015-09-15 08:16:10 +00:00
machenbach
e925e0b69f [test] Let perf runner interleave try executions.
This allows running two executables interleaved to reduce
flakiness of tryjobs. The executables must reside in
different out directories.

BUG=chromium:507213
LOG=n
NOTRY=true
TEST=python -m unittest run_perf_test

Review URL: https://codereview.chromium.org/1215273003

Cr-Commit-Position: refs/heads/master@{#29572}
2015-07-10 13:02:25 +00:00
machenbach
af1eb2258c Make perf test runner more robust.
Convert all results to floats and log errors on failing
conversions. Otherwise these results bubble up into
buildbot.

TBR=tandrii@chromium.org
BUG=chromium:374740
LOG=n
NOTRY=true

Review URL: https://codereview.chromium.org/811483008

Cr-Commit-Position: refs/heads/master@{#25913}
2014-12-21 10:25:32 +00:00
machenbach
6a3460129c Add Android platform to performance runner.
BUG=chromium:374740
LOG=n
TEST=python -m unittest run_perf_test
NOTRY=true

Review URL: https://codereview.chromium.org/768143002

Cr-Commit-Position: refs/heads/master@{#25614}
2014-12-02 14:24:00 +00:00
machenbach
7c92531255 Make resources config ready for android test runner.
This switches off the resources feature. It will be solely
used for configuring files to be copied to android devices
in a follow up CL.

TBR=svenpanne@chromium.org
NOTRY=true

Review URL: https://codereview.chromium.org/764013002

Cr-Commit-Position: refs/heads/master@{#25563}
2014-11-28 11:35:44 +00:00
machenbach
e42fda5ebb Add test flags feature to perf runner.
TBR=svenpanne@chromium.org
NOTRY=true

Review URL: https://codereview.chromium.org/722023006

Cr-Commit-Position: refs/heads/master@{#25559}
2014-11-28 10:34:35 +00:00
machenbach@chromium.org
bb117b4dde Make benchmark runner understand chromium perf output.
BUG=406405
LOG=n
TEST=python -m unittest run_benchmarks_test
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/498163002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24509 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:12:38 +00:00
machenbach@chromium.org
5826c82058 Make timeout configurable in performance test runner.
BUG=374740
LOG=n
TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/569213002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23945 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-15 13:00:32 +00:00
machenbach@chromium.org
b77cb4a793 Revert "Refactoring: Make gtest testsuite the default."
This reverts commit cb8f08658767e7e3182e5deab0473f264f6ce92f.

TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/532673002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23584 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:17:26 +00:00
machenbach@chromium.org
a050734c35 Refactoring: Make gtest testsuite the default.
BUG=v8:3489
R=bmeurer@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/526133003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:11:13 +00:00
machenbach@chromium.org
68ec08240c Add test driver with the notion of perf tests.
BUG=374740
LOG=n
R=dslomov@chromium.org

Review URL: https://codereview.chromium.org/526953005

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23569 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-01 14:27:44 +00:00