This is part of the refactoring needed to implement a feature that allows
re-running benchmarks until needed confidence is reached.
R=machenbach@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:880724
Change-Id: I45f584a3503ecf567f4c2661a302a74fc5e516af
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1581605
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61037}
This is dead code as no benchmarks seem to be using it and removing this
additional logic will reduce maintenance cost for the perf infrastructure.
R=machenbach@chromium.org, tmrts@chromium.org
No-Try: true
No-Tree-Checks: true
Change-Id: I952af069d4d18d63624510b4c461fa1891703db9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1570008
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60947}
This is part of the refactoring to allow exporting more information about test
execution to the recipes and upload this information to ChromePerf.
This fixes running secondary even after primary run fails, which will allow us
to differentiate between test and infra failures as latter ones will also affect
refbuilds and re-runs without patch.
R=machenbach@chromium.org, tmrts@chromium.org
Bug: chromium:841700
Change-Id: I29ce49d2f8c5e73158f1d41a73c51f2b35929f36
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1570006
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60946}
This is part of the refactoring to allow exporting more information about
test execution to the recipes and upload this information to ChromePerf.
R=machenbach@chromium.org,tmrts@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:841700
Change-Id: Iab400e8922231d8eac91a6fa22ce8f45053f7ac6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1569442
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60944}
- Remove all relative imports from mock and os
- Fix mocking in a few tests to prevent cross-test side-effects
- Add run_perf_test.py to v8_presubmit.py
- The vpython config was not added since root .vpython already includes
coverage and mock libraries
- Convert all double-quoted strings to single-quoted (PS8->PS9)
R=sergiyb@chromium.org
Bug: chromium:123456
Change-Id: I7b3a08dc5d950b0f51cc7a5eb3a012ea953ca824
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1564206
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60810}
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
This CL was uploaded by git cl split.
Bug: v8:8594
Change-Id: I661c52a70527e8ddde841fee6d4dcba282b4a938
Reviewed-on: https://chromium-review.googlesource.com/c/1470123
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59675}
We define a TestFailedError exception and raise it when we can reliably detect
that a test has crashed. All other exceptions are treated as infra failures and
are captured by the try-catch clause in MainWrapper function.
This also fixes all tests in run_perf_test.py, run_tests_test.py and makes sure
that both are run on any changes in tools directory.
R=machenbach@chromium.org
Bug: chromium:899028
Change-Id: I283bc87b31c814be476bebe9fdda414975494183
Reviewed-on: https://chromium-review.googlesource.com/c/1303293
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57134}
This is a reland of 3b06511052
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
Bug: v8:6917
Change-Id: Ia39010a0a0f63537ad12490dfab17897d70d4930
Reviewed-on: https://chromium-review.googlesource.com/806034
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49830}
This reverts commit 3b06511052.
Reason for revert: Broke fuzzers:
https://build.chromium.org/p/client.v8.clusterfuzz/builders/V8%20Deopt%20Fuzzer/builds/384
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: v8:6917
Change-Id: I4938642c4396366be1e13daf6998c4b8538b688b
Reviewed-on: https://chromium-review.googlesource.com/804254
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49805}
This is a reland of 98cc9e862f
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
Bug: v8:6917
Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
Reviewed-on: https://chromium-review.googlesource.com/800370
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49756}
This reverts commit 98cc9e862f.
Reason for revert: Breaks test isolation:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20builder/builds/29746
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I44b99468d18fd093833f4185dad067a9eeaf2bc1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/800292
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49747}
Immutable command class with shell, flags and
environment.
Command creation moved from worker to the main
process. Because of that there is no need to send
test cases beyond process boundaries and load test
suites in worker processes.
Bug: v8:6917
Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
Reviewed-on: https://chromium-review.googlesource.com/791710
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49746}
The runs are now called as primary (no suffix) and secondary. This is in
preparation to adding secondary builds on CI, which will run tests on the latest
released stable V8 binary (aka as ref builds).
R=machenbach@chromium.org
Bug: chromium:783763
Change-Id: Ie6560012887bd5bb0d948bc8d34a9256d922137c
Reviewed-on: https://chromium-review.googlesource.com/781941
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49560}
This adds the possibility to specify a python script for post-processing stdout.
This also adds some system tests for testing the new feature.
NOTRY=true
Change-Id: I0383afb3e23513629508feeb639ed2dfce56b54a
Reviewed-on: https://chromium-review.googlesource.com/443449
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43257}
Reason for revert:
[Sheriff] Still fails on android.
Original issue's description:
> reland [tools] add --pretty switch to run_perf.py
>
> This CL improves running our internal benchmarks locally by adding the
> --pretty option to tools/run_perf.py. With the flag enabled we print
> the run-time of each benchmark directly and avoid the json output at
> the end.
>
> NOTRY=true
>
> Committed: https://crrev.com/b543c40d78c3aa8283a7d9fccb6c9e26a9a81f90
> Cr-Commit-Position: refs/heads/master@{#34023}
TBR=cbruni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/1700953002
Cr-Commit-Position: refs/heads/master@{#34033}
This CL improves running our internal benchmarks locally by adding the
--pretty option to tools/run_perf.py. With the flag enabled we print
the run-time of each benchmark directly and avoid the json output at
the end.
NOTRY=true
Review URL: https://codereview.chromium.org/1681283004
Cr-Commit-Position: refs/heads/master@{#34023}
Reason for revert:
[Sheriff] Breaks android runs. Will look later why.
Original issue's description:
> [tools] add --pretty switch to run_perf.py
>
> This CL improves running our internal benchmarks locally by adding the
> --pretty option to tools/run_perf.py. With the flag enabled we print
> the run-time of each benchmark directly and avoid the json output at
> the end.
>
> NOTRY=true
>
> Committed: https://crrev.com/83f69507ab1b9380b56758b747d4f3fabc849e49
> Cr-Commit-Position: refs/heads/master@{#33981}
TBR=cbruni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/1696293002
Cr-Commit-Position: refs/heads/master@{#34017}
This CL improves running our internal benchmarks locally by adding the
--pretty option to tools/run_perf.py. With the flag enabled we print
the run-time of each benchmark directly and avoid the json output at
the end.
NOTRY=true
Review URL: https://codereview.chromium.org/1681283004
Cr-Commit-Position: refs/heads/master@{#33981}
This allows running two executables interleaved to reduce
flakiness of tryjobs. The executables must reside in
different out directories.
BUG=chromium:507213
LOG=n
NOTRY=true
TEST=python -m unittest run_perf_test
Review URL: https://codereview.chromium.org/1215273003
Cr-Commit-Position: refs/heads/master@{#29572}
Convert all results to floats and log errors on failing
conversions. Otherwise these results bubble up into
buildbot.
TBR=tandrii@chromium.org
BUG=chromium:374740
LOG=n
NOTRY=true
Review URL: https://codereview.chromium.org/811483008
Cr-Commit-Position: refs/heads/master@{#25913}
This switches off the resources feature. It will be solely
used for configuring files to be copied to android devices
in a follow up CL.
TBR=svenpanne@chromium.org
NOTRY=true
Review URL: https://codereview.chromium.org/764013002
Cr-Commit-Position: refs/heads/master@{#25563}
