This is a reland of 98cc9e862f
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
Bug: v8:6917
Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
Reviewed-on: https://chromium-review.googlesource.com/800370
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49756}
This reverts commit 98cc9e862f.
Reason for revert: Breaks test isolation:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20builder/builds/29746
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I44b99468d18fd093833f4185dad067a9eeaf2bc1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/800292
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49747}
Immutable command class with shell, flags and
environment.
Command creation moved from worker to the main
process. Because of that there is no need to send
test cases beyond process boundaries and load test
suites in worker processes.
Bug: v8:6917
Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
Reviewed-on: https://chromium-review.googlesource.com/791710
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49746}
The runs are now called as primary (no suffix) and secondary. This is in
preparation to adding secondary builds on CI, which will run tests on the latest
released stable V8 binary (aka as ref builds).
R=machenbach@chromium.org
Bug: chromium:783763
Change-Id: Ie6560012887bd5bb0d948bc8d34a9256d922137c
Reviewed-on: https://chromium-review.googlesource.com/781941
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49560}
Added a flag, process_size, to collect the maximum memory size used by
a test run.
Bug:
Change-Id: I5d659ff03bfcf7ab4a868aafbda3303ffcb9fa78
Reviewed-on: https://chromium-review.googlesource.com/542415
Commit-Queue: Mircea Trofin <mtrofin@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46713}
This is a highly requested feature!
Bug: v8:6276
Change-Id: I17b606ae0ff8fa9dfdd0fa74fd1f7ad0dd3fc4f8
Reviewed-on: https://chromium-review.googlesource.com/488044
Commit-Queue: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44916}
Inspired by bsuite/compare-baseline.py and tools/perf-to-html.py. The main difference
from the latter is that the new script does not require json files preprocessing and it
is able to compare multiple json files at once (just like bsuite/compare-baseline.py).
This CL also makes tools/run_perf.py relative path friendly.
BUG=
Change-Id: I7be796fa920f8360703806f32767b992eec0f9c2
Reviewed-on: https://chromium-review.googlesource.com/456286
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43982}
For results processor to work, we need to maintain the assumption that the cwd is equal to the currently running benchmark directory.
NOTRY=true
TBR=clemensh@chromium.org
Change-Id: I585c301dea846194652645973470786ad43aa280
Reviewed-on: https://chromium-review.googlesource.com/444784
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43280}
This adds the possibility to specify a python script for post-processing stdout.
This also adds some system tests for testing the new feature.
NOTRY=true
Change-Id: I0383afb3e23513629508feeb639ed2dfce56b54a
Reviewed-on: https://chromium-review.googlesource.com/443449
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43257}
For benchmarks with multiple measurements (e.g. runtime, compile
time, generated code size), in case of error, we'd get only the
last portion of the name ("Compile"), which is ambiguous and makes
debugging dificult. This change includes the full name, e.g.
"Unity/Physics/Compile".
BUG=
Review-Url: https://codereview.chromium.org/2615623003
Cr-Commit-Position: refs/heads/master@{#42078}
I plan to drive these options from the infra recipe.
Raising the priority of the benchmarked process, and fixing
its affinity to some other CPU than 0 improves stability of
measurements.
BUG=
Review-Url: https://codereview.chromium.org/1915303002
Cr-Commit-Position: refs/heads/master@{#35912}
Reason for revert:
[Sheriff] Still fails on android.
Original issue's description:
> reland [tools] add --pretty switch to run_perf.py
>
> This CL improves running our internal benchmarks locally by adding the
> --pretty option to tools/run_perf.py. With the flag enabled we print
> the run-time of each benchmark directly and avoid the json output at
> the end.
>
> NOTRY=true
>
> Committed: https://crrev.com/b543c40d78c3aa8283a7d9fccb6c9e26a9a81f90
> Cr-Commit-Position: refs/heads/master@{#34023}
TBR=cbruni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/1700953002
Cr-Commit-Position: refs/heads/master@{#34033}
This CL improves running our internal benchmarks locally by adding the
--pretty option to tools/run_perf.py. With the flag enabled we print
the run-time of each benchmark directly and avoid the json output at
the end.
NOTRY=true
Review URL: https://codereview.chromium.org/1681283004
Cr-Commit-Position: refs/heads/master@{#34023}
Reason for revert:
[Sheriff] Breaks android runs. Will look later why.
Original issue's description:
> [tools] add --pretty switch to run_perf.py
>
> This CL improves running our internal benchmarks locally by adding the
> --pretty option to tools/run_perf.py. With the flag enabled we print
> the run-time of each benchmark directly and avoid the json output at
> the end.
>
> NOTRY=true
>
> Committed: https://crrev.com/83f69507ab1b9380b56758b747d4f3fabc849e49
> Cr-Commit-Position: refs/heads/master@{#33981}
TBR=cbruni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/1696293002
Cr-Commit-Position: refs/heads/master@{#34017}
This CL improves running our internal benchmarks locally by adding the
--pretty option to tools/run_perf.py. With the flag enabled we print
the run-time of each benchmark directly and avoid the json output at
the end.
NOTRY=true
Review URL: https://codereview.chromium.org/1681283004
Cr-Commit-Position: refs/heads/master@{#33981}
Improved flexibility for the perf runner, by adding option to
specify precisely shell binary.
NOTRY=true
Review URL: https://codereview.chromium.org/1659483003
Cr-Commit-Position: refs/heads/master@{#33649}
Enhance the perf runner to run with the profiler and print the summary for each d8 run. This automates running the profiler with multiple benchmarks.
BUG=None
LOG=N
TBR=machenbach@chromium.org
Review URL: https://codereview.chromium.org/1327033003
Cr-Commit-Position: refs/heads/master@{#30720}
This allows running two executables interleaved to reduce
flakiness of tryjobs. The executables must reside in
different out directories.
BUG=chromium:507213
LOG=n
NOTRY=true
TEST=python -m unittest run_perf_test
Review URL: https://codereview.chromium.org/1215273003
Cr-Commit-Position: refs/heads/master@{#29572}
This prepares for making multiple measurements of one trace.
For this, the suite/trace configurations need to be
independent of the measurement instances.
BUG=chromium:507213
LOG=n
NOTRY=true
Review URL: https://codereview.chromium.org/1227033002
Cr-Commit-Position: refs/heads/master@{#29531}
The android_* configs were never real v8 target
architectures, only make targets. This doesn't make sense
with ninja anymore.
BUG=chromium:502176
LOG=n
NOTRY=true
Review URL: https://codereview.chromium.org/1220743002
Cr-Commit-Position: refs/heads/master@{#29349}
Convert all results to floats and log errors on failing
conversions. Otherwise these results bubble up into
buildbot.
TBR=tandrii@chromium.org
BUG=chromium:374740
LOG=n
NOTRY=true
Review URL: https://codereview.chromium.org/811483008
Cr-Commit-Position: refs/heads/master@{#25913}
This adds a common perf configuration for JSTests including
Classes, Collections, Iterators and Strings. This allows
the android test runner to handle subdirectories correctly
and to share the base.js resource in the parent directory.
The new json config has added resources configs for the
Android runner.
The perf runner's relative paths on the device are fixed as
well. Resources are only pushed on the configuration node
where they are specified. They are pushed to a dir on the
device that follows the same directory structure as on the
host. The binary is executed in the benchmark folder on the
device like on the host to allow relative path file
loading.
BUG=chromium:374740
LOG=n
TEST=python -m unittest run_perf_test
TBR=ulan@chromium.org
NOTRY=true
Review URL: https://codereview.chromium.org/779923002
Cr-Commit-Position: refs/heads/master@{#25655}
This is basically a refactoring that extracts desktop
specific code into a separate class.
Also adds flags for android runs (not used yet).
BUG=chromium:374740
LOG=n
TEST=python -m unittest run_perf_test
NOTRY=true
Review URL: https://codereview.chromium.org/762163002
Cr-Commit-Position: refs/heads/master@{#25576}
This switches off the resources feature. It will be solely
used for configuring files to be copied to android devices
in a follow up CL.
TBR=svenpanne@chromium.org
NOTRY=true
Review URL: https://codereview.chromium.org/764013002
Cr-Commit-Position: refs/heads/master@{#25563}
