This is dead code as no benchmarks seem to be using it and removing this
additional logic will reduce maintenance cost for the perf infrastructure.
R=machenbach@chromium.org, tmrts@chromium.org
No-Try: true
No-Tree-Checks: true
Change-Id: I952af069d4d18d63624510b4c461fa1891703db9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1570008
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60947}
This is part of the refactoring to allow exporting more information about test
execution to the recipes and upload this information to ChromePerf.
This fixes running secondary even after primary run fails, which will allow us
to differentiate between test and infra failures as latter ones will also affect
refbuilds and re-runs without patch.
R=machenbach@chromium.org, tmrts@chromium.org
Bug: chromium:841700
Change-Id: I29ce49d2f8c5e73158f1d41a73c51f2b35929f36
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1570006
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60946}
This is part of the refactoring to allow exporting more information about
test execution to the recipes and upload this information to ChromePerf.
R=machenbach@chromium.org,tmrts@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:841700
Change-Id: Iab400e8922231d8eac91a6fa22ce8f45053f7ac6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1569442
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60944}
We've originally added them in https://crrev.com/c/1159361 and then removed
again in https://crrev.com/c/1291370 since they were implemented in LogDog
Viewer. However, looks like logs from the swarming tasks are all dumped into
step stdout at the same time and thus logdog assigns idential timestamp to all
lines making it fairly useless.
TBR=machenbach@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:841700
Change-Id: I49bfb23f8007e4a0facdfea6a09cde911f8e5a51
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1569432
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60875}
This is not fixing the root cause of the flakiness, but prevents us from loosing
data in the short to medium term as investigation proved to be difficult and
will likely take even more time.
R=machenbach@chromium.org, tmrts@chromium.org
Bug: chromium:872257
Change-Id: Id5fbd0a00058f8612089ee4d6a858193924bd868
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1564204
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60814}
- Remove all relative imports from mock and os
- Fix mocking in a few tests to prevent cross-test side-effects
- Add run_perf_test.py to v8_presubmit.py
- The vpython config was not added since root .vpython already includes
coverage and mock libraries
- Convert all double-quoted strings to single-quoted (PS8->PS9)
R=sergiyb@chromium.org
Bug: chromium:123456
Change-Id: I7b3a08dc5d950b0f51cc7a5eb3a012ea953ca824
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1564206
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60810}
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
This CL was uploaded by git cl split.
Bug: v8:8594
Change-Id: I661c52a70527e8ddde841fee6d4dcba282b4a938
Reviewed-on: https://chromium-review.googlesource.com/c/1470123
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59675}
This reverts commit 6e03d7ee42.
Reason for revert: This breaks the Android bot:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2933
It's quite hidden due to https://crbug.com/v8/8731 but all shards
time out. It looks like to to this change, testing takes
dramatically longer, maybe the pushing takes now much longer than
before. If we want decide for this, the builder needs to get
many more shards.
Original change's description:
> Reland "[tools] Push files using high-level device.PushChangedFiles method"
>
> This is a reland of d045f66682
>
> Original change's description:
> > [tools] Push files using high-level device.PushChangedFiles method
> >
> > R=machenbach@chromium.org
> >
> > No-Try: true
> > Bug: chromium:893593
> > Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> > Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#58407}
>
> Bug: chromium:893593
> Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
> Reviewed-on: https://chromium-review.googlesource.com/c/1443055
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59221}
TBR=machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:893593
Change-Id: Ifea307b5de8f39b660966fc6bef54601df91d841
Reviewed-on: https://chromium-review.googlesource.com/c/1450119
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59305}
We define a TestFailedError exception and raise it when we can reliably detect
that a test has crashed. All other exceptions are treated as infra failures and
are captured by the try-catch clause in MainWrapper function.
This also fixes all tests in run_perf_test.py, run_tests_test.py and makes sure
that both are run on any changes in tools directory.
R=machenbach@chromium.org
Bug: chromium:899028
Change-Id: I283bc87b31c814be476bebe9fdda414975494183
Reviewed-on: https://chromium-review.googlesource.com/c/1303293
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57134}
Originally timestamps were added in crrev.com/c/1159361 to ease debugging. Since
then, however, timestamps were introduce to the LogDog viewer and are rendered
in a separate column, making it easier to scan text for important messages
starting with '>>>'. We do keep the log level, but make sure that it always
takes a fixed width such that the actual messages are aligned vertically.
R=machenbach@chromium.org
No-Try: true
Change-Id: I3a741069824aa25e674aca79da03a725017cb956
Reviewed-on: https://chromium-review.googlesource.com/c/1291370
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56834}
This will allow us to determine whether tests are not printing anything or
whether our infrastructure is failing to retrieve the stdout.
TBR=machenbach@google.com
No-Try: true
Bug: chromium:891314
Change-Id: I0786e77b4db9f247e02a25924acd6d6773c0db9f
Reviewed-on: https://chromium-review.googlesource.com/c/1282962
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56701}
This is a reland of 4c0943424c
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
Bug: chromium:866862
Change-Id: Icf7e04c75d4abeab7254d10ba21240e46b0022ae
Reviewed-on: https://chromium-review.googlesource.com/1170643
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55059}
All status messages will be printed into STDERR and only the results will
continue to be written to STDOUT as documentation suggests. Additionally the
following changes are made:
- on Buildbot each log entry will have a timestamp to make it easy to measure
time between status messages printed by test runner, users will continue to
see just the message itself
- when devil (or any other module) logs error, they will not be interleaved
with our own output since logging module is thread-safe
- critical logs replaced with warning since they do not immediately stop
the test runner
- logging.exception is used for all exceptions to get formatted stack trace
TBR=machenbach@chromium.org
No-Try: true
Bug: chromium:838864
Change-Id: I197fcc3bc5890130ce8b35c3e21237f9614fc468
Reviewed-on: https://chromium-review.googlesource.com/1159361
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54851}
This is a reland of 40d66d8bf8
The fix disambiguates duplicate symbols in the generated embedded
builtins file.
Original change's description:
> [build] Make separate snapshot for trusted variant
>
> This enables side-by-side snapshots with and without untrusted-code
> mitigations. It'll be the default in all V8 stand-alone builds
> with external startup data. Internal snapshots are not supported.
>
> The files snapshot_blob.bin and snapshot_blob_trusted.bin will be
> bundled with V8 on swarming and the correct file is loaded dependent
> on the --untrusted-code-mitigations runtime flag.
>
> Likewise we embed two snapshots for builtins.
>
> Side-by-side snapshots won't be supported in Chromium.
>
> Bug: v8:7441
> Change-Id: I2949ddfd5773649946b1c8e74751d48ad1d9c524
> Reviewed-on: https://chromium-review.googlesource.com/960004
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Jakob Gruber <jgruber@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#52028}
Bug: v8:7441
Change-Id: I626171d4e07389f0453b4d0a698e2772fd37e8c5
Reviewed-on: https://chromium-review.googlesource.com/968623
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52055}
This reverts commit 40d66d8bf8.
Reason for revert:
https://build.chromium.org/p/client.v8/builders/V8%20Mac64/builds/20910
Original change's description:
> [build] Make separate snapshot for trusted variant
>
> This enables side-by-side snapshots with and without untrusted-code
> mitigations. It'll be the default in all V8 stand-alone builds
> with external startup data. Internal snapshots are not supported.
>
> The files snapshot_blob.bin and snapshot_blob_trusted.bin will be
> bundled with V8 on swarming and the correct file is loaded dependent
> on the --untrusted-code-mitigations runtime flag.
>
> Likewise we embed two snapshots for builtins.
>
> Side-by-side snapshots won't be supported in Chromium.
>
> Bug: v8:7441
> Change-Id: I2949ddfd5773649946b1c8e74751d48ad1d9c524
> Reviewed-on: https://chromium-review.googlesource.com/960004
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Jakob Gruber <jgruber@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#52028}
TBR=rmcilroy@chromium.org,machenbach@chromium.org,yangguo@chromium.org,mstarzinger@chromium.org,jgruber@chromium.org
Change-Id: I5b18f7aff7c05a17842fc68d785bb617eeca4a41
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7441
Reviewed-on: https://chromium-review.googlesource.com/968622
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52029}
This enables side-by-side snapshots with and without untrusted-code
mitigations. It'll be the default in all V8 stand-alone builds
with external startup data. Internal snapshots are not supported.
The files snapshot_blob.bin and snapshot_blob_trusted.bin will be
bundled with V8 on swarming and the correct file is loaded dependent
on the --untrusted-code-mitigations runtime flag.
Likewise we embed two snapshots for builtins.
Side-by-side snapshots won't be supported in Chromium.
Bug: v8:7441
Change-Id: I2949ddfd5773649946b1c8e74751d48ad1d9c524
Reviewed-on: https://chromium-review.googlesource.com/960004
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52028}
Modify "run_perf.py --filter Array/" so that it will let match "Array" but
not "ArrayLiteralSpread". Previously --filter Array/ would match nothing.
Change-Id: I1f9c677e3558fc2256a71306af5fc1a8394ac698
Reviewed-on: https://chromium-review.googlesource.com/895453
Commit-Queue: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51098}
This is a reland of 3b06511052
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
Bug: v8:6917
Change-Id: Ia39010a0a0f63537ad12490dfab17897d70d4930
Reviewed-on: https://chromium-review.googlesource.com/806034
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49830}
This reverts commit 3b06511052.
Reason for revert: Broke fuzzers:
https://build.chromium.org/p/client.v8.clusterfuzz/builders/V8%20Deopt%20Fuzzer/builds/384
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: v8:6917
Change-Id: I4938642c4396366be1e13daf6998c4b8538b688b
Reviewed-on: https://chromium-review.googlesource.com/804254
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49805}
This is a reland of 98cc9e862f
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
Bug: v8:6917
Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
Reviewed-on: https://chromium-review.googlesource.com/800370
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49756}
This reverts commit 98cc9e862f.
Reason for revert: Breaks test isolation:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20builder/builds/29746
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I44b99468d18fd093833f4185dad067a9eeaf2bc1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/800292
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49747}
Immutable command class with shell, flags and
environment.
Command creation moved from worker to the main
process. Because of that there is no need to send
test cases beyond process boundaries and load test
suites in worker processes.
Bug: v8:6917
Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
Reviewed-on: https://chromium-review.googlesource.com/791710
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49746}
The runs are now called as primary (no suffix) and secondary. This is in
preparation to adding secondary builds on CI, which will run tests on the latest
released stable V8 binary (aka as ref builds).
R=machenbach@chromium.org
Bug: chromium:783763
Change-Id: Ie6560012887bd5bb0d948bc8d34a9256d922137c
Reviewed-on: https://chromium-review.googlesource.com/781941
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49560}
Added a flag, process_size, to collect the maximum memory size used by
a test run.
Bug:
Change-Id: I5d659ff03bfcf7ab4a868aafbda3303ffcb9fa78
Reviewed-on: https://chromium-review.googlesource.com/542415
Commit-Queue: Mircea Trofin <mtrofin@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46713}
This is a highly requested feature!
Bug: v8:6276
Change-Id: I17b606ae0ff8fa9dfdd0fa74fd1f7ad0dd3fc4f8
Reviewed-on: https://chromium-review.googlesource.com/488044
Commit-Queue: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44916}
Inspired by bsuite/compare-baseline.py and tools/perf-to-html.py. The main difference
from the latter is that the new script does not require json files preprocessing and it
is able to compare multiple json files at once (just like bsuite/compare-baseline.py).
This CL also makes tools/run_perf.py relative path friendly.
BUG=
Change-Id: I7be796fa920f8360703806f32767b992eec0f9c2
Reviewed-on: https://chromium-review.googlesource.com/456286
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43982}
