This is a partial revert of:
https://crrev.com/c/890938 and https://crrev.com/c/893982
Before this CL, the test runner blocked on ongoing tests in order to
process their results after an internal timeout. However, the logic
required for this feature was overly complicated and prevented an
acceptable implementation for fast aborts. Furthermore, also the fuzzers
suffered from timeouts on swarming due to hanging tests.
Instead, we now abort immediately on internal timeout (used on
fuzzers), SIGINT (Ctrl-C) and SIGTERM. Ongoing tests are immediately
terminated and their results are disregarded. On SIGTERM and SIGINT,
we return with non-zero exit codes, and zero on internal timeout.
This will also properly return json output, when the external hard
timeout is reached on swarming (causes SIGTERM).
TBR=sergiyb@chromium.org
Bug: v8:7423, chromium:813065
Change-Id: Ib20f835f58a0970693bdd3b21dc5d766d8e115d8
Reviewed-on: https://chromium-review.googlesource.com/924852
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51399}
The suppression flag must be set before loading the tests to have an effect.
This got turned around in https://crrev.com/c/899366 and is fixed again by
this CL.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6917
Change-Id: Ie58465288a0d6eec9a99a23d610710de9e1cdddd
Reviewed-on: https://chromium-review.googlesource.com/911114
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51209}
Timeouts on num-fuzzer only lead to false positives, as tests might
take unpredictably longer.
This CL forgives timeouts through a global override mechanism of the
expected outcomes. This allows to remove already existing scattered
code that allowed timeouts in some test suites only.
Bug: v8:6917
Change-Id: Ib131765d360e335789c1952bc6793ed051e016ea
Reviewed-on: https://chromium-review.googlesource.com/908454
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51188}
Load mjsunit.js inside the realm as otherwise the functions are not
available in the realm's scope.
This also prints timestamps after each test to easier track down slow
tests.
We also pass --omit-quit to not stop too early.
This also adds the ability to skip certain tests for endurance
fuzzing and skips some tests with known problems.
TBR=ulan@chromium.org,hpayer@chromium.org
Bug: v8:6972, v8:7400
Change-Id: I44464c28bfb10c84f2e59972e7b86945a47ca3b3
Reviewed-on: https://chromium-review.googlesource.com/899008
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51066}
Only test processors code left. It enabled to move more stuff to
the base runner, like progress indicators creation.
Bug: v8:6917
Change-Id: Ie6dd211cec561a07d92bcc4431ea88eb1842c8fa
Reviewed-on: https://chromium-review.googlesource.com/897624
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51030}
For mips, if 'mips_arch_variant=="r6"' and if 'mips_use_msa' flag is set
to 'true', then test-run-wasm-simd tests won't be skipped for mips. It
will also force 'MIPS_SIMD' bit in CpuFeatures to be set.
ARM processors are assumed to support SIMD.
Change-Id: Iea668b97ef995ca4949ddbf2ffc734aad89d3aa3
Reviewed-on: https://chromium-review.googlesource.com/868430
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#50981}
This is a reland of 0db74d4974.
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
This reverts commit 0db74d4974.
Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
This adds back an option for interrupt budget available in no-snap
builds. This also adds a fuzzer configuration for numfuzz that enables
fuzzing the interrupt budget option. A new flag --disable-analysis
allows to generally skip the fuzzer's analysis phase, which can be
chosen for interrupt budget, which doesn't support an analysis phase.
Bug: v8:6917
Change-Id: I546dd9ee41c3e0fb027108ef4606a34514f230d4
Reviewed-on: https://chromium-review.googlesource.com/885805
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50929}
Introduce option to run fuzzer processor without analysis phase.
It will be used in fuzzing combined tests.
Bug: v8:6917
Change-Id: Ic96d6b8c5a35c81da80340555bdd75c0d518cb5a
Reviewed-on: https://chromium-review.googlesource.com/880948
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50810}
This is added only to uniformly call all tools by the infra side.
NOTRY=true
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: I85a14ea51abfe1bfc775fd73d2fee02bfb2fb31e
Reviewed-on: https://chromium-review.googlesource.com/878361
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50754}
This adds flags to numfuzz for rerunning flaky tests and for storing
json test results. With those flags added, the infra-side can call
numfuzz with the same API as the standard test-runner.
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: I02d1cd02d90677c83f10e072383f3650c041cab1
Reviewed-on: https://chromium-review.googlesource.com/877890
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50753}
