Also enable test runner to differentiate between clang and gcc.
Bug: v8:8919
Change-Id: Icdcae0aba3644a1b1b9ddc6c037eabde27d717f7
Reviewed-on: https://chromium-review.googlesource.com/c/1491634
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59906}
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
This CL was uploaded by git cl split.
Bug: v8:8594
Change-Id: I661c52a70527e8ddde841fee6d4dcba282b4a938
Reviewed-on: https://chromium-review.googlesource.com/c/1470123
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59675}
Using test generators meant that we had to remove the progress indicator since
the total number of tests weren't known before-hand.
This CL implements a progress indicator using test number estimations.
cctest and unittests progress indicator is accurate, however estimating
means the progress will terminate over 100% in big test suites and sometimes
under 100%.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8769
Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
Reviewed-on: https://chromium-review.googlesource.com/c/1460471
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59538}
Testrunner test-suite doesn't have automated tests for the Fuzzer. A significant
change for the testrunner may break the fuzzers. We don't have a way to catch
even the syntax errors for the fuzzer before landing CLs.
This CL creates a rudimentary test-suite that runs the fuzzer without any tests
to see if APIs still work as intended.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
No-Try: true
Bug: v8:8763
Change-Id: Ib4519ca093de2af5279964edd164cee04aae77a8
Reviewed-on: https://chromium-review.googlesource.com/c/1466301
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59535}
Testrunner runs variants of a test sequentially without taking sharding into
account. A slow test with slow variants slows down the whole test run no matter
the sharding configuration.
This CL implements a test hashing algorithm and variant sharding for test
variants.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I15f8c547fa2f361fb6c53bf8d5df055d3df38d3e
Reviewed-on: https://chromium-review.googlesource.com/c/1458016
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59530}
The test name alone isn't informative alone.
This CL prints the variant name during the progress reporting.
tested by running test262 with all variants
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8728
Change-Id: I3d7d1fc6eacef3712d87404c7276fd5523bb445a
Reviewed-on: https://chromium-review.googlesource.com/c/1458156
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59437}
This is a reland of 81eec150f6
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
Bug: v8:8174, v8:8790
Change-Id: I38ab9d37bca76057441a970f26e2102e4387a857
Reviewed-on: https://chromium-review.googlesource.com/c/1454724
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59387}
This reverts commit 81eec150f6.
Reason for revert: windows mozilla test failures
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org
Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
This is a reland of 7f92ad0ab6
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
This reverts commit 7f92ad0ab6.
Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org
Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
Every testsuite configuration consist of at least 30% code duplication.
The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.
This CL removes all the duplication by refactoring the common functionality.
This CL contains structural changes without any logical changes % small bug
fixes.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
With and without infra_staging flag testrunner behaves the same for old
features. This CL removes duplicate tests testing the same behavior.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
No-Try: true
Bug: v8:8174
Change-Id: Icf7bea15b2343b90697016d050fa0d918a99997d
Reviewed-on: https://chromium-review.googlesource.com/c/1424859
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58951}
Moving to a lazy test loading approach makes counting the total number of tests
non-trivial.
For CI runs, we output the total number of tests after the run.
For users, progress indicator signals the status of the run.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I0731964515aac60a3629acee6c7243433a2b4e04
Reviewed-on: https://chromium-review.googlesource.com/c/1420677
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58944}
testrunner has tests asserting about the implementation details, assertions
about the behavior of the testrunner (already in-place) provides robust coverage
already.
This cl remove the brittle assertions.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I6583c971b7cf7eb2eb7dfa2b6737d6aa67957feb
Reviewed-on: https://chromium-review.googlesource.com/c/1421359
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58928}
Adds a flag to specify whether to disable the linter caching.
R=machenbach@chromium.org,sergiyb@chromium.org
CC=yangguo@chromium.org
Bug: v8:8482
Change-Id: I62a9b7cffb3adb50b136659568ad52078675ca4b
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/1370029
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58329}
Added tests for the existing FileContentsCache, and created a superclass
that removes the duplicated code from Torque and CPP linters
R=machenbach@chromium.org,sergiyb@chromium.org
CC=yangguo@chromium.org
NOTRY=true
Bug: v8:8482
Change-Id: Ic7a0b3d58c64f395e790d4ff668fa804c05478be
Reviewed-on: https://chromium-review.googlesource.com/c/1369949
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58321}
The purpose of this is to simplify tools tests imports.
For example, with this commit, 'from ..v8_presubmit import TestClass' is
possible in 'tools/unittests/v8_presubmit_test.py'.
R=machenbach@chromium.org,sergiyb@chromium.org
Change-Id: Ib19a6e3af4027095c72d4c87a9d759ac85a2e1a0
Reviewed-on: https://chromium-review.googlesource.com/c/1379934
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58276}
This commit adds a --run-skipped flag to the test runner that will
bypass the 'SKIP' status.
Bug: v8:8485
Change-Id: Iac012bdaf2de6b0f8e44ed3a65bc9330709527bb
Reviewed-on: https://chromium-review.googlesource.com/c/1346490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Sigurd Schneider <sigurds@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57710}
We define a TestFailedError exception and raise it when we can reliably detect
that a test has crashed. All other exceptions are treated as infra failures and
are captured by the try-catch clause in MainWrapper function.
This also fixes all tests in run_perf_test.py, run_tests_test.py and makes sure
that both are run on any changes in tools directory.
R=machenbach@chromium.org
Bug: chromium:899028
Change-Id: I283bc87b31c814be476bebe9fdda414975494183
Reviewed-on: https://chromium-review.googlesource.com/c/1303293
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57134}
This is a reland of 4c0943424c
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
Bug: chromium:866862
Change-Id: Icf7e04c75d4abeab7254d10ba21240e46b0022ae
Reviewed-on: https://chromium-review.googlesource.com/1170643
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55059}
This reverts commit 4c0943424c.
Reason for revert: Unfortunately this broke all perf builders.
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org
Change-Id: If80129810586b709dab762c9b5724888e15daec2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:866862
Reviewed-on: https://chromium-review.googlesource.com/1170962
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55058}
This adds a new command abstraction for running commands on Android
using dockered devices on swarming.
The new abstraction handles pushing all required files to the device.
The logic used for pushing and running is reused from the perf runner.
This adds only the mjsunit test suite. Others will be handled in
follow up CLs. The suite logic is enhanced with auto-detection of files
to be pushed to devices, for e.g. load or import statements.
Some test cases need an extra resource section for specifying required
files.
Remaining failing tests are marked in the status files for later
triage.
Bug: chromium:866862
Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
Reviewed-on: https://chromium-review.googlesource.com/1150153
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55041}
This is a partial revert of:
https://crrev.com/c/890938 and https://crrev.com/c/893982
Before this CL, the test runner blocked on ongoing tests in order to
process their results after an internal timeout. However, the logic
required for this feature was overly complicated and prevented an
acceptable implementation for fast aborts. Furthermore, also the fuzzers
suffered from timeouts on swarming due to hanging tests.
Instead, we now abort immediately on internal timeout (used on
fuzzers), SIGINT (Ctrl-C) and SIGTERM. Ongoing tests are immediately
terminated and their results are disregarded. On SIGTERM and SIGINT,
we return with non-zero exit codes, and zero on internal timeout.
This will also properly return json output, when the external hard
timeout is reached on swarming (causes SIGTERM).
TBR=sergiyb@chromium.org
Bug: v8:7423, chromium:813065
Change-Id: Ib20f835f58a0970693bdd3b21dc5d766d8e115d8
Reviewed-on: https://chromium-review.googlesource.com/924852
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51399}
System tests updated to pass. I will cleanup the code in a
separate CL removing the old code.
Bug: v8:6917
Change-Id: Ie49efe06c43faade846e322b35c4a2068137a88b
Reviewed-on: https://chromium-review.googlesource.com/897565
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51025}
This is a reland of 0db74d4974.
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
This reverts commit 0db74d4974.
Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
This will affect all manual test runs with x64. Most bots on x64 already
migrated.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:7343
Change-Id: I87f46f1848a813c0b320b3e9901481b9232025a5
Reviewed-on: https://chromium-review.googlesource.com/883101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50825}
Since test processors create tests dynamically we cannot simply
count how many tests will be run. Instead we count only base tests
that we've loaded, before creating variants.
Bug: v8:6917
Change-Id: Ibc5b9a73f6afad423572afa575f477ca661a99d5
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/868290
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50642}
This is a reland of 96f5579669
Original change's description:
> [test] Add test runner system tests for flaky tests
>
> This uses a temporary file in the mocked d8 script to let d8 flip
> between FAIL and PASS on rerun.
>
> This adds a separate test root with dedicated test suite for testing
> flakes to no interfere with existing status file configs.
>
> NOTRY=true
>
> Bug: v8:6917
> Change-Id: Id43753650195fb74cceb2a3ee9014100cabad546
> Reviewed-on: https://chromium-review.googlesource.com/867917
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50621}
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6917
Change-Id: I4e7bca272dfd8778fbb8d012fcd6fd8406158e43
Reviewed-on: https://chromium-review.googlesource.com/868433
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50625}
This reverts commit 96f5579669.
Reason for revert: breaks CI presubmit:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20presubmit/builds/22115
Original change's description:
> [test] Add test runner system tests for flaky tests
>
> This uses a temporary file in the mocked d8 script to let d8 flip
> between FAIL and PASS on rerun.
>
> This adds a separate test root with dedicated test suite for testing
> flakes to no interfere with existing status file configs.
>
> NOTRY=true
>
> Bug: v8:6917
> Change-Id: Id43753650195fb74cceb2a3ee9014100cabad546
> Reviewed-on: https://chromium-review.googlesource.com/867917
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50621}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I103bf8f0e926a3f56921f14bdd7e875ce839ea71
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/868530
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50624}
This uses a temporary file in the mocked d8 script to let d8 flip
between FAIL and PASS on rerun.
This adds a separate test root with dedicated test suite for testing
flakes to no interfere with existing status file configs.
NOTRY=true
Bug: v8:6917
Change-Id: Id43753650195fb74cceb2a3ee9014100cabad546
Reviewed-on: https://chromium-review.googlesource.com/867917
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50621}
