This is a reland of 96f5579669
Original change's description:
> [test] Add test runner system tests for flaky tests
>
> This uses a temporary file in the mocked d8 script to let d8 flip
> between FAIL and PASS on rerun.
>
> This adds a separate test root with dedicated test suite for testing
> flakes to no interfere with existing status file configs.
>
> NOTRY=true
>
> Bug: v8:6917
> Change-Id: Id43753650195fb74cceb2a3ee9014100cabad546
> Reviewed-on: https://chromium-review.googlesource.com/867917
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50621}
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6917
Change-Id: I4e7bca272dfd8778fbb8d012fcd6fd8406158e43
Reviewed-on: https://chromium-review.googlesource.com/868433
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50625}
This reverts commit 96f5579669.
Reason for revert: breaks CI presubmit:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20presubmit/builds/22115
Original change's description:
> [test] Add test runner system tests for flaky tests
>
> This uses a temporary file in the mocked d8 script to let d8 flip
> between FAIL and PASS on rerun.
>
> This adds a separate test root with dedicated test suite for testing
> flakes to no interfere with existing status file configs.
>
> NOTRY=true
>
> Bug: v8:6917
> Change-Id: Id43753650195fb74cceb2a3ee9014100cabad546
> Reviewed-on: https://chromium-review.googlesource.com/867917
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50621}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I103bf8f0e926a3f56921f14bdd7e875ce839ea71
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/868530
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50624}
This uses a temporary file in the mocked d8 script to let d8 flip
between FAIL and PASS on rerun.
This adds a separate test root with dedicated test suite for testing
flakes to no interfere with existing status file configs.
NOTRY=true
Bug: v8:6917
Change-Id: Id43753650195fb74cceb2a3ee9014100cabad546
Reviewed-on: https://chromium-review.googlesource.com/867917
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50621}
This will act as a switch so that we can enable Node.js GN builds
via V8 change.
R=machenbach@chromium.org
Bug: v8:6105
Change-Id: Ie786a4dad7f53e4bbf7d8830bb72b1f8031cee12
Reviewed-on: https://chromium-review.googlesource.com/866839
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50612}
Also sort all variants in configuration mapping.
Bug: v8:7285
Change-Id: I2d3032b911e48c34557e47e50345fb29da7bbe85
Reviewed-on: https://chromium-review.googlesource.com/866502
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50584}
This will add three gc fuzzing flags with 5% likelihood each to the second
correctness fuzzing config. The random checks are determined by the
top-level random-seed passed to the script.
This change depends on setting the fuzzer random seed as default to the
standard random seed, since the former isn't explicitly passed by
clusterfuzz.
NOTRY=true
TBR=hpayer@chromium.org
Bug: v8:7012
Change-Id: I794dc48bb953b6a95bbc4fc4305ad561bc13b6ee
Reviewed-on: https://chromium-review.googlesource.com/865912
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50577}
This reverts commit 2de796fc57.
Reason for revert: Fails tests on Linux dbg:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/18708
Original change's description:
> [test] Add more d8-based test suites to d8_default
>
> This will run those suites with gc stress, code serializer, and verify-
> predictable mode.
>
> TBR=sergiyb@chromium.org
>
> Bug: v8:7285
> Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
> Reviewed-on: https://chromium-review.googlesource.com/864922
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50555}
TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I8d769829bdb96820e57b1cefcf92fb6976a1031d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7285
Reviewed-on: https://chromium-review.googlesource.com/865334
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50556}
This will run those suites with gc stress, code serializer, and verify-
predictable mode.
TBR=sergiyb@chromium.org
Bug: v8:7285
Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
Reviewed-on: https://chromium-review.googlesource.com/864922
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50555}
This adds a reduced version of bot_default with only d8-based test suites.
Those can be used for gc stress testing, fuzzing and predictable mode.
This also adds a missing explicit build rule for the debugger test suite.
It's working as it's part of the isolate file, but incremental
re-isolation might break on code changes.
Running the tests is still commented out to allow the infra-side to
land. Afterwards, we'll activate the extra testing as a V8 CL
commenting the lines back in.
Bug: v8:7285
Change-Id: I7b758c212f0c4ae3d2f79beea5ec597cee479565
Reviewed-on: https://chromium-review.googlesource.com/864045
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50545}
- Turbolizer highlights input and output nodes on hover.
- The three panes support resizing now (snap to side still works).
Bug:
Change-Id: Ida1513fd714a02ab772885ea1fdf6d9da8d540f6
Reviewed-on: https://chromium-review.googlesource.com/837068
Commit-Queue: Sigurd Schneider <sigurds@chromium.org>
Reviewed-by: Daniel Clifford <danno@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50523}
Also change fetch_deps.py to no longer be a no-op and rename
Main function for importing from other scripts.
R=machenbach@chromium.org
Bug: v8:6105
Change-Id: I067a212827316248f60e97ff27e9bb2dc20addfd
Reviewed-on: https://chromium-review.googlesource.com/860007
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50522}
This removes the explicit --write-protect-code-memory flag from the list
of flags in the "stress_incremental_marking". The feature is enabled by
default by now and no longer needs explicit testing.
R=ulan@chromium.org
BUG=v8:6792
Change-Id: I5d6ba21dff261488bbe1b0148ce204bf78d57334
Reviewed-on: https://chromium-review.googlesource.com/860661
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50500}
In collaboration with Qingyan Li <qingyan.liqy@alibaba-inc.com>.
R=jgruber@chromium.org, mlippautz@chromium.org
Bug: v8:7249
Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
Change-Id: I87f62103ec5b31de274fa22ad275f1c1bcb3ed86
Reviewed-on: https://chromium-review.googlesource.com/846750
Commit-Queue: Yang Guo <yangguo@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50433}
Temporary workaround to avoid the same variant name for multiple
variants of the same test.
Bug: v8:6917
Change-Id: I9a25dcaf81d35da0dc2617c089cb4811c2a958cb
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/852833
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50428}
Test processors can be enabled with --infra-staging flag.
Rerunning tests, execution and verbose progress indicator already
work as test processors.
Bug: v8:6917
Change-Id: I40fc42db94dbc8629e8000a3d363030045532fe3
Reviewed-on: https://chromium-review.googlesource.com/850398
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50399}
Turns out gclient calls download_from_google_storage, which needs to be
included in PATH.
TBR=machenbach@chromium.org
Bug: v8:6105
Change-Id: I15d44d67152f6fa0d20ae7bc7e44bc5e0393b519
Reviewed-on: https://chromium-review.googlesource.com/852616
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50389}
I added additional exception logging in the execution.py since
errors in processing results were really difficult to debug.
There is a problem on Windows with class serialization when
it comes from dynamically loaded module. To fix it I moved all
output processors to the tools/testrunner/outproc/ and import
them in test/*/testcfg.py.
Bug: v8:6917
Change-Id: Ida604641d659b006e91faf1d56a37769ec47f5f3
Reviewed-on: https://chromium-review.googlesource.com/842784
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50361}
This reverts commit 1685b5d27a.
Reason for revert: Was probably caused by infra change:
https://crrev.com/c/845781
Original change's description:
> Revert "[test] Move has unexpected output to outproc."
>
> This reverts commit 71605b3ea4.
>
> Reason for revert: Seems to break static-initializers step:
> https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
>
> Original change's description:
> > [test] Move has unexpected output to outproc.
> >
> > Expected outcomes optimized to serialize [PASS] as None.
> >
> > Keeping expected outcomes inside output processors should be
> > optimized in the future. Few possible optimizations:
> > - separate classes for tests that are expected to PASS - done as
> > an example in mozilla test suite.
> > - cache output processors inside testcase.
> > - share output processors between copies of the same test - needs
> > some updates to the create_variant to update outproc only if
> > expected outcomes changed.
> >
> > Bug: v8:6917
> > Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> > Reviewed-on: https://chromium-review.googlesource.com/843025
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#50347}
>
> TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
>
> Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6917
> Reviewed-on: https://chromium-review.googlesource.com/849572
> Reviewed-by: Bill Budge <bbudge@chromium.org>
> Commit-Queue: Bill Budge <bbudge@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50348}
TBR=bbudge@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I7a522b6487de6e96985d223524533493eb9171f0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/848975
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50350}
This reverts commit 71605b3ea4.
Reason for revert: Seems to break static-initializers step:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
Original change's description:
> [test] Move has unexpected output to outproc.
>
> Expected outcomes optimized to serialize [PASS] as None.
>
> Keeping expected outcomes inside output processors should be
> optimized in the future. Few possible optimizations:
> - separate classes for tests that are expected to PASS - done as
> an example in mozilla test suite.
> - cache output processors inside testcase.
> - share output processors between copies of the same test - needs
> some updates to the create_variant to update outproc only if
> expected outcomes changed.
>
> Bug: v8:6917
> Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> Reviewed-on: https://chromium-review.googlesource.com/843025
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50347}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/849572
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50348}
Expected outcomes optimized to serialize [PASS] as None.
Keeping expected outcomes inside output processors should be
optimized in the future. Few possible optimizations:
- separate classes for tests that are expected to PASS - done as
an example in mozilla test suite.
- cache output processors inside testcase.
- share output processors between copies of the same test - needs
some updates to the create_variant to update outproc only if
expected outcomes changed.
Bug: v8:6917
Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
Reviewed-on: https://chromium-review.googlesource.com/843025
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50347}
This seperates OS-specific code of the Command class into a Posix and a
WindowsCommand to simplify scattered OS checks.
This also removes some temporary mac debug output that's obsolete after
https://crbug.com/v8/6927 got resolved.
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: Iaab3b527ce556dfba797a164ae58e8dd358eb56f
Reviewed-on: https://chromium-review.googlesource.com/847000
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50346}
This replaces multiprocessing with threading in tests to make python
coverage work.
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: Idff763dfefa4a7fc782133d94089b3a5b00a194d
Reviewed-on: https://chromium-review.googlesource.com/844735
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50344}
The tests already run on upload and commit. This makes them run on the
corresponding continuous tester.
NOTRY=true
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: I7dedd482ea54f0879854055c97b27f825f06d6e2
Reviewed-on: https://chromium-review.googlesource.com/846807
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50343}
