- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
Since we have only unittests that are under GoogleTestSuite there
is no need to keep it as a default suite and we can make it
specific for unittests.
Bug: v8:6917
Change-Id: Ie2d57342773f228dea72184ab0f2abfc9d2daa70
Reviewed-on: https://chromium-review.googlesource.com/819253
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50011}
This moves the verify-predictable logic from the test runner into
a python wrapper script.
This revealed two more tests that don't print allocations, which are
now skipped.
Bug: v8:7166, v8:7177
Change-Id: Ie4a541cb2a20900414ffe1caf4b3fccc4a5edb52
Reviewed-on: https://chromium-review.googlesource.com/808971
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49964}
This is a reland of 3b06511052
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
Bug: v8:6917
Change-Id: Ia39010a0a0f63537ad12490dfab17897d70d4930
Reviewed-on: https://chromium-review.googlesource.com/806034
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49830}
This reverts commit 3b06511052.
Reason for revert: Broke fuzzers:
https://build.chromium.org/p/client.v8.clusterfuzz/builders/V8%20Deopt%20Fuzzer/builds/384
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: v8:6917
Change-Id: I4938642c4396366be1e13daf6998c4b8538b688b
Reviewed-on: https://chromium-review.googlesource.com/804254
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49805}
This is a reland of 98cc9e862f
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
Bug: v8:6917
Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
Reviewed-on: https://chromium-review.googlesource.com/800370
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49756}
This reverts commit 98cc9e862f.
Reason for revert: Breaks test isolation:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20builder/builds/29746
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I44b99468d18fd093833f4185dad067a9eeaf2bc1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/800292
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49747}
Immutable command class with shell, flags and
environment.
Command creation moved from worker to the main
process. Because of that there is no need to send
test cases beyond process boundaries and load test
suites in worker processes.
Bug: v8:6917
Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
Reviewed-on: https://chromium-review.googlesource.com/791710
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49746}
Filtering by status file split to four parts:
1. Getting outcomes - reads both variant dependent and
independent outcomes, no more need to do it twice.
2. Checking unused rules - has a switch to check only variant
dependent/independent rules.
3. Reading flags - if outcome starts with '--' it is treated as a flag.
4. Actual filtering.
Outcomes removed from the testcase object, can be accessed
by call to its testsuite.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I35762f891010ddda926250452b88656047433daa
Reviewed-on: https://chromium-review.googlesource.com/775160
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49547}
This is a reland of 712fa67554.
Original change's description:
> [test] Add Liftoff variant
>
> Add a variant for testing the current state of the Liftoff
> implementation.
> This variant will only run on a subset of the bots, just like the
> --future variant.
>
> R=machenbach@chromium.org, hablich@chromium.org
>
> Bug: v8:7088, v8:6600
> Change-Id: If49fad3a8ed579356504b821a787326754f24e78
> Reviewed-on: https://chromium-review.googlesource.com/779420
> Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49504}
TBR=machenbach@chromium.orgCC=hablich@chromium.org
Bug: v8:7088, v8:6600
Change-Id: Ieb20020f07c70acaa64bb421763a41aa163a261b
Reviewed-on: https://chromium-review.googlesource.com/781499
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49531}
This reverts commit 712fa67554.
Reason for revert: Infra side change was reverted. Please reland
after it stays in for a bot cycle...
Original change's description:
> [test] Add Liftoff variant
>
> Add a variant for testing the current state of the Liftoff
> implementation.
> This variant will only run on a subset of the bots, just like the
> --future variant.
>
> R=machenbach@chromium.org, hablich@chromium.org
>
> Bug: v8:7088, v8:6600
> Change-Id: If49fad3a8ed579356504b821a787326754f24e78
> Reviewed-on: https://chromium-review.googlesource.com/779420
> Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49504}
TBR=machenbach@chromium.org,hablich@chromium.org,clemensh@chromium.org
Change-Id: Ib6b2e79cea5d9f99f8933c72bbb9d9dddbd6ae07
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7088, v8:6600
Reviewed-on: https://chromium-review.googlesource.com/779719
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49507}
Add a variant for testing the current state of the Liftoff
implementation.
This variant will only run on a subset of the bots, just like the
--future variant.
R=machenbach@chromium.org, hablich@chromium.org
Bug: v8:7088, v8:6600
Change-Id: If49fad3a8ed579356504b821a787326754f24e78
Reviewed-on: https://chromium-review.googlesource.com/779420
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49504}
Adds a stress-background-compile mode which runs compilation on a background
thread at the same time as compiling on the main thread to flush out races.
This is added as one of the variants, removing stress_asm_validation since this
is no longer a useful variant.
BUG=v8:5203
Change-Id: I73dd9f21d9b6da4a4cb39c05061aa24116e6c5ad
Reviewed-on: https://chromium-review.googlesource.com/741746
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49376}
This is a reland of 697b2c09a5
Original change's description:
> Setting up the environment for testrunner refactored
>
> Bug: v8:6917
> Change-Id: Ibc3c738ef807d37d8b76f440d9765c4d0405c021
> Reviewed-on: https://chromium-review.googlesource.com/735421
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48923}
Bug: v8:6917
Change-Id: I07918814ee8277eca96bcb2d330e6b08f6a0505d
Reviewed-on: https://chromium-review.googlesource.com/741621
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49078}
This reverts commit 4054cf278f.
Reason for revert: Just exposes existing issues.
Original change's description:
> Revert "[test] Fix win-asan symbolizer path"
>
> This reverts commit 135576ffb6.
>
> Reason for revert: V8 Win32 ASAN failures: https://build.chromium.org/p/client.v8/builders/V8%20Win32%20ASAN/builds/73
>
> It appears these failures were lurking there already, but were hidden because of the bug this CL fixed. Opened https://crbug.com/v8/6953 about these issues.
>
> Original change's description:
> > [test] Fix win-asan symbolizer path
> >
> > This makes the symbolizer path relative, as the absolute paths contain
> > a drive letter + colon on windows. The colon is confused by the
> > sanitizer as an option separator.
> >
> > The test driver changes the cwd to the V8 root dir in each
> > invocation.
> >
> > Bug: chromium:726584
> > Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
> > Reviewed-on: https://chromium-review.googlesource.com/721124
> > Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#48652}
>
> TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,etienneb@chromium.org
>
> Change-Id: Ic78527950f6a239a03658e042d7244c9781d05db
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: chromium:726584
> Reviewed-on: https://chromium-review.googlesource.com/723825
> Reviewed-by: Eric Holk <eholk@chromium.org>
> Commit-Queue: Eric Holk <eholk@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48653}
TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,eholk@chromium.org,sergiyb@chromium.org,etienneb@chromium.org
Change-Id: I8ea3b1d74ece09bed4758522f51cbee56a7792e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:726584
Reviewed-on: https://chromium-review.googlesource.com/725319
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48662}
This reverts commit 135576ffb6.
Reason for revert: V8 Win32 ASAN failures: https://build.chromium.org/p/client.v8/builders/V8%20Win32%20ASAN/builds/73
It appears these failures were lurking there already, but were hidden because of the bug this CL fixed. Opened https://crbug.com/v8/6953 about these issues.
Original change's description:
> [test] Fix win-asan symbolizer path
>
> This makes the symbolizer path relative, as the absolute paths contain
> a drive letter + colon on windows. The colon is confused by the
> sanitizer as an option separator.
>
> The test driver changes the cwd to the V8 root dir in each
> invocation.
>
> Bug: chromium:726584
> Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
> Reviewed-on: https://chromium-review.googlesource.com/721124
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48652}
TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,etienneb@chromium.org
Change-Id: Ic78527950f6a239a03658e042d7244c9781d05db
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:726584
Reviewed-on: https://chromium-review.googlesource.com/723825
Reviewed-by: Eric Holk <eholk@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48653}
This makes the symbolizer path relative, as the absolute paths contain
a drive letter + colon on windows. The colon is confused by the
sanitizer as an option separator.
The test driver changes the cwd to the V8 root dir in each
invocation.
Bug: chromium:726584
Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
Reviewed-on: https://chromium-review.googlesource.com/721124
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48652}
Also fix asan environment for testing on windows.
TBR=sergiyb@chromium.org
Bug: chromium:726584
Change-Id: Ic9e6afa714f4757ad1b0f2ebfa742e742e1c04b9
Reviewed-on: https://chromium-review.googlesource.com/720811
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48583}
Preparation for sharing more code between deopt fuzzer and normal test runner.
Bug: v8:6917
Change-Id: Id8022c7597544ffddf5d1b0a0d73bb4a9d163fc3
Reviewed-on: https://chromium-review.googlesource.com/715417
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48521}
