This enable test suites to check which test driver framework is used.
When using number fuzzer on mjsunit, we add a JS file that
switches off the optimization-state assertions.
Checking intrinsic state is not feasible on the number fuzzer and in
the past, we needed to skip tests on demand, which is a maintenance
burden. The main function of the fuzzer, to check for dcheck errors and
tsan issues, is retained.
Bug: v8:9127
Change-Id: I699b85d5f7c9aaed337a2130d9eddc160c059d7b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1565892
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60816}
Using test generators meant that we had to remove the progress indicator since
the total number of tests weren't known before-hand.
This CL implements a progress indicator using test number estimations.
cctest and unittests progress indicator is accurate, however estimating
means the progress will terminate over 100% in big test suites and sometimes
under 100%.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8769
Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
Reviewed-on: https://chromium-review.googlesource.com/c/1460471
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59538}
V8 testrunner is loading every test it has to run into memory greedily in order
to sort by slowness of the test case. The memory and CPU overhead for loading
the test-suites are non-trivial.
This CL restructures it by changing the sorting method.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8174
Change-Id: I08331182147b92cf4ac54823eea0e2b472f51e84
Reviewed-on: https://chromium-review.googlesource.com/c/1406684
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58821}
TestSuite has a static method LoadTestSuite that should properly configure the
TestSuite instance (i.e. loaded status files and tests), however the method
leaves some configuration logic to the caller.
The leaky abstraction causes the caller to do a bunch of loading operations (see
the removed methods in base_runner.py).
This CL isolates the TestSuite loading logic to the static method only.
This is a refactoring only change without any intended logical changes.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I105059c9c9e050f03bb584174e2bd7ceeae2b228
Reviewed-on: https://chromium-review.googlesource.com/c/1396417
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58744}
- Removed the old test.
- Created a fake test suite and added a test for loading it with a TestConfig
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: Ib7587ceec9e31ecd4cb8f45c3158e73c79a9bc5b
Reviewed-on: https://chromium-review.googlesource.com/c/1396082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58593}
Fix unittests since they were incompatible with the
new testcase and testsuite API.
Bug: v8:6917
Change-Id: I917bf58e21402e0b90bc91c0483ade0e7c90bdd6
Reviewed-on: https://chromium-review.googlesource.com/832392
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50201}
- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
Filtering by status file split to four parts:
1. Getting outcomes - reads both variant dependent and
independent outcomes, no more need to do it twice.
2. Checking unused rules - has a switch to check only variant
dependent/independent rules.
3. Reading flags - if outcome starts with '--' it is treated as a flag.
4. Actual filtering.
Outcomes removed from the testcase object, can be accessed
by call to its testsuite.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I35762f891010ddda926250452b88656047433daa
Reviewed-on: https://chromium-review.googlesource.com/775160
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49547}
Test suite contract changes:
- support * only at the end of the rule.
- loading status file is mandatory before filtering by status file.
Bug: v8:6917
Change-Id: Ia345ebfa7827c50f13f20e5cb7489e62c53f3357
Reviewed-on: https://chromium-review.googlesource.com/779185
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49496}
This adds the possibility to address test cases in the
status file with the variant under which the test is running.
This is only allowed in top-level sections.
Example:
[{
'test-case': [PASS, SLOW],
}]
['variant == foo', {
'test-case': [FAIL],
}]
The test case "test-case" is marked as slow in all variants.
Additionally, in variant foo, it'll be expected to fail.
This CL also exemplifies the new feature with test cases
running under the ignition_turbofan variant. The
corresponding legacy flag is deprecated.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2203013002
Cr-Commit-Position: refs/heads/master@{#38342}
