Testrunner runs variants of a test sequentially without taking sharding into
account. A slow test with slow variants slows down the whole test run no matter
the sharding configuration.
This CL implements a test hashing algorithm and variant sharding for test
variants.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I15f8c547fa2f361fb6c53bf8d5df055d3df38d3e
Reviewed-on: https://chromium-review.googlesource.com/c/1458016
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59530}
The test name alone isn't informative alone.
This CL prints the variant name during the progress reporting.
tested by running test262 with all variants
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8728
Change-Id: I3d7d1fc6eacef3712d87404c7276fd5523bb445a
Reviewed-on: https://chromium-review.googlesource.com/c/1458156
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59437}
V8 test suites return tests by loading them into memory up-front. Up-front
loading has memory and loading-time overhead for the testrunner.
This CL converts the test-suites to load tests during the test run in parallel.
After CL, most test suites start running immediately and all test suites start
running after 3 seconds on a modern SSD.
This CL eliminates the 200 MiB memory spikes that causes memory problems in mac
minis.
Overhead of running test262 before CL:
Line # Mem usage Increment Line Contents
=================================================
264 225.760 MiB 195.8.000 MiB tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
Overhead of running test262 after CL:
Line # Mem usage Increment Line Contents
================================================
264 28.840 MiB 0.000 MiB tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174, v8:8552, v8:8728
Change-Id: Iab540b9410239b05dc80b4a5228db25d8b8fb3b9
Reviewed-on: https://chromium-review.googlesource.com/c/1454478
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59408}
This is a reland of 81eec150f6
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
Bug: v8:8174, v8:8790
Change-Id: I38ab9d37bca76057441a970f26e2102e4387a857
Reviewed-on: https://chromium-review.googlesource.com/c/1454724
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59387}
This reverts commit 81eec150f6.
Reason for revert: windows mozilla test failures
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org
Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
This is a reland of 7f92ad0ab6
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
This reverts commit 7f92ad0ab6.
Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org
Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
Every testsuite configuration consist of at least 30% code duplication.
The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.
This CL removes all the duplication by refactoring the common functionality.
This CL contains structural changes without any logical changes % small bug
fixes.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
This reverts commit 6e03d7ee42.
Reason for revert: This breaks the Android bot:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2933
It's quite hidden due to https://crbug.com/v8/8731 but all shards
time out. It looks like to to this change, testing takes
dramatically longer, maybe the pushing takes now much longer than
before. If we want decide for this, the builder needs to get
many more shards.
Original change's description:
> Reland "[tools] Push files using high-level device.PushChangedFiles method"
>
> This is a reland of d045f66682
>
> Original change's description:
> > [tools] Push files using high-level device.PushChangedFiles method
> >
> > R=machenbach@chromium.org
> >
> > No-Try: true
> > Bug: chromium:893593
> > Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> > Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#58407}
>
> Bug: chromium:893593
> Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
> Reviewed-on: https://chromium-review.googlesource.com/c/1443055
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59221}
TBR=machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:893593
Change-Id: Ifea307b5de8f39b660966fc6bef54601df91d841
Reviewed-on: https://chromium-review.googlesource.com/c/1450119
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59305}
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.
This CL eliminates that overhead by utilizing generators and threading.
LoadingProc now loads test after receiving the results of the loaded tests.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174,v8:8731
Change-Id: Ifee79c3e213da568f092de0f1623016174e9410c
Reviewed-on: https://chromium-review.googlesource.com/c/1439240
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59223}
Procs return the result by increasing recursion through result_for.
This CL eliminates that mechanism from the Processor interface and uses boolen
return values for sending tests to signal success or the failure to load the
test into the execution queue.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174,v8:8731
Change-Id: I073a86ca84bcf88da11132b90013d4c8455bc61e
Reviewed-on: https://chromium-review.googlesource.com/c/1439239
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59201}
This adds the new runtime flag as a variant and runs it on a subset of
builders corresponding to the "extra" testing set.
Currently failing tests are skipped in the new variant.
After https://crrev.com/c/1433777 this costs only little additional
resources.
Bug: v8:8678
Change-Id: Ibd0e38872814d11252e55a7c6a58d313aa84ebe3
Reviewed-on: https://chromium-review.googlesource.com/c/1433774
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59080}
This reverts commit 25457c60a7.
Reason for revert: https://crbug.com/v8/8731
Original change's description:
> [testrunner] load tests concurrently into test execution processor
>
> loading every test up-front into the processing queue costs about 224MB for a
> x64 testsuite run.
>
> This CL eliminates that overhead by utilizing generators and threading.
>
> LoadingProc now loads test after receiving the results of the loaded tests.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174
> Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
> Reviewed-on: https://chromium-review.googlesource.com/c/1420678
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59056}
TBR=machenbach@chromium.org,tmrts@chromium.org
Change-Id: I1e074a031dced367a32a93827b9e863b0331340f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174
Reviewed-on: https://chromium-review.googlesource.com/c/1433792
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59065}
Previously, trusted (or no-mitigations) has been tested on a subset of builders
from all platforms. This reduces it to arm-sim and native Android devices.
Change-Id: I90066686e6a92db4a944025538e01a117f324421
Reviewed-on: https://chromium-review.googlesource.com/c/1433777
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59062}
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.
This CL eliminates that overhead by utilizing generators and threading.
LoadingProc now loads test after receiving the results of the loaded tests.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
Reviewed-on: https://chromium-review.googlesource.com/c/1420678
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59056}
Testrunner has ancient support for JUnit compatible XML output.
This CL removes this old feature.
R=mstarzinger@chromium.org,jgruber@chromium.org,jkummerow@chromium.org
CC=machenbach@chromium.org
Bug: v8:8728
Change-Id: I7e1beb011dbaec3aa1a27398a5c52abdd778eaf0
Reviewed-on: https://chromium-review.googlesource.com/c/1430065
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59045}
Moving to a lazy test loading approach makes counting the total number of tests
non-trivial.
For CI runs, we output the total number of tests after the run.
For users, progress indicator signals the status of the run.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I0731964515aac60a3629acee6c7243433a2b4e04
Reviewed-on: https://chromium-review.googlesource.com/c/1420677
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58944}
V8 testrunner is loading every test it has to run into memory greedily in order
to sort by slowness of the test case. The memory and CPU overhead for loading
the test-suites are non-trivial.
This CL restructures it by changing the sorting method.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8174
Change-Id: I08331182147b92cf4ac54823eea0e2b472f51e84
Reviewed-on: https://chromium-review.googlesource.com/c/1406684
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58821}
TestSuite has a static method LoadTestSuite that should properly configure the
TestSuite instance (i.e. loaded status files and tests), however the method
leaves some configuration logic to the caller.
The leaky abstraction causes the caller to do a bunch of loading operations (see
the removed methods in base_runner.py).
This CL isolates the TestSuite loading logic to the static method only.
This is a refactoring only change without any intended logical changes.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I105059c9c9e050f03bb584174e2bd7ceeae2b228
Reviewed-on: https://chromium-review.googlesource.com/c/1396417
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58744}
- Removed the old test.
- Created a fake test suite and added a test for loading it with a TestConfig
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: Ib7587ceec9e31ecd4cb8f45c3158e73c79a9bc5b
Reviewed-on: https://chromium-review.googlesource.com/c/1396082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58593}
Also disables --stress-flush-bytecode on some mjsunit tests which fail
when bytecode flushing is stressed due to test invariants.
Bug=v8:8395
Change-Id: If627910214b3c266e7776340ba182829148e8289
Reviewed-on: https://chromium-review.googlesource.com/c/1372071
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58230}
The additional variant does not use wrapper disabling phase tests and negated
outcome processor. This allows to ensure that tests marked FAIL_PHASE_ONLY, do
actually fail without it.
R=machenbach@chromium.org
Bug: v8:8467
Change-Id: I66e07bd7107520872cc013bf0f33fdc6664baf56
Reviewed-on: https://chromium-review.googlesource.com/c/1361164
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58211}
This commit adds a --run-skipped flag to the test runner that will
bypass the 'SKIP' status.
Bug: v8:8485
Change-Id: Iac012bdaf2de6b0f8e44ed3a65bc9330709527bb
Reviewed-on: https://chromium-review.googlesource.com/c/1346490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Sigurd Schneider <sigurds@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57710}
This reverts commit f401cd4b2c.
Reason for revert: did not help
Original change's description:
> [tools] Re-land: Add retries when trying to discover the device
>
> This is an experimental change that may help mitigate the issue.
>
> TBR=machenbach@chromium.org
>
> No-Try: true
> No-Tree-Checks: true
> Bug: chromium:893593
> Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
> Reviewed-on: https://chromium-review.googlesource.com/c/1339401
> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57558}
TBR=machenbach@chromium.org,sergiyb@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:893593
Change-Id: I30c529a627d1e6fa52099939c5c209110e9d0eb9
Reviewed-on: https://chromium-review.googlesource.com/c/1342931
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57650}
This is an experimental change that may help mitigate the issue.
TBR=machenbach@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
Reviewed-on: https://chromium-review.googlesource.com/c/1339401
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57558}
This is an experimental change that may help mitigate the issue.
TBR=machenbach@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Ideb74a83b9937dbe917e8c7c93305d9824b48a93
Reviewed-on: https://chromium-review.googlesource.com/c/1339419
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57556}
Also add the ability to skip tests only when --optimize-for-size is passed.
TBR=sergiyb@chromium.org
Bug: v8:7819
Change-Id: I1dcc26ea8664d014b6c50f0d636c13bf21e26ff2
Reviewed-on: https://chromium-review.googlesource.com/c/1328945
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57396}
'simulator_run' is now used exclusively in test/*/*.status.
Change-Id: I501b7ffd19e2476f4c803ed3d25cba69d67039fa
Reviewed-on: https://chromium-review.googlesource.com/c/1318610
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57300}
This is a reland of a12203c64b
Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
>
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
>
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}
Change-Id: I25fcd95bfc1aee1d21da390359423e5dfed112a4
Reviewed-on: https://chromium-review.googlesource.com/c/1286952
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Ben Smith <binji@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56791}
This reverts commit a12203c64b.
Reason for revert: Breaks isolate_tests
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux%20-%20builder/36777
Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
>
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
>
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}
TBR=binji@chromium.org,machenbach@chromium.org,yangguo@chromium.org,ahaas@chromium.org,clemensh@chromium.org,mathias@chromium.org
Change-Id: I2edd0ca94cb5990322571879c81671fa835f3ecd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/1286526
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56746}
These changes were necessary to run with the new style of jsapi tests
introduced in https://github.com/WebAssembly/spec/pull/883.
Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
Reviewed-on: https://chromium-review.googlesource.com/c/1277724
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56745}
The regexp for finding resources to be pushed to Android devices was
too lax. On empty strings it tried to check for more resources on a
directory and hung.
The last test262 roll contains tests with empty imports that started
hanging in this way.
TBR=neis@chromium.org
NOTRY=true
Bug: v8:7834
Change-Id: Ie58f1b18bdd99b7b40c1fb39b25e2f481932e0f3
Reviewed-on: https://chromium-review.googlesource.com/c/1270579
Reviewed-by: Georg Neis <neis@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56465}
The cctest and unittests suites call the test executable to list the
tests, which requires pushing the executables to the device on Android.
NOTRY=true
TBR=sergiyb@chromium.org
Bug: chromium:866862
Change-Id: I318dff7af2b2de3b8642ec4b6ba30b602da808bb
Reviewed-on: https://chromium-review.googlesource.com/1213202
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55723}
Path names in import/export statements are relative to the file they are in.
This fixes the logic and unblocks using the messages test suite on Android,
which has cases importing files from mjsunit, which import more files from
there.
Bug: chromium:866862
Change-Id: I8d2ff645f69b67fbdaf4a622d06308e55298b0ce
Reviewed-on: https://chromium-review.googlesource.com/1206570
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55643}
This shares logic for finding additional resources in JS source code.
Previously the logic was implemented for mjsunit, now it will be used
across all d8-based test cases.
This'll enable adding those test suites for Android testing.
Bug: chromium:866862
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I7c89ba141483aaf692a03c0e168edb61bbb7b010
Reviewed-on: https://chromium-review.googlesource.com/1193873
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55503}
This is a reland of 4c0943424c
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
Bug: chromium:866862
Change-Id: Icf7e04c75d4abeab7254d10ba21240e46b0022ae
Reviewed-on: https://chromium-review.googlesource.com/1170643
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55059}
This reverts commit 4c0943424c.
Reason for revert: Unfortunately this broke all perf builders.
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org
Change-Id: If80129810586b709dab762c9b5724888e15daec2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:866862
Reviewed-on: https://chromium-review.googlesource.com/1170962
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55058}
This adds a new command abstraction for running commands on Android
using dockered devices on swarming.
The new abstraction handles pushing all required files to the device.
The logic used for pushing and running is reused from the perf runner.
This adds only the mjsunit test suite. Others will be handled in
follow up CLs. The suite logic is enhanced with auto-detection of files
to be pushed to devices, for e.g. load or import statements.
Some test cases need an extra resource section for specifying required
files.
Remaining failing tests are marked in the status files for later
triage.
Bug: chromium:866862
Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
Reviewed-on: https://chromium-review.googlesource.com/1150153
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55041}
The test driver compares command-line arguments against test names
it finds on disk. Using Python's "fnmatch" for this nicely handles
wildcards, but is relatively slow. For given test names that don't
contain any '*', we can use string equality testing, which is much
faster.
Example: the time to evaluate
tools/run-tests.py --arch x64 --mode release \
$(grep 'object-spread' -l -r test/test262/data/test/ | \
sed -E 's|\.js$||' | \
sed -E 's|^test/test262/data/test/|test262/|')
goes from "I gave up and killed the process after several minutes"
to a couple of seconds with this patch.
Change-Id: I9ec404b7516afd801fe6126347f6dff533d1977c
Reviewed-on: https://chromium-review.googlesource.com/1149196
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54704}
Before flipping the flag, some tests need to be adapted. This CL
prepares these tests, such that the flag flip CL really just flips a
flag.
R=titzer@chromium.org, hablich@chromium.org
Bug: v8:6600, chromium:787421
Change-Id: I8030df69cda5f3fb81354350a37f65c0d1c669bd
Reviewed-on: https://chromium-review.googlesource.com/1110363
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53959}
This is to speed up slow nosnap runs. Allocation verification is covered by
running mksnapshot on other builders.
Change-Id: I9c286f7dd4abac9cf1be45be4a483a7b36d09f8a
Reviewed-on: https://chromium-review.googlesource.com/1059113
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53234}
D8 enables the Wasm trap handler by default now, but we need to make sure the
older bounds check case still gets test coverage too, as bounds checks will
continue to be a supported configuration.
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I5b0bdded6929a9b3a8480e87d038398b8d2a0fd8
Reviewed-on: https://chromium-review.googlesource.com/1048835
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53078}
This makes testing with random gc stress use the status file entrees
for gc fuzzing as well, since many test cases not suitable for one, are
not suitable for the other.
This also skips two more tests that rely on assert(Un)Optimized, which
is unreliable with gc fuzzing.
TBR=sigurds@chromium.org
Change-Id: I33a617b251d5cf65cf6e486d07ec55cde050b8ab
Reviewed-on: https://chromium-review.googlesource.com/1021082
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52701}
Embedded builtins are now based off the v8_enable_embedded_builtins
gn flag instead, which conditionally defines V8_EMBEDDED_BUILTINS.
Bug: v8:6666
Change-Id: I44d40d30fce3a3ed9bbf973d46c4990ba3fade40
Reviewed-on: https://chromium-review.googlesource.com/964361
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52092}
The feature in question has been enabled by default for a while and we
no longer need to maintain a configuration without it enabled. Note that
this change only removes the mechanical pieces. Further cleanup enabled
by this will be done as follow-ups.
R=clemensh@chromium.org
BUG=v8:7549
Change-Id: I90e5bcddabe74a18a4d2a88132e8dc93317bcff4
Reviewed-on: https://chromium-review.googlesource.com/958424
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51883}
We now unconditionally both parse and compile StreamedSource on the background
thread.
BUG=v8:5203
Change-Id: I42d6fe9059bc1745da3a415d270f46cf1c08b306
Reviewed-on: https://chromium-review.googlesource.com/948854
Reviewed-by: Mythri Alle <mythria@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51758}
