For standard test failures, the variant name is enough to deduce the
full command line of the test. This is used to create the flako
command line on the infra side.
Test failures from numfuzz have additional variant_flags, calculated
by the fuzzer, which don't match a variant name. Exposing those in
the test results will enable printing a proper flako command line
on infra side for numfuzz cases.
Bug: v8:8971
Change-Id: Ie47d42a0b34037da458b474f2a9ab38f1a5d238a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1554689
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60656}
Reading --verify_predictable makes me sad, whereas --verify-predictable
makes me happy. This CL introduces more happiness.
R=machenbach@chromium.org
Bug: v8:8834
Change-Id: Id51a75f32e6d5a2f87aed81e058a8b6dff189758
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1550399
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60599}
The next built-in function throws a "StopIteration" when iterator is exhausted
which will crash and stop the test runner. Asking it to return a "None" instead will
fix the issue.
Change-Id: I3ca4ff8f28f359798c9faf6f1ffbbaa6bb95f156
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1524719
Commit-Queue: Milad Farazmand <miladfar@ca.ibm.com>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60294}
This makes the test runner and numfuzz share the same exit code behavior on
errors. This is needed as they also share the same infrastructure logic
to collect swarming tasks.
Bug: chromium:937228
Change-Id: I155b37c7b10dd22959a4dcf30bbd0321c452236b
Reviewed-on: https://chromium-review.googlesource.com/c/1495987
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59976}
After python3 migration, the new print usage started causing leftover character
issues.
This CL fixes the print usage.
R=clemensh@chromium.org,neis@chromium.org
CC=machenbach@chromium.org
Bug: v8:8918
Change-Id: Ibee06677c3bae3e1141579693aa16a539309a566
Reviewed-on: https://chromium-review.googlesource.com/c/1495558
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59961}
This makes the output a lot smaller and thus makes it easier to see the
relevant part of the command.
R=machenbach@chromium.org
Change-Id: I62ac7218be4a02f0270a2d88a2f69b6ced45a041
Reviewed-on: https://chromium-review.googlesource.com/c/1491597
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59931}
WebAssembly's specification tests use a timeout annotation.
This change allows the shared testrunner to use it to calculate
the testcase timeout.
Currently, the allowed timeout values are: long. Other values will
emit a warning.
Change-Id: Id7f453f5fd49854c8f53ff86ef2ec58aa0ae8748
Reviewed-on: https://chromium-review.googlesource.com/c/1480376
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sven Sauleau <ssauleau@igalia.com>
Cr-Commit-Position: refs/heads/master@{#59919}
Also enable test runner to differentiate between clang and gcc.
Bug: v8:8919
Change-Id: Icdcae0aba3644a1b1b9ddc6c037eabde27d717f7
Reviewed-on: https://chromium-review.googlesource.com/c/1491634
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59906}
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
This CL was uploaded by git cl split.
Bug: v8:8594
Change-Id: I2a90aaecb270f03aed1c0fc92da1a0e2621b0eb2
Reviewed-on: https://chromium-review.googlesource.com/c/1470101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59678}
Progress indicator calculates the percentage using the estimated amount of
tests.
When base tests produce more tests or when testrunner filters some tests, the
percentage terminates over 100% or under it.
This CL adds an informative message about how the percentage behaves.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8728
Change-Id: I91cafd2579ea1894ac347ff7483c307cd46c545d
Reviewed-on: https://chromium-review.googlesource.com/c/1477056
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59660}
This also cleans up some OWNERS files.
NOTRY=true
Change-Id: Ic49ecee02bb3e339dc4c0de4ba69f00c36c076aa
Reviewed-on: https://chromium-review.googlesource.com/c/1475470
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59622}
Using test generators meant that we had to remove the progress indicator since
the total number of tests weren't known before-hand.
This CL implements a progress indicator using test number estimations.
cctest and unittests progress indicator is accurate, however estimating
means the progress will terminate over 100% in big test suites and sometimes
under 100%.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8769
Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
Reviewed-on: https://chromium-review.googlesource.com/c/1460471
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59538}
Testrunner runs variants of a test sequentially without taking sharding into
account. A slow test with slow variants slows down the whole test run no matter
the sharding configuration.
This CL implements a test hashing algorithm and variant sharding for test
variants.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I15f8c547fa2f361fb6c53bf8d5df055d3df38d3e
Reviewed-on: https://chromium-review.googlesource.com/c/1458016
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59530}
The test name alone isn't informative alone.
This CL prints the variant name during the progress reporting.
tested by running test262 with all variants
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8728
Change-Id: I3d7d1fc6eacef3712d87404c7276fd5523bb445a
Reviewed-on: https://chromium-review.googlesource.com/c/1458156
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59437}
V8 test suites return tests by loading them into memory up-front. Up-front
loading has memory and loading-time overhead for the testrunner.
This CL converts the test-suites to load tests during the test run in parallel.
After CL, most test suites start running immediately and all test suites start
running after 3 seconds on a modern SSD.
This CL eliminates the 200 MiB memory spikes that causes memory problems in mac
minis.
Overhead of running test262 before CL:
Line # Mem usage Increment Line Contents
=================================================
264 225.760 MiB 195.8.000 MiB tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
Overhead of running test262 after CL:
Line # Mem usage Increment Line Contents
================================================
264 28.840 MiB 0.000 MiB tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174, v8:8552, v8:8728
Change-Id: Iab540b9410239b05dc80b4a5228db25d8b8fb3b9
Reviewed-on: https://chromium-review.googlesource.com/c/1454478
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59408}
This is a reland of 81eec150f6
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
Bug: v8:8174, v8:8790
Change-Id: I38ab9d37bca76057441a970f26e2102e4387a857
Reviewed-on: https://chromium-review.googlesource.com/c/1454724
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59387}
This reverts commit 81eec150f6.
Reason for revert: windows mozilla test failures
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org
Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
This is a reland of 7f92ad0ab6
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
This reverts commit 7f92ad0ab6.
Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org
Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
Every testsuite configuration consist of at least 30% code duplication.
The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.
This CL removes all the duplication by refactoring the common functionality.
This CL contains structural changes without any logical changes % small bug
fixes.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
This reverts commit 6e03d7ee42.
Reason for revert: This breaks the Android bot:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2933
It's quite hidden due to https://crbug.com/v8/8731 but all shards
time out. It looks like to to this change, testing takes
dramatically longer, maybe the pushing takes now much longer than
before. If we want decide for this, the builder needs to get
many more shards.
Original change's description:
> Reland "[tools] Push files using high-level device.PushChangedFiles method"
>
> This is a reland of d045f66682
>
> Original change's description:
> > [tools] Push files using high-level device.PushChangedFiles method
> >
> > R=machenbach@chromium.org
> >
> > No-Try: true
> > Bug: chromium:893593
> > Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> > Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#58407}
>
> Bug: chromium:893593
> Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
> Reviewed-on: https://chromium-review.googlesource.com/c/1443055
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59221}
TBR=machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:893593
Change-Id: Ifea307b5de8f39b660966fc6bef54601df91d841
Reviewed-on: https://chromium-review.googlesource.com/c/1450119
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59305}
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.
This CL eliminates that overhead by utilizing generators and threading.
LoadingProc now loads test after receiving the results of the loaded tests.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174,v8:8731
Change-Id: Ifee79c3e213da568f092de0f1623016174e9410c
Reviewed-on: https://chromium-review.googlesource.com/c/1439240
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59223}
Procs return the result by increasing recursion through result_for.
This CL eliminates that mechanism from the Processor interface and uses boolen
return values for sending tests to signal success or the failure to load the
test into the execution queue.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174,v8:8731
Change-Id: I073a86ca84bcf88da11132b90013d4c8455bc61e
Reviewed-on: https://chromium-review.googlesource.com/c/1439239
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59201}
This adds the new runtime flag as a variant and runs it on a subset of
builders corresponding to the "extra" testing set.
Currently failing tests are skipped in the new variant.
After https://crrev.com/c/1433777 this costs only little additional
resources.
Bug: v8:8678
Change-Id: Ibd0e38872814d11252e55a7c6a58d313aa84ebe3
Reviewed-on: https://chromium-review.googlesource.com/c/1433774
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59080}
This reverts commit 25457c60a7.
Reason for revert: https://crbug.com/v8/8731
Original change's description:
> [testrunner] load tests concurrently into test execution processor
>
> loading every test up-front into the processing queue costs about 224MB for a
> x64 testsuite run.
>
> This CL eliminates that overhead by utilizing generators and threading.
>
> LoadingProc now loads test after receiving the results of the loaded tests.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174
> Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
> Reviewed-on: https://chromium-review.googlesource.com/c/1420678
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59056}
TBR=machenbach@chromium.org,tmrts@chromium.org
Change-Id: I1e074a031dced367a32a93827b9e863b0331340f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174
Reviewed-on: https://chromium-review.googlesource.com/c/1433792
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59065}
Previously, trusted (or no-mitigations) has been tested on a subset of builders
from all platforms. This reduces it to arm-sim and native Android devices.
Change-Id: I90066686e6a92db4a944025538e01a117f324421
Reviewed-on: https://chromium-review.googlesource.com/c/1433777
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59062}
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.
This CL eliminates that overhead by utilizing generators and threading.
LoadingProc now loads test after receiving the results of the loaded tests.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
Reviewed-on: https://chromium-review.googlesource.com/c/1420678
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59056}
Testrunner has ancient support for JUnit compatible XML output.
This CL removes this old feature.
R=mstarzinger@chromium.org,jgruber@chromium.org,jkummerow@chromium.org
CC=machenbach@chromium.org
Bug: v8:8728
Change-Id: I7e1beb011dbaec3aa1a27398a5c52abdd778eaf0
Reviewed-on: https://chromium-review.googlesource.com/c/1430065
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59045}
Moving to a lazy test loading approach makes counting the total number of tests
non-trivial.
For CI runs, we output the total number of tests after the run.
For users, progress indicator signals the status of the run.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I0731964515aac60a3629acee6c7243433a2b4e04
Reviewed-on: https://chromium-review.googlesource.com/c/1420677
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58944}
V8 testrunner is loading every test it has to run into memory greedily in order
to sort by slowness of the test case. The memory and CPU overhead for loading
the test-suites are non-trivial.
This CL restructures it by changing the sorting method.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8174
Change-Id: I08331182147b92cf4ac54823eea0e2b472f51e84
Reviewed-on: https://chromium-review.googlesource.com/c/1406684
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58821}
TestSuite has a static method LoadTestSuite that should properly configure the
TestSuite instance (i.e. loaded status files and tests), however the method
leaves some configuration logic to the caller.
The leaky abstraction causes the caller to do a bunch of loading operations (see
the removed methods in base_runner.py).
This CL isolates the TestSuite loading logic to the static method only.
This is a refactoring only change without any intended logical changes.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I105059c9c9e050f03bb584174e2bd7ceeae2b228
Reviewed-on: https://chromium-review.googlesource.com/c/1396417
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58744}
- Removed the old test.
- Created a fake test suite and added a test for loading it with a TestConfig
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: Ib7587ceec9e31ecd4cb8f45c3158e73c79a9bc5b
Reviewed-on: https://chromium-review.googlesource.com/c/1396082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58593}
