Commit Graph

532 Commits

Author SHA1 Message Date
Vadim Gorbachev (bmsdave)
6cad46b4f3 Preparing v8 to use with python3 /tools/testrunner
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds

This CL was uploaded by git cl split.

Bug: v8:8594
Change-Id: I2a90aaecb270f03aed1c0fc92da1a0e2621b0eb2
Reviewed-on: https://chromium-review.googlesource.com/c/1470101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59678}
2019-02-19 09:23:11 +00:00
Tamer Tas
5b957fa169 [testrunner] explain how the testrunner progress percentage is calculated
Progress indicator calculates the percentage using the estimated amount of
tests.

When base tests produce more tests or when testrunner filters some tests, the
percentage terminates over 100% or under it.

This CL adds an informative message about how the percentage behaves.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8728
Change-Id: I91cafd2579ea1894ac347ff7483c307cd46c545d
Reviewed-on: https://chromium-review.googlesource.com/c/1477056
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59660}
2019-02-18 13:42:37 +00:00
Michael Achenbach
2324bbbd73 Make tmrts an infra OWNER
This also cleans up some OWNERS files.

NOTRY=true

Change-Id: Ic49ecee02bb3e339dc4c0de4ba69f00c36c076aa
Reviewed-on: https://chromium-review.googlesource.com/c/1475470
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59622}
2019-02-15 09:02:24 +00:00
Tamer Tas
baeb4e324d [testrunner] enable the progress indicator
Using test generators meant that we had to remove the progress indicator since
the total number of tests weren't known before-hand.

This CL implements a progress indicator using test number estimations.

cctest and unittests progress indicator is accurate, however estimating
means the progress will terminate over 100% in big test suites and sometimes
under 100%.

R=machenbach@chromium.org
CC=​sergiyb@chromium.org,yangguo@chromium.org

Bug: v8:8769
Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
Reviewed-on: https://chromium-review.googlesource.com/c/1460471
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59538}
2019-02-12 16:03:17 +00:00
Tamer Tas
ad5b6d7dcb [testrunner] enable variant sharding
Testrunner runs variants of a test sequentially without taking sharding into
account. A slow test with slow variants slows down the whole test run no matter
the sharding configuration.

This CL implements a test hashing algorithm and variant sharding for test
variants.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I15f8c547fa2f361fb6c53bf8d5df055d3df38d3e
Reviewed-on: https://chromium-review.googlesource.com/c/1458016
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59530}
2019-02-12 13:40:33 +00:00
Tamer Tas
5aee0e1710 [testrunner] show the variant type along with the test name
The test name alone isn't informative alone.

This CL prints the variant name during the progress reporting.

tested by running test262 with all variants

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8728
Change-Id: I3d7d1fc6eacef3712d87404c7276fd5523bb445a
Reviewed-on: https://chromium-review.googlesource.com/c/1458156
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59437}
2019-02-07 13:43:57 +00:00
Tamer Tas
8c30a2ccc0 [testrunner] migrate testsuites to lazy loading
V8 test suites return tests by loading them into memory up-front. Up-front
loading has memory and loading-time overhead for the testrunner.

This CL converts the test-suites to load tests during the test run in parallel.

After CL, most test suites start running immediately and all test suites start
running after 3 seconds on a modern SSD.

This CL eliminates the 200 MiB memory spikes that causes memory problems in mac
minis.

Overhead of running test262 before CL:
Line #    Mem usage     Increment   Line Contents
=================================================
   264  225.760 MiB 195.8.000 MiB         tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
Overhead of running test262 after CL:
Line #    Mem usage    Increment   Line Contents
================================================
   264   28.840 MiB    0.000 MiB         tests = self._load_testsuite_generators(args, options)
-------------------------------------------------

R=machenbach@chromium.org
CC=​​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174, v8:8552, v8:8728
Change-Id: Iab540b9410239b05dc80b4a5228db25d8b8fb3b9
Reviewed-on: https://chromium-review.googlesource.com/c/1454478
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59408}
2019-02-06 14:18:24 +00:00
Sergiy Belozorov
f71dd204bf [tools] Add -d/--device options to all test launchers
This allows to select device when multiple devices are connected.

R=machenbach@chromium.org, tmrts@chromium.org

Bug: chromium:893593
Change-Id: I3dfd8b98251f613f5c93d29acd5035b236731ea6
Reviewed-on: https://chromium-review.googlesource.com/c/1452441
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59388}
2019-02-06 09:10:09 +00:00
Tamer Tas
df630e67fd Reland "Reland "[test] refactor testsuite configuration""
This is a reland of 81eec150f6

Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}

Bug: v8:8174, v8:8790
Change-Id: I38ab9d37bca76057441a970f26e2102e4387a857
Reviewed-on: https://chromium-review.googlesource.com/c/1454724
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59387}
2019-02-06 09:02:09 +00:00
Tamer Tas
ea4412ad33 Revert "Reland "[test] refactor testsuite configuration""
This reverts commit 81eec150f6.

Reason for revert: windows mozilla test failures

Original change's description:
> Reland "[test] refactor testsuite configuration"
> 
> This is a reland of 7f92ad0ab6
> 
> Original change's description:
> > [test] refactor testsuite configuration
> > 
> > Every testsuite configuration consist of at least 30% code duplication.
> > 
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> > 
> > This CL removes all the duplication by refactoring the common functionality.
> > 
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> > 
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> > 
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
> 
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org

Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
2019-02-05 15:20:46 +00:00
Tamer Tas
81eec150f6 Reland "[test] refactor testsuite configuration"
This is a reland of 7f92ad0ab6

Original change's description:
> [test] refactor testsuite configuration
> 
> Every testsuite configuration consist of at least 30% code duplication.
> 
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
> 
> This CL removes all the duplication by refactoring the common functionality.
> 
> This CL contains structural changes without any logical changes % small bug
> fixes.
> 
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
2019-02-05 14:53:05 +00:00
Tamer Tas
97068800fe Revert "[test] refactor testsuite configuration"
This reverts commit 7f92ad0ab6.

Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148

Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=​machenbach@chromium.org
> CC=​yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org

Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
2019-02-05 12:06:18 +00:00
Tamer Tas
7f92ad0ab6 [test] refactor testsuite configuration
Every testsuite configuration consist of at least 30% code duplication.

The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.

This CL removes all the duplication by refactoring the common functionality.

This CL contains structural changes without any logical changes % small bug
fixes.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
2019-02-05 11:37:51 +00:00
Michael Achenbach
b7b43b7dc0 [test] Run jitless on all bots
Bug: v8:8778
Change-Id: I384ad4387743d534a79ebad03130e8a688cc4631
Reviewed-on: https://chromium-review.googlesource.com/c/1449691
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59332}
2019-02-04 12:04:08 +00:00
Michael Achenbach
090687700a Revert "Reland "[tools] Push files using high-level device.PushChangedFiles method""
This reverts commit 6e03d7ee42.

Reason for revert: This breaks the Android bot:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2933
It's quite hidden due to https://crbug.com/v8/8731 but all shards
time out. It looks like to to this change, testing takes
dramatically longer, maybe the pushing takes now much longer than
before. If we want decide for this, the builder needs to get
many more shards.

Original change's description:
> Reland "[tools] Push files using high-level device.PushChangedFiles method"
> 
> This is a reland of d045f66682
> 
> Original change's description:
> > [tools] Push files using high-level device.PushChangedFiles method
> >
> > R=machenbach@chromium.org
> >
> > No-Try: true
> > Bug: chromium:893593
> > Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> > Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#58407}
> 
> Bug: chromium:893593
> Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
> Reviewed-on: https://chromium-review.googlesource.com/c/1443055
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59221}

TBR=machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:893593
Change-Id: Ifea307b5de8f39b660966fc6bef54601df91d841
Reviewed-on: https://chromium-review.googlesource.com/c/1450119
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59305}
2019-02-01 20:42:11 +00:00
Tamer Tas
af2f203db9 Reland "[testrunner] catch unforeseen exceptions"
This is a reland of 101fd4ddb7

Original change's description:
> [testrunner] catch unforeseen exceptions
>
> Testrunner doesn't catch exceptions except TestRunnererror, KeyboardInterrupt.
>
> Unforeseen exceptions should fail the CI step.
>
> This CL returns an error exit code for uncaught exceptions.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8731
> Change-Id: I7fb20dad4a3eea29f1dfa87ef91d45381ee08692
> Reviewed-on: https://chromium-review.googlesource.com/c/1434034
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59278}

Bug: v8:8731
No-Try: true
No-Tree-Checks: true
Change-Id: Ic580b334697ecebec923e76ff4a3e09fdf24dc8c
Reviewed-on: https://chromium-review.googlesource.com/c/1449632
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59283}
2019-02-01 11:52:30 +00:00
Tamer Tas
3198d0ad8f Revert "[testrunner] catch unforeseen exceptions"
This reverts commit 101fd4ddb7.

Reason for revert: Uncaught exception in the recipe API https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux/29928

Original change's description:
> [testrunner] catch unforeseen exceptions
> 
> Testrunner doesn't catch exceptions except TestRunnererror, KeyboardInterrupt.
> 
> Unforeseen exceptions should fail the CI step.
> 
> This CL returns an error exit code for uncaught exceptions.
> 
> R=​machenbach@chromium.org
> CC=​yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8731
> Change-Id: I7fb20dad4a3eea29f1dfa87ef91d45381ee08692
> Reviewed-on: https://chromium-review.googlesource.com/c/1434034
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59278}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: Ia2b661c1a71d1799693383bc942a9a1d47668b24
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8731
Reviewed-on: https://chromium-review.googlesource.com/c/1449692
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59282}
2019-02-01 11:37:09 +00:00
Michael Achenbach
f4a4104fe5 [tools] Clean up presubmit and run all unittests
NOTRY=true

Bug: v8:8763
Change-Id: I9b4e2edbb5f1eeeaa88d35efaa25a4eb0c35c95c
Reviewed-on: https://chromium-review.googlesource.com/c/1449612
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59279}
2019-02-01 10:27:29 +00:00
Tamer Tas
101fd4ddb7 [testrunner] catch unforeseen exceptions
Testrunner doesn't catch exceptions except TestRunnererror, KeyboardInterrupt.

Unforeseen exceptions should fail the CI step.

This CL returns an error exit code for uncaught exceptions.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8731
Change-Id: I7fb20dad4a3eea29f1dfa87ef91d45381ee08692
Reviewed-on: https://chromium-review.googlesource.com/c/1434034
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59278}
2019-02-01 10:25:39 +00:00
Tamer Tas
0b23bb1e4b [fuzzer] fix names
R=machenbach@chromium.org

No-Try: true
Change-Id: I3ec0adc9be2ea09f63c12bf71803865f224fba09
Reviewed-on: https://chromium-review.googlesource.com/c/1449611
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59275}
2019-02-01 09:58:19 +00:00
Tamer Tas
9d41bbc7c6 [fuzzer] fix missing argument
R=machenbach@chromium.org

No-Try: true
Change-Id: Iadb464e55fd30cc01ad6f250efd01588610407d0
Reviewed-on: https://chromium-review.googlesource.com/c/1449534
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59274}
2019-02-01 09:44:58 +00:00
Tamer Tas
cc70b55046 [fuzzer] fix the missing parenthesis
R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

No-Try: True
Bug: v8:8174
Change-Id: Ia77a42f3dce741f661f3d7aee3af2237e53ae0f0
Reviewed-on: https://chromium-review.googlesource.com/c/1447713
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59233}
2019-01-31 10:20:01 +00:00
Tamer Tas
581eb3cbc7 [testrunner] prevent from presubmit masking testrunner end-to-end tests
R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

No-Try: true
Bug: v8:8174,v8:8728,v8:8763
Change-Id: I27faa43e775b4a834a7c2af3ec1407afb71bb06e
Reviewed-on: https://chromium-review.googlesource.com/c/1447714
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59231}
2019-01-31 10:02:03 +00:00
Tamer Tas
3f83accb09 [testrunner] load tests concurrently into test execution processor
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.

This CL eliminates that overhead by utilizing generators and threading.

LoadingProc now loads test after receiving the results of the loaded tests.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174,v8:8731
Change-Id: Ifee79c3e213da568f092de0f1623016174e9410c
Reviewed-on: https://chromium-review.googlesource.com/c/1439240
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59223}
2019-01-31 08:19:06 +00:00
Sergiy Belozorov
6e03d7ee42 Reland "[tools] Push files using high-level device.PushChangedFiles method"
This is a reland of d045f66682

Original change's description:
> [tools] Push files using high-level device.PushChangedFiles method
>
> R=machenbach@chromium.org
>
> No-Try: true
> Bug: chromium:893593
> Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58407}

Bug: chromium:893593
Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
Reviewed-on: https://chromium-review.googlesource.com/c/1443055
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59221}
2019-01-31 02:40:51 +00:00
Tamer Tas
d852ae6f2b [testrunner] remove recursive result calls in chain processors
Procs return the result by increasing recursion through result_for.

This CL eliminates that mechanism from the Processor interface and uses boolen
return values for sending tests to signal success or the failure to load the
test into the execution queue.

R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174,v8:8731
Change-Id: I073a86ca84bcf88da11132b90013d4c8455bc61e
Reviewed-on: https://chromium-review.googlesource.com/c/1439239
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59201}
2019-01-30 13:04:07 +00:00
Michael Achenbach
4d2a3cc9a5 [test] Add interpreted-regexp testing variant
This adds the new runtime flag as a variant and runs it on a subset of
builders corresponding to the "extra" testing set.

Currently failing tests are skipped in the new variant.

After https://crrev.com/c/1433777 this costs only little additional
resources.

Bug: v8:8678
Change-Id: Ibd0e38872814d11252e55a7c6a58d313aa84ebe3
Reviewed-on: https://chromium-review.googlesource.com/c/1433774
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59080}
2019-01-24 19:11:22 +00:00
Farazmand
ac1e76e96c dot progress indicator does not currently display failed results
Change-Id: Ib17dd4a258809bb9ca521f724ab95d7eade6d15c
Reviewed-on: https://chromium-review.googlesource.com/c/1427359
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Milad Farazmand <miladfar@ca.ibm.com>
Cr-Commit-Position: refs/heads/master@{#59066}
2019-01-24 14:30:27 +00:00
Michael Achenbach
1c0b17b03f Revert "[testrunner] load tests concurrently into test execution processor"
This reverts commit 25457c60a7.

Reason for revert: https://crbug.com/v8/8731

Original change's description:
> [testrunner] load tests concurrently into test execution processor
> 
> loading every test up-front into the processing queue costs about 224MB for a
> x64 testsuite run.
> 
> This CL eliminates that overhead by utilizing generators and threading.
> 
> LoadingProc now loads test after receiving the results of the loaded tests.
> 
> R=​machenbach@chromium.org
> CC=​​yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8174
> Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
> Reviewed-on: https://chromium-review.googlesource.com/c/1420678
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59056}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: I1e074a031dced367a32a93827b9e863b0331340f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174
Reviewed-on: https://chromium-review.googlesource.com/c/1433792
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59065}
2019-01-24 13:56:04 +00:00
Michael Achenbach
588373412c [test] Reduce testing 'trusted' variant to arm builders
Previously, trusted (or no-mitigations) has been tested on a subset of builders
from all platforms. This reduces it to arm-sim and native Android devices.

Change-Id: I90066686e6a92db4a944025538e01a117f324421
Reviewed-on: https://chromium-review.googlesource.com/c/1433777
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59062}
2019-01-24 12:42:41 +00:00
Tamer Tas
25457c60a7 [testrunner] load tests concurrently into test execution processor
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.

This CL eliminates that overhead by utilizing generators and threading.

LoadingProc now loads test after receiving the results of the loaded tests.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
Reviewed-on: https://chromium-review.googlesource.com/c/1420678
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59056}
2019-01-24 11:19:12 +00:00
Tamer Tas
bd019bdb72 [testrunner] delete ancient junit compatible format support
Testrunner has ancient support for JUnit compatible XML output.

This CL removes this old feature.

R=mstarzinger@chromium.org,jgruber@chromium.org,jkummerow@chromium.org
CC=​machenbach@chromium.org

Bug: v8:8728
Change-Id: I7e1beb011dbaec3aa1a27398a5c52abdd778eaf0
Reviewed-on: https://chromium-review.googlesource.com/c/1430065
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59045}
2019-01-23 16:30:55 +00:00
Tamer Tas
a8b8a4408b [testrunner] remove the up-front test counter
Moving to a lazy test loading approach makes counting the total number of tests
non-trivial.

For CI runs, we output the total number of tests after the run.
For users, progress indicator signals the status of the run.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I0731964515aac60a3629acee6c7243433a2b4e04
Reviewed-on: https://chromium-review.googlesource.com/c/1420677
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58944}
2019-01-21 07:34:53 +00:00
Tamer Tas
d6c915189f [testrunner] use selection sort generator instead of timsort for test cases
V8 testrunner is loading every test it has to run into memory greedily in order
to sort by slowness of the test case. The memory and CPU overhead for loading
the test-suites are non-trivial.

This CL restructures it by changing the sorting method.

R=machenbach@chromium.org
CC=​​sergiyb@chromium.org,yangguo@chromium.org

Bug: v8:8174
Change-Id: I08331182147b92cf4ac54823eea0e2b472f51e84
Reviewed-on: https://chromium-review.googlesource.com/c/1406684
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58821}
2019-01-15 11:27:19 +00:00
Tamer Tas
234f27b5ae [testrunner] fix leaky abstraction in TestSuite loading process
TestSuite has a static method LoadTestSuite that should properly configure the
TestSuite instance (i.e. loaded status files and tests), however the method
leaves some configuration logic to the caller.

The leaky abstraction causes the caller to do a bunch of loading operations (see
the removed methods in base_runner.py).

This CL isolates the TestSuite loading logic to the static method only.

This is a refactoring only change without any intended logical changes.

R=machenbach@chromium.org
CC=​​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I105059c9c9e050f03bb584174e2bd7ceeae2b228
Reviewed-on: https://chromium-review.googlesource.com/c/1396417
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58744}
2019-01-11 12:58:08 +00:00
Michael Achenbach
1efb130a8e [test] Remove leftovers of the interrupt-budget fuzzer
TBR=sergiyb@chromium.org

Bug: v8:8174, v8:8457
Change-Id: Ie87eddcc6986e1c724040b11b036b502e399dd05
Reviewed-on: https://chromium-review.googlesource.com/c/1404437
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58735}
2019-01-11 11:21:01 +00:00
Tamer Tas
4eec444957 [testrunner] add tests for loading TestSuite from disk
- Removed the old test.
- Created a fake test suite and added a test for loading it with a TestConfig

R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: Ib7587ceec9e31ecd4cb8f45c3158e73c79a9bc5b
Reviewed-on: https://chromium-review.googlesource.com/c/1396082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58593}
2019-01-07 15:02:04 +00:00
Tamer Tas
6df42e36e5 [testrunner] remove dead code
This code is very old and is not referenced anywhere else.

Verifying that the code isn't called anywhere else:
- https://cs.chromium.org/search/?q=FilterTestCasesByArgs&type=cs
- https://cs.chromium.org/search/?q=FilterTestCasesByStatus&type=cs

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

No-Try: true
Change-Id: I18b0309430d86649046e64e863ca252951786061
Reviewed-on: https://chromium-review.googlesource.com/c/1394553
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58539}
2019-01-04 09:40:53 +00:00
Maya Lekova
2ebbfc308d Revert "[tools] Pass large_output to avoid re-running tests that generate lots of output"
This reverts commit b3c85c9e26.

Reason for revert: Causing Check failures on Android - https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2391

Original change's description:
> [tools] Pass large_output to avoid re-running tests that generate lots of output
> 
> R=​machenbach@chromium.org
> 
> Bug: chromium:893593
> Change-Id: Ic3a3612fc557b1f35b4632cfab553b4492ec6222
> Reviewed-on: https://chromium-review.googlesource.com/c/1382734
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58399}

TBR=machenbach@chromium.org,sergiyb@chromium.org

Change-Id: I8ac42d1b53ce5840e80071625466135999a3c42c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1387502
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58431}
2018-12-21 12:26:16 +00:00
Maya Lekova
6e5fb0041a Revert "[tools] Push files using high-level device.PushChangedFiles method"
This reverts commit d045f66682.

Reason for revert: Probably causing a timeout on Android bots - https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2393

Original change's description:
> [tools] Push files using high-level device.PushChangedFiles method
> 
> R=​machenbach@chromium.org
> 
> No-Try: true
> Bug: chromium:893593
> Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58407}

TBR=machenbach@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org

Change-Id: I23b0a5ef9fb6c1cca56e4b4bfe44fdfe4b80a390
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1387489
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58422}
2018-12-21 10:13:55 +00:00
Sergiy Belozorov
d045f66682 [tools] Push files using high-level device.PushChangedFiles method
R=machenbach@chromium.org

No-Try: true
Bug: chromium:893593
Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
Reviewed-on: https://chromium-review.googlesource.com/c/1382468
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58407}
2018-12-20 15:05:43 +00:00
Sergiy Belozorov
b3c85c9e26 [tools] Pass large_output to avoid re-running tests that generate lots of output
R=machenbach@chromium.org

Bug: chromium:893593
Change-Id: Ic3a3612fc557b1f35b4632cfab553b4492ec6222
Reviewed-on: https://chromium-review.googlesource.com/c/1382734
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58399}
2018-12-20 13:20:29 +00:00
Igor Sheludko
a4993b73c5 [testrunner] Add mode detection for v8_enable_pointer_compression
Tbr: machenbach@chromium.org
Bug: v8:8562
Change-Id: If2242c6271d400e7c4c031e10871b06438c2636d
Reviewed-on: https://chromium-review.googlesource.com/c/1384088
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58373}
2018-12-19 16:59:46 +00:00
Sergiy Belozorov
9fa98c139f [tools] Use device_utils.HealthyDevices to handle USB bus issues
TBR=machenbach@chromium.org

No-Try: true
Bug: chromium:893593
Change-Id: Ia1512e55426dc64f260b074dd964cc7dcc822dcf
Reviewed-on: https://chromium-review.googlesource.com/c/1382457
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58330}
2018-12-18 15:52:13 +00:00
Tamer Tas
2a7a827f68 [tools] skipping the debug mode in status files imply skipping in dcheck_always_on as well
R=machenbach@chromium.org
CC=yangguo@chromium.org

Bug: v8:8491
Change-Id: I8379825c194e588da582a3000201eea75b59140a
Reviewed-on: https://chromium-review.googlesource.com/c/1371826
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58243}
2018-12-14 14:26:49 +00:00
Ross McIlroy
d37d767b92 [Test] Add --stress-flush-bytecode to gc-stress tester.
Also disables --stress-flush-bytecode on some mjsunit tests which fail
when bytecode flushing is stressed due to test invariants.

Bug=v8:8395

Change-Id: If627910214b3c266e7776340ba182829148e8289
Reviewed-on: https://chromium-review.googlesource.com/c/1372071
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58230}
2018-12-13 18:37:56 +00:00
Sergiy Belozorov
996cf21954 [tools] Generate additional variant for FAIL_PHASE_ONLY tests
The additional variant does not use wrapper disabling phase tests and negated
outcome processor. This allows to ensure that tests marked FAIL_PHASE_ONLY, do
actually fail without it.

R=machenbach@chromium.org

Bug: v8:8467
Change-Id: I66e07bd7107520872cc013bf0f33fdc6664baf56
Reviewed-on: https://chromium-review.googlesource.com/c/1361164
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58211}
2018-12-13 11:28:57 +00:00
Sergiy Byelozyorov
948cf2f642 [tools] Add FAIL_PHASE_ONLY status file modifier for test262 tests
R=machenbach@chromium.org, mathias@chromium.org

Bug: v8:8467
Change-Id: I7369fa993d0762cf9c3c6f55361d074d067a0eb5
Reviewed-on: https://chromium-review.googlesource.com/c/1338347
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57734}
2018-11-22 12:43:05 +00:00
Michael Achenbach
ab18455532 [test] Add feature to keep running skipped tests.
This commit adds a --run-skipped flag to the test runner that will
bypass the 'SKIP' status.

Bug: v8:8485
Change-Id: Iac012bdaf2de6b0f8e44ed3a65bc9330709527bb
Reviewed-on: https://chromium-review.googlesource.com/c/1346490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Sigurd Schneider <sigurds@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57710}
2018-11-22 08:12:21 +00:00
Sergiy Byelozyorov
281a320fa2 Revert "[tools] Re-land: Add retries when trying to discover the device"
This reverts commit f401cd4b2c.

Reason for revert: did not help

Original change's description:
> [tools] Re-land: Add retries when trying to discover the device
> 
> This is an experimental change that may help mitigate the issue.
> 
> TBR=machenbach@chromium.org
> 
> No-Try: true
> No-Tree-Checks: true
> Bug: chromium:893593
> Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
> Reviewed-on: https://chromium-review.googlesource.com/c/1339401
> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57558}

TBR=machenbach@chromium.org,sergiyb@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:893593
Change-Id: I30c529a627d1e6fa52099939c5c209110e9d0eb9
Reviewed-on: https://chromium-review.googlesource.com/c/1342931
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57650}
2018-11-20 15:30:58 +00:00