Commit Graph

359 Commits

Author SHA1 Message Date
Dan Elphick
6d20778e4a [tests] Add --stress-lazy-source-positions to stress variant
Bug: v8:8510
Change-Id: Ie2b1cbbc136fecf3a81716b129e51857bd8db157
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1762294
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Dan Elphick <delphick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63310}
2019-08-21 12:42:05 +00:00
Michael Achenbach
cdfadf4a99 Reland "[test] Print hanging tests on linux on test-runner termination"
This is a reland of 3fc9663159

The private method on_event in timeout proc is now renamed to be truly
private.

Original change's description:
> [test] Print hanging tests on linux on test-runner termination
>
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
>
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}

Bug: v8:8292
Change-Id: Ibad1172666d6f4d2c07884a54edfe9d6499b57fe
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735318
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63072}
2019-08-05 11:50:18 +00:00
Michael Achenbach
cfdbab36d9 Revert "[test] Print hanging tests on linux on test-runner termination"
This reverts commit 3fc9663159.

Reason for revert: _on_event clashes with a the same function in timeout proc.

Original change's description:
> [test] Print hanging tests on linux on test-runner termination
> 
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
> 
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: Ic29013e92c77a9905e06ee13a2449e033285eb36
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8292
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735317
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63066}
2019-08-05 10:52:53 +00:00
Michael Achenbach
3fc9663159 [test] Print hanging tests on linux on test-runner termination
This will print the list of processes still running before and after
joining workers during termination. This will help debugging hanging
tests during flake-bisect or with num-fuzzer, which both terminate
on total timeout and currently still sometimes hang without printing
processes.

Bug: v8:8292
Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63065}
2019-08-05 10:21:48 +00:00
Michael Achenbach
cfe51d6bf1 [test] Add --assert-types flag to linux x64 release builders
Bug: v8:9476
Change-Id: Id3660d40fe219ee0816ac8550ee8affd1be4b95b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1698399
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62682}
2019-07-12 14:30:49 +00:00
Tamer Tas
6b52a54b9a [testrunner] generalize test loader to accept multiple extensions
testrunner assumes that each test suite has a single extension for base tests.
".mjs" extension can be used for ECMAScript modules in addition to the standard
extension ".js" we use for the base tests.

This CL generalizes the {TestLoader} to accept multiple extensions for
a single test suite.

R=mathias@chromium.org
TBR=machenbach@chromium.org
CC=​gsathya@chromium.org

Bug: v8:9395
Change-Id: Ibc155f4963472fe9f989458cd839f3642ffbddea
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1675961
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Mathias Bynens <mathias@chromium.org>
Auto-Submit: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62356}
2019-06-25 12:12:11 +00:00
Clemens Hammacher
01821ab3b6 [test] Stress wasm code gc in "stress_background_compile"
The "stress_background_compile" variant runs on all our bots. We
combine it with testing wasm code GC (which kind of fits into
background compile stressing) to get more coverage for that. Both
features are orthogonal, so we can test both at the same time without
loosing any coverage.

R=machenbach@chromium.org
CC=rmcilroy@chromium.org

Bug: v8:8217
Change-Id: Ib17decd4869978ff98e302694fa73d70ceec120e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1588472
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61151}
2019-05-02 09:56:05 +00:00
Michael Achenbach
d533da3a36 [test] Be more verbose when killing hanging tests
TBR=tmrts@chromium.org

Bug: v8:9145,v8:9098
Change-Id: Ib2170819f6d9aa00174fefb6bef4b0a27c11acfc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1578503
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60956}
2019-04-23 16:48:23 +00:00
Michael Achenbach
1ea09f0b4f [test] Ignore optimization state in tests on number fuzzer
This enable test suites to check which test driver framework is used.
When using number fuzzer on mjsunit, we add a JS file that
switches off the optimization-state assertions.

Checking intrinsic state is not feasible on the number fuzzer and in
the past, we needed to skip tests on demand, which is a maintenance
burden. The main function of the fuzzer, to check for dcheck errors and
tsan issues, is retained.

Bug: v8:9127
Change-Id: I699b85d5f7c9aaed337a2130d9eddc160c059d7b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1565892
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60816}
2019-04-12 12:50:55 +00:00
Clemens Hammacher
31b49aeeb0 [test] Test more wasm variants
Since wasm optimizations are independent of JS optimizations, we can
just combine the "no_liftoff" variant with the existing "stress"
variant (which has the similar "--always-opt" option for JS), and add
a "Liftoff only" variant as part of "nooptimization".

This gives more coverage to find bugs like https://crrev.com/c/1543354
more easy.

R=mstarzinger@chromium.org, machenbach@chromium.org

Change-Id: I81bb22074c59dcb650a05252da43a4170cd467ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1559740
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60714}
2019-04-09 13:41:52 +00:00
Clemens Hammacher
ac8c78e01d [cleanup] Replace underscore by minus in d8 flags
Reading --verify_predictable makes me sad, whereas --verify-predictable
makes me happy. This CL introduces more happiness.

R=machenbach@chromium.org

Bug: v8:8834
Change-Id: Id51a75f32e6d5a2f87aed81e058a8b6dff189758
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1550399
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60599}
2019-04-03 12:32:08 +00:00
Milad Farazmand
cb7607b875 [testrunner] preventing a StopIteration exception when running tests
The next built-in function throws a "StopIteration" when iterator is exhausted
which will crash and stop the test runner. Asking it to return a "None" instead will
fix the issue.

Change-Id: I3ca4ff8f28f359798c9faf6f1ffbbaa6bb95f156
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1524719
Commit-Queue: Milad Farazmand <miladfar@ca.ibm.com>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60294}
2019-03-18 13:16:31 +00:00
Michael Achenbach
a59ca7358c [test] Make test runner use shell
Bug: v8:8889
Change-Id: If0a8e71dca2ae5660481ba78bf79eb311d60c9e4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1495557
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60079}
2019-03-07 08:16:27 +00:00
Vadim Gorbachev (bmsdave)
6cad46b4f3 Preparing v8 to use with python3 /tools/testrunner
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds

This CL was uploaded by git cl split.

Bug: v8:8594
Change-Id: I2a90aaecb270f03aed1c0fc92da1a0e2621b0eb2
Reviewed-on: https://chromium-review.googlesource.com/c/1470101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59678}
2019-02-19 09:23:11 +00:00
Tamer Tas
baeb4e324d [testrunner] enable the progress indicator
Using test generators meant that we had to remove the progress indicator since
the total number of tests weren't known before-hand.

This CL implements a progress indicator using test number estimations.

cctest and unittests progress indicator is accurate, however estimating
means the progress will terminate over 100% in big test suites and sometimes
under 100%.

R=machenbach@chromium.org
CC=​sergiyb@chromium.org,yangguo@chromium.org

Bug: v8:8769
Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
Reviewed-on: https://chromium-review.googlesource.com/c/1460471
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59538}
2019-02-12 16:03:17 +00:00
Tamer Tas
8c30a2ccc0 [testrunner] migrate testsuites to lazy loading
V8 test suites return tests by loading them into memory up-front. Up-front
loading has memory and loading-time overhead for the testrunner.

This CL converts the test-suites to load tests during the test run in parallel.

After CL, most test suites start running immediately and all test suites start
running after 3 seconds on a modern SSD.

This CL eliminates the 200 MiB memory spikes that causes memory problems in mac
minis.

Overhead of running test262 before CL:
Line #    Mem usage     Increment   Line Contents
=================================================
   264  225.760 MiB 195.8.000 MiB         tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
Overhead of running test262 after CL:
Line #    Mem usage    Increment   Line Contents
================================================
   264   28.840 MiB    0.000 MiB         tests = self._load_testsuite_generators(args, options)
-------------------------------------------------

R=machenbach@chromium.org
CC=​​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174, v8:8552, v8:8728
Change-Id: Iab540b9410239b05dc80b4a5228db25d8b8fb3b9
Reviewed-on: https://chromium-review.googlesource.com/c/1454478
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59408}
2019-02-06 14:18:24 +00:00
Sergiy Belozorov
f71dd204bf [tools] Add -d/--device options to all test launchers
This allows to select device when multiple devices are connected.

R=machenbach@chromium.org, tmrts@chromium.org

Bug: chromium:893593
Change-Id: I3dfd8b98251f613f5c93d29acd5035b236731ea6
Reviewed-on: https://chromium-review.googlesource.com/c/1452441
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59388}
2019-02-06 09:10:09 +00:00
Tamer Tas
df630e67fd Reland "Reland "[test] refactor testsuite configuration""
This is a reland of 81eec150f6

Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}

Bug: v8:8174, v8:8790
Change-Id: I38ab9d37bca76057441a970f26e2102e4387a857
Reviewed-on: https://chromium-review.googlesource.com/c/1454724
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59387}
2019-02-06 09:02:09 +00:00
Tamer Tas
ea4412ad33 Revert "Reland "[test] refactor testsuite configuration""
This reverts commit 81eec150f6.

Reason for revert: windows mozilla test failures

Original change's description:
> Reland "[test] refactor testsuite configuration"
> 
> This is a reland of 7f92ad0ab6
> 
> Original change's description:
> > [test] refactor testsuite configuration
> > 
> > Every testsuite configuration consist of at least 30% code duplication.
> > 
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> > 
> > This CL removes all the duplication by refactoring the common functionality.
> > 
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> > 
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> > 
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
> 
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org

Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
2019-02-05 15:20:46 +00:00
Tamer Tas
81eec150f6 Reland "[test] refactor testsuite configuration"
This is a reland of 7f92ad0ab6

Original change's description:
> [test] refactor testsuite configuration
> 
> Every testsuite configuration consist of at least 30% code duplication.
> 
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
> 
> This CL removes all the duplication by refactoring the common functionality.
> 
> This CL contains structural changes without any logical changes % small bug
> fixes.
> 
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
2019-02-05 14:53:05 +00:00
Tamer Tas
97068800fe Revert "[test] refactor testsuite configuration"
This reverts commit 7f92ad0ab6.

Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148

Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=​machenbach@chromium.org
> CC=​yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org

Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
2019-02-05 12:06:18 +00:00
Tamer Tas
7f92ad0ab6 [test] refactor testsuite configuration
Every testsuite configuration consist of at least 30% code duplication.

The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.

This CL removes all the duplication by refactoring the common functionality.

This CL contains structural changes without any logical changes % small bug
fixes.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
2019-02-05 11:37:51 +00:00
Michael Achenbach
b7b43b7dc0 [test] Run jitless on all bots
Bug: v8:8778
Change-Id: I384ad4387743d534a79ebad03130e8a688cc4631
Reviewed-on: https://chromium-review.googlesource.com/c/1449691
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59332}
2019-02-04 12:04:08 +00:00
Michael Achenbach
090687700a Revert "Reland "[tools] Push files using high-level device.PushChangedFiles method""
This reverts commit 6e03d7ee42.

Reason for revert: This breaks the Android bot:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2933
It's quite hidden due to https://crbug.com/v8/8731 but all shards
time out. It looks like to to this change, testing takes
dramatically longer, maybe the pushing takes now much longer than
before. If we want decide for this, the builder needs to get
many more shards.

Original change's description:
> Reland "[tools] Push files using high-level device.PushChangedFiles method"
> 
> This is a reland of d045f66682
> 
> Original change's description:
> > [tools] Push files using high-level device.PushChangedFiles method
> >
> > R=machenbach@chromium.org
> >
> > No-Try: true
> > Bug: chromium:893593
> > Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> > Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#58407}
> 
> Bug: chromium:893593
> Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
> Reviewed-on: https://chromium-review.googlesource.com/c/1443055
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59221}

TBR=machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:893593
Change-Id: Ifea307b5de8f39b660966fc6bef54601df91d841
Reviewed-on: https://chromium-review.googlesource.com/c/1450119
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59305}
2019-02-01 20:42:11 +00:00
Sergiy Belozorov
6e03d7ee42 Reland "[tools] Push files using high-level device.PushChangedFiles method"
This is a reland of d045f66682

Original change's description:
> [tools] Push files using high-level device.PushChangedFiles method
>
> R=machenbach@chromium.org
>
> No-Try: true
> Bug: chromium:893593
> Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58407}

Bug: chromium:893593
Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
Reviewed-on: https://chromium-review.googlesource.com/c/1443055
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59221}
2019-01-31 02:40:51 +00:00
Michael Achenbach
4d2a3cc9a5 [test] Add interpreted-regexp testing variant
This adds the new runtime flag as a variant and runs it on a subset of
builders corresponding to the "extra" testing set.

Currently failing tests are skipped in the new variant.

After https://crrev.com/c/1433777 this costs only little additional
resources.

Bug: v8:8678
Change-Id: Ibd0e38872814d11252e55a7c6a58d313aa84ebe3
Reviewed-on: https://chromium-review.googlesource.com/c/1433774
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59080}
2019-01-24 19:11:22 +00:00
Tamer Tas
bd019bdb72 [testrunner] delete ancient junit compatible format support
Testrunner has ancient support for JUnit compatible XML output.

This CL removes this old feature.

R=mstarzinger@chromium.org,jgruber@chromium.org,jkummerow@chromium.org
CC=​machenbach@chromium.org

Bug: v8:8728
Change-Id: I7e1beb011dbaec3aa1a27398a5c52abdd778eaf0
Reviewed-on: https://chromium-review.googlesource.com/c/1430065
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59045}
2019-01-23 16:30:55 +00:00
Tamer Tas
d6c915189f [testrunner] use selection sort generator instead of timsort for test cases
V8 testrunner is loading every test it has to run into memory greedily in order
to sort by slowness of the test case. The memory and CPU overhead for loading
the test-suites are non-trivial.

This CL restructures it by changing the sorting method.

R=machenbach@chromium.org
CC=​​sergiyb@chromium.org,yangguo@chromium.org

Bug: v8:8174
Change-Id: I08331182147b92cf4ac54823eea0e2b472f51e84
Reviewed-on: https://chromium-review.googlesource.com/c/1406684
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58821}
2019-01-15 11:27:19 +00:00
Tamer Tas
234f27b5ae [testrunner] fix leaky abstraction in TestSuite loading process
TestSuite has a static method LoadTestSuite that should properly configure the
TestSuite instance (i.e. loaded status files and tests), however the method
leaves some configuration logic to the caller.

The leaky abstraction causes the caller to do a bunch of loading operations (see
the removed methods in base_runner.py).

This CL isolates the TestSuite loading logic to the static method only.

This is a refactoring only change without any intended logical changes.

R=machenbach@chromium.org
CC=​​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I105059c9c9e050f03bb584174e2bd7ceeae2b228
Reviewed-on: https://chromium-review.googlesource.com/c/1396417
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58744}
2019-01-11 12:58:08 +00:00
Michael Achenbach
1efb130a8e [test] Remove leftovers of the interrupt-budget fuzzer
TBR=sergiyb@chromium.org

Bug: v8:8174, v8:8457
Change-Id: Ie87eddcc6986e1c724040b11b036b502e399dd05
Reviewed-on: https://chromium-review.googlesource.com/c/1404437
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58735}
2019-01-11 11:21:01 +00:00
Tamer Tas
4eec444957 [testrunner] add tests for loading TestSuite from disk
- Removed the old test.
- Created a fake test suite and added a test for loading it with a TestConfig

R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: Ib7587ceec9e31ecd4cb8f45c3158e73c79a9bc5b
Reviewed-on: https://chromium-review.googlesource.com/c/1396082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58593}
2019-01-07 15:02:04 +00:00
Tamer Tas
6df42e36e5 [testrunner] remove dead code
This code is very old and is not referenced anywhere else.

Verifying that the code isn't called anywhere else:
- https://cs.chromium.org/search/?q=FilterTestCasesByArgs&type=cs
- https://cs.chromium.org/search/?q=FilterTestCasesByStatus&type=cs

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

No-Try: true
Change-Id: I18b0309430d86649046e64e863ca252951786061
Reviewed-on: https://chromium-review.googlesource.com/c/1394553
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58539}
2019-01-04 09:40:53 +00:00
Maya Lekova
2ebbfc308d Revert "[tools] Pass large_output to avoid re-running tests that generate lots of output"
This reverts commit b3c85c9e26.

Reason for revert: Causing Check failures on Android - https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2391

Original change's description:
> [tools] Pass large_output to avoid re-running tests that generate lots of output
> 
> R=​machenbach@chromium.org
> 
> Bug: chromium:893593
> Change-Id: Ic3a3612fc557b1f35b4632cfab553b4492ec6222
> Reviewed-on: https://chromium-review.googlesource.com/c/1382734
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58399}

TBR=machenbach@chromium.org,sergiyb@chromium.org

Change-Id: I8ac42d1b53ce5840e80071625466135999a3c42c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1387502
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58431}
2018-12-21 12:26:16 +00:00
Maya Lekova
6e5fb0041a Revert "[tools] Push files using high-level device.PushChangedFiles method"
This reverts commit d045f66682.

Reason for revert: Probably causing a timeout on Android bots - https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2393

Original change's description:
> [tools] Push files using high-level device.PushChangedFiles method
> 
> R=​machenbach@chromium.org
> 
> No-Try: true
> Bug: chromium:893593
> Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58407}

TBR=machenbach@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org

Change-Id: I23b0a5ef9fb6c1cca56e4b4bfe44fdfe4b80a390
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1387489
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58422}
2018-12-21 10:13:55 +00:00
Sergiy Belozorov
d045f66682 [tools] Push files using high-level device.PushChangedFiles method
R=machenbach@chromium.org

No-Try: true
Bug: chromium:893593
Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
Reviewed-on: https://chromium-review.googlesource.com/c/1382468
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58407}
2018-12-20 15:05:43 +00:00
Sergiy Belozorov
b3c85c9e26 [tools] Pass large_output to avoid re-running tests that generate lots of output
R=machenbach@chromium.org

Bug: chromium:893593
Change-Id: Ic3a3612fc557b1f35b4632cfab553b4492ec6222
Reviewed-on: https://chromium-review.googlesource.com/c/1382734
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58399}
2018-12-20 13:20:29 +00:00
Sergiy Belozorov
9fa98c139f [tools] Use device_utils.HealthyDevices to handle USB bus issues
TBR=machenbach@chromium.org

No-Try: true
Bug: chromium:893593
Change-Id: Ia1512e55426dc64f260b074dd964cc7dcc822dcf
Reviewed-on: https://chromium-review.googlesource.com/c/1382457
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58330}
2018-12-18 15:52:13 +00:00
Sergiy Byelozyorov
948cf2f642 [tools] Add FAIL_PHASE_ONLY status file modifier for test262 tests
R=machenbach@chromium.org, mathias@chromium.org

Bug: v8:8467
Change-Id: I7369fa993d0762cf9c3c6f55361d074d067a0eb5
Reviewed-on: https://chromium-review.googlesource.com/c/1338347
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57734}
2018-11-22 12:43:05 +00:00
Sergiy Byelozyorov
281a320fa2 Revert "[tools] Re-land: Add retries when trying to discover the device"
This reverts commit f401cd4b2c.

Reason for revert: did not help

Original change's description:
> [tools] Re-land: Add retries when trying to discover the device
> 
> This is an experimental change that may help mitigate the issue.
> 
> TBR=machenbach@chromium.org
> 
> No-Try: true
> No-Tree-Checks: true
> Bug: chromium:893593
> Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
> Reviewed-on: https://chromium-review.googlesource.com/c/1339401
> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57558}

TBR=machenbach@chromium.org,sergiyb@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:893593
Change-Id: I30c529a627d1e6fa52099939c5c209110e9d0eb9
Reviewed-on: https://chromium-review.googlesource.com/c/1342931
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57650}
2018-11-20 15:30:58 +00:00
Sergiy Byelozyorov
f401cd4b2c [tools] Re-land: Add retries when trying to discover the device
This is an experimental change that may help mitigate the issue.

TBR=machenbach@chromium.org

No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
Reviewed-on: https://chromium-review.googlesource.com/c/1339401
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57558}
2018-11-16 07:03:37 +00:00
Sergiy Byelozyorov
02f1529b1b Revert "[tools] Add retries when trying to discover the device"
This reverts commit f1741bdd2b.

Reason for revert: forgot to import 'time' module

Original change's description:
> [tools] Add retries when trying to discover the device
> 
> This is an experimental change that may help mitigate the issue.
> 
> TBR=machenbach@chromium.org
> 
> No-Try: true
> No-Tree-Checks: true
> Bug: chromium:893593
> Change-Id: Ideb74a83b9937dbe917e8c7c93305d9824b48a93
> Reviewed-on: https://chromium-review.googlesource.com/c/1339419
> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57556}

TBR=machenbach@chromium.org,sergiyb@chromium.org

Change-Id: I5ae591e099f630fdb4cd63d18bfb2f1bf347f929
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1339519
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57557}
2018-11-16 06:54:07 +00:00
Sergiy Byelozyorov
f1741bdd2b [tools] Add retries when trying to discover the device
This is an experimental change that may help mitigate the issue.

TBR=machenbach@chromium.org

No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Ideb74a83b9937dbe917e8c7c93305d9824b48a93
Reviewed-on: https://chromium-review.googlesource.com/c/1339419
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57556}
2018-11-16 06:36:32 +00:00
Sergiy Byelozyorov
691dbd2fbe [tools] Stop passing ensure_logs_on_timeout since it defaults to True now
TBR=machenbach@chromium.org

Bug: chromium:899721
Change-Id: Ifdb4864e47a56cb0fe2565e44e79f3def496b138
Reviewed-on: https://chromium-review.googlesource.com/c/1317821
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57253}
2018-11-05 20:57:55 +00:00
Michael Achenbach
ba48855133 Revert "[test] Be more chatty when killing hanging tests"
This reverts commit fad1c1c94f.

Reason for revert: Debug printing not needed anymore.

Original change's description:
> [test] Be more chatty when killing hanging tests
> 
> Bug: v8:8292
> Change-Id: I74fd304692e90adfb694b73ecf5e7858e3b66607
> Reviewed-on: https://chromium-review.googlesource.com/c/1275814
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Maya Lekova <mslekova@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56562}

TBR=machenbach@chromium.org,sergiyb@chromium.org,mslekova@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: v8:8292
Change-Id: Ieaba51f866b93d49fe168b6370bd126752993afb
Reviewed-on: https://chromium-review.googlesource.com/c/1288632
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56774}
2018-10-18 12:04:41 +00:00
Sergiy Byelozyorov
6fdf6e228e [tools] Use a flag to ensure that we keep logs on timeout
R=machenbach@chromium.org

Bug: chromium:891314
Change-Id: Iaab09ac543b0928f6a81275f529df73949693c83
Reviewed-on: https://chromium-review.googlesource.com/c/1276425
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56630}
2018-10-15 12:02:31 +00:00
Michael Achenbach
fad1c1c94f [test] Be more chatty when killing hanging tests
Bug: v8:8292
Change-Id: I74fd304692e90adfb694b73ecf5e7858e3b66607
Reviewed-on: https://chromium-review.googlesource.com/c/1275814
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56562}
2018-10-11 12:17:42 +00:00
Sergiy Byelozyorov
1e25524841 [tools] Implement --exit-after-n-failures functionality via a flag
R=machenbach@chromium.org

Bug: v8:8239
Change-Id: I2451230f92fa6ad66ce6446f97efaa7fafd04e12
Reviewed-on: https://chromium-review.googlesource.com/1251524
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56337}
2018-10-02 14:18:48 +00:00
Sergiy Byelozyorov
1a236fb899 [tools] Print command output on timeout
R=machenbach@chromium.org

Bug: chromium:872257
Change-Id: I6eb1f70e0d44c76e18eb4f6f936c7f2cf0380444
Reviewed-on: https://chromium-review.googlesource.com/1217942
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55834}
2018-09-12 15:38:39 +00:00
Michael Achenbach
e3a0b362db [test] Fix test-listing command on Android
The cctest and unittests suites call the test executable to list the
tests, which requires pushing the executables to the device on Android.

NOTRY=true
TBR=sergiyb@chromium.org

Bug: chromium:866862
Change-Id: I318dff7af2b2de3b8642ec4b6ba30b602da808bb
Reviewed-on: https://chromium-review.googlesource.com/1213202
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55723}
2018-09-07 12:16:58 +00:00
Sergiy Byelozyorov
0e0157a42c [tools] Upload adb logcat to logdog when running tests
R=machenbach@chromium.org

No-Try: true
Bug: chromium:872257
Change-Id: I4de1a9dfccbb0d123c1c42c31a1f697e628623fa
Reviewed-on: https://chromium-review.googlesource.com/1171224
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55067}
2018-08-11 09:08:14 +00:00