Commit Graph

773 Commits

Author SHA1 Message Date
Tamer Tas
ea4412ad33 Revert "Reland "[test] refactor testsuite configuration""
This reverts commit 81eec150f6.

Reason for revert: windows mozilla test failures

Original change's description:
> Reland "[test] refactor testsuite configuration"
> 
> This is a reland of 7f92ad0ab6
> 
> Original change's description:
> > [test] refactor testsuite configuration
> > 
> > Every testsuite configuration consist of at least 30% code duplication.
> > 
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> > 
> > This CL removes all the duplication by refactoring the common functionality.
> > 
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> > 
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> > 
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
> 
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org

Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
2019-02-05 15:20:46 +00:00
Tamer Tas
81eec150f6 Reland "[test] refactor testsuite configuration"
This is a reland of 7f92ad0ab6

Original change's description:
> [test] refactor testsuite configuration
> 
> Every testsuite configuration consist of at least 30% code duplication.
> 
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
> 
> This CL removes all the duplication by refactoring the common functionality.
> 
> This CL contains structural changes without any logical changes % small bug
> fixes.
> 
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
2019-02-05 14:53:05 +00:00
Tamer Tas
97068800fe Revert "[test] refactor testsuite configuration"
This reverts commit 7f92ad0ab6.

Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148

Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=​machenbach@chromium.org
> CC=​yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org

Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
2019-02-05 12:06:18 +00:00
Tamer Tas
7f92ad0ab6 [test] refactor testsuite configuration
Every testsuite configuration consist of at least 30% code duplication.

The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.

This CL removes all the duplication by refactoring the common functionality.

This CL contains structural changes without any logical changes % small bug
fixes.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
2019-02-05 11:37:51 +00:00
Michael Achenbach
b7b43b7dc0 [test] Run jitless on all bots
Bug: v8:8778
Change-Id: I384ad4387743d534a79ebad03130e8a688cc4631
Reviewed-on: https://chromium-review.googlesource.com/c/1449691
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59332}
2019-02-04 12:04:08 +00:00
Michael Achenbach
090687700a Revert "Reland "[tools] Push files using high-level device.PushChangedFiles method""
This reverts commit 6e03d7ee42.

Reason for revert: This breaks the Android bot:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2933
It's quite hidden due to https://crbug.com/v8/8731 but all shards
time out. It looks like to to this change, testing takes
dramatically longer, maybe the pushing takes now much longer than
before. If we want decide for this, the builder needs to get
many more shards.

Original change's description:
> Reland "[tools] Push files using high-level device.PushChangedFiles method"
> 
> This is a reland of d045f66682
> 
> Original change's description:
> > [tools] Push files using high-level device.PushChangedFiles method
> >
> > R=machenbach@chromium.org
> >
> > No-Try: true
> > Bug: chromium:893593
> > Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> > Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#58407}
> 
> Bug: chromium:893593
> Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
> Reviewed-on: https://chromium-review.googlesource.com/c/1443055
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59221}

TBR=machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:893593
Change-Id: Ifea307b5de8f39b660966fc6bef54601df91d841
Reviewed-on: https://chromium-review.googlesource.com/c/1450119
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59305}
2019-02-01 20:42:11 +00:00
Tamer Tas
af2f203db9 Reland "[testrunner] catch unforeseen exceptions"
This is a reland of 101fd4ddb7

Original change's description:
> [testrunner] catch unforeseen exceptions
>
> Testrunner doesn't catch exceptions except TestRunnererror, KeyboardInterrupt.
>
> Unforeseen exceptions should fail the CI step.
>
> This CL returns an error exit code for uncaught exceptions.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8731
> Change-Id: I7fb20dad4a3eea29f1dfa87ef91d45381ee08692
> Reviewed-on: https://chromium-review.googlesource.com/c/1434034
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59278}

Bug: v8:8731
No-Try: true
No-Tree-Checks: true
Change-Id: Ic580b334697ecebec923e76ff4a3e09fdf24dc8c
Reviewed-on: https://chromium-review.googlesource.com/c/1449632
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59283}
2019-02-01 11:52:30 +00:00
Tamer Tas
3198d0ad8f Revert "[testrunner] catch unforeseen exceptions"
This reverts commit 101fd4ddb7.

Reason for revert: Uncaught exception in the recipe API https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux/29928

Original change's description:
> [testrunner] catch unforeseen exceptions
> 
> Testrunner doesn't catch exceptions except TestRunnererror, KeyboardInterrupt.
> 
> Unforeseen exceptions should fail the CI step.
> 
> This CL returns an error exit code for uncaught exceptions.
> 
> R=​machenbach@chromium.org
> CC=​yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8731
> Change-Id: I7fb20dad4a3eea29f1dfa87ef91d45381ee08692
> Reviewed-on: https://chromium-review.googlesource.com/c/1434034
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59278}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: Ia2b661c1a71d1799693383bc942a9a1d47668b24
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8731
Reviewed-on: https://chromium-review.googlesource.com/c/1449692
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59282}
2019-02-01 11:37:09 +00:00
Michael Achenbach
f4a4104fe5 [tools] Clean up presubmit and run all unittests
NOTRY=true

Bug: v8:8763
Change-Id: I9b4e2edbb5f1eeeaa88d35efaa25a4eb0c35c95c
Reviewed-on: https://chromium-review.googlesource.com/c/1449612
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59279}
2019-02-01 10:27:29 +00:00
Tamer Tas
101fd4ddb7 [testrunner] catch unforeseen exceptions
Testrunner doesn't catch exceptions except TestRunnererror, KeyboardInterrupt.

Unforeseen exceptions should fail the CI step.

This CL returns an error exit code for uncaught exceptions.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8731
Change-Id: I7fb20dad4a3eea29f1dfa87ef91d45381ee08692
Reviewed-on: https://chromium-review.googlesource.com/c/1434034
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59278}
2019-02-01 10:25:39 +00:00
Tamer Tas
0b23bb1e4b [fuzzer] fix names
R=machenbach@chromium.org

No-Try: true
Change-Id: I3ec0adc9be2ea09f63c12bf71803865f224fba09
Reviewed-on: https://chromium-review.googlesource.com/c/1449611
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59275}
2019-02-01 09:58:19 +00:00
Tamer Tas
9d41bbc7c6 [fuzzer] fix missing argument
R=machenbach@chromium.org

No-Try: true
Change-Id: Iadb464e55fd30cc01ad6f250efd01588610407d0
Reviewed-on: https://chromium-review.googlesource.com/c/1449534
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59274}
2019-02-01 09:44:58 +00:00
Tamer Tas
cc70b55046 [fuzzer] fix the missing parenthesis
R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

No-Try: True
Bug: v8:8174
Change-Id: Ia77a42f3dce741f661f3d7aee3af2237e53ae0f0
Reviewed-on: https://chromium-review.googlesource.com/c/1447713
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59233}
2019-01-31 10:20:01 +00:00
Tamer Tas
581eb3cbc7 [testrunner] prevent from presubmit masking testrunner end-to-end tests
R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

No-Try: true
Bug: v8:8174,v8:8728,v8:8763
Change-Id: I27faa43e775b4a834a7c2af3ec1407afb71bb06e
Reviewed-on: https://chromium-review.googlesource.com/c/1447714
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59231}
2019-01-31 10:02:03 +00:00
Tamer Tas
3f83accb09 [testrunner] load tests concurrently into test execution processor
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.

This CL eliminates that overhead by utilizing generators and threading.

LoadingProc now loads test after receiving the results of the loaded tests.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174,v8:8731
Change-Id: Ifee79c3e213da568f092de0f1623016174e9410c
Reviewed-on: https://chromium-review.googlesource.com/c/1439240
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59223}
2019-01-31 08:19:06 +00:00
Sergiy Belozorov
6e03d7ee42 Reland "[tools] Push files using high-level device.PushChangedFiles method"
This is a reland of d045f66682

Original change's description:
> [tools] Push files using high-level device.PushChangedFiles method
>
> R=machenbach@chromium.org
>
> No-Try: true
> Bug: chromium:893593
> Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58407}

Bug: chromium:893593
Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
Reviewed-on: https://chromium-review.googlesource.com/c/1443055
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59221}
2019-01-31 02:40:51 +00:00
Tamer Tas
d852ae6f2b [testrunner] remove recursive result calls in chain processors
Procs return the result by increasing recursion through result_for.

This CL eliminates that mechanism from the Processor interface and uses boolen
return values for sending tests to signal success or the failure to load the
test into the execution queue.

R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174,v8:8731
Change-Id: I073a86ca84bcf88da11132b90013d4c8455bc61e
Reviewed-on: https://chromium-review.googlesource.com/c/1439239
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59201}
2019-01-30 13:04:07 +00:00
Michael Achenbach
4d2a3cc9a5 [test] Add interpreted-regexp testing variant
This adds the new runtime flag as a variant and runs it on a subset of
builders corresponding to the "extra" testing set.

Currently failing tests are skipped in the new variant.

After https://crrev.com/c/1433777 this costs only little additional
resources.

Bug: v8:8678
Change-Id: Ibd0e38872814d11252e55a7c6a58d313aa84ebe3
Reviewed-on: https://chromium-review.googlesource.com/c/1433774
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59080}
2019-01-24 19:11:22 +00:00
Farazmand
ac1e76e96c dot progress indicator does not currently display failed results
Change-Id: Ib17dd4a258809bb9ca521f724ab95d7eade6d15c
Reviewed-on: https://chromium-review.googlesource.com/c/1427359
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Milad Farazmand <miladfar@ca.ibm.com>
Cr-Commit-Position: refs/heads/master@{#59066}
2019-01-24 14:30:27 +00:00
Michael Achenbach
1c0b17b03f Revert "[testrunner] load tests concurrently into test execution processor"
This reverts commit 25457c60a7.

Reason for revert: https://crbug.com/v8/8731

Original change's description:
> [testrunner] load tests concurrently into test execution processor
> 
> loading every test up-front into the processing queue costs about 224MB for a
> x64 testsuite run.
> 
> This CL eliminates that overhead by utilizing generators and threading.
> 
> LoadingProc now loads test after receiving the results of the loaded tests.
> 
> R=​machenbach@chromium.org
> CC=​​yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8174
> Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
> Reviewed-on: https://chromium-review.googlesource.com/c/1420678
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59056}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: I1e074a031dced367a32a93827b9e863b0331340f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174
Reviewed-on: https://chromium-review.googlesource.com/c/1433792
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59065}
2019-01-24 13:56:04 +00:00
Michael Achenbach
588373412c [test] Reduce testing 'trusted' variant to arm builders
Previously, trusted (or no-mitigations) has been tested on a subset of builders
from all platforms. This reduces it to arm-sim and native Android devices.

Change-Id: I90066686e6a92db4a944025538e01a117f324421
Reviewed-on: https://chromium-review.googlesource.com/c/1433777
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59062}
2019-01-24 12:42:41 +00:00
Tamer Tas
25457c60a7 [testrunner] load tests concurrently into test execution processor
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.

This CL eliminates that overhead by utilizing generators and threading.

LoadingProc now loads test after receiving the results of the loaded tests.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
Reviewed-on: https://chromium-review.googlesource.com/c/1420678
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59056}
2019-01-24 11:19:12 +00:00
Tamer Tas
bd019bdb72 [testrunner] delete ancient junit compatible format support
Testrunner has ancient support for JUnit compatible XML output.

This CL removes this old feature.

R=mstarzinger@chromium.org,jgruber@chromium.org,jkummerow@chromium.org
CC=​machenbach@chromium.org

Bug: v8:8728
Change-Id: I7e1beb011dbaec3aa1a27398a5c52abdd778eaf0
Reviewed-on: https://chromium-review.googlesource.com/c/1430065
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59045}
2019-01-23 16:30:55 +00:00
Tamer Tas
a8b8a4408b [testrunner] remove the up-front test counter
Moving to a lazy test loading approach makes counting the total number of tests
non-trivial.

For CI runs, we output the total number of tests after the run.
For users, progress indicator signals the status of the run.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I0731964515aac60a3629acee6c7243433a2b4e04
Reviewed-on: https://chromium-review.googlesource.com/c/1420677
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58944}
2019-01-21 07:34:53 +00:00
Tamer Tas
d6c915189f [testrunner] use selection sort generator instead of timsort for test cases
V8 testrunner is loading every test it has to run into memory greedily in order
to sort by slowness of the test case. The memory and CPU overhead for loading
the test-suites are non-trivial.

This CL restructures it by changing the sorting method.

R=machenbach@chromium.org
CC=​​sergiyb@chromium.org,yangguo@chromium.org

Bug: v8:8174
Change-Id: I08331182147b92cf4ac54823eea0e2b472f51e84
Reviewed-on: https://chromium-review.googlesource.com/c/1406684
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58821}
2019-01-15 11:27:19 +00:00
Tamer Tas
234f27b5ae [testrunner] fix leaky abstraction in TestSuite loading process
TestSuite has a static method LoadTestSuite that should properly configure the
TestSuite instance (i.e. loaded status files and tests), however the method
leaves some configuration logic to the caller.

The leaky abstraction causes the caller to do a bunch of loading operations (see
the removed methods in base_runner.py).

This CL isolates the TestSuite loading logic to the static method only.

This is a refactoring only change without any intended logical changes.

R=machenbach@chromium.org
CC=​​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I105059c9c9e050f03bb584174e2bd7ceeae2b228
Reviewed-on: https://chromium-review.googlesource.com/c/1396417
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58744}
2019-01-11 12:58:08 +00:00
Michael Achenbach
1efb130a8e [test] Remove leftovers of the interrupt-budget fuzzer
TBR=sergiyb@chromium.org

Bug: v8:8174, v8:8457
Change-Id: Ie87eddcc6986e1c724040b11b036b502e399dd05
Reviewed-on: https://chromium-review.googlesource.com/c/1404437
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58735}
2019-01-11 11:21:01 +00:00
Tamer Tas
4eec444957 [testrunner] add tests for loading TestSuite from disk
- Removed the old test.
- Created a fake test suite and added a test for loading it with a TestConfig

R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: Ib7587ceec9e31ecd4cb8f45c3158e73c79a9bc5b
Reviewed-on: https://chromium-review.googlesource.com/c/1396082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58593}
2019-01-07 15:02:04 +00:00
Tamer Tas
6df42e36e5 [testrunner] remove dead code
This code is very old and is not referenced anywhere else.

Verifying that the code isn't called anywhere else:
- https://cs.chromium.org/search/?q=FilterTestCasesByArgs&type=cs
- https://cs.chromium.org/search/?q=FilterTestCasesByStatus&type=cs

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

No-Try: true
Change-Id: I18b0309430d86649046e64e863ca252951786061
Reviewed-on: https://chromium-review.googlesource.com/c/1394553
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58539}
2019-01-04 09:40:53 +00:00
Maya Lekova
2ebbfc308d Revert "[tools] Pass large_output to avoid re-running tests that generate lots of output"
This reverts commit b3c85c9e26.

Reason for revert: Causing Check failures on Android - https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2391

Original change's description:
> [tools] Pass large_output to avoid re-running tests that generate lots of output
> 
> R=​machenbach@chromium.org
> 
> Bug: chromium:893593
> Change-Id: Ic3a3612fc557b1f35b4632cfab553b4492ec6222
> Reviewed-on: https://chromium-review.googlesource.com/c/1382734
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58399}

TBR=machenbach@chromium.org,sergiyb@chromium.org

Change-Id: I8ac42d1b53ce5840e80071625466135999a3c42c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1387502
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58431}
2018-12-21 12:26:16 +00:00
Maya Lekova
6e5fb0041a Revert "[tools] Push files using high-level device.PushChangedFiles method"
This reverts commit d045f66682.

Reason for revert: Probably causing a timeout on Android bots - https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2393

Original change's description:
> [tools] Push files using high-level device.PushChangedFiles method
> 
> R=​machenbach@chromium.org
> 
> No-Try: true
> Bug: chromium:893593
> Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58407}

TBR=machenbach@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org

Change-Id: I23b0a5ef9fb6c1cca56e4b4bfe44fdfe4b80a390
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1387489
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58422}
2018-12-21 10:13:55 +00:00
Sergiy Belozorov
d045f66682 [tools] Push files using high-level device.PushChangedFiles method
R=machenbach@chromium.org

No-Try: true
Bug: chromium:893593
Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
Reviewed-on: https://chromium-review.googlesource.com/c/1382468
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58407}
2018-12-20 15:05:43 +00:00
Sergiy Belozorov
b3c85c9e26 [tools] Pass large_output to avoid re-running tests that generate lots of output
R=machenbach@chromium.org

Bug: chromium:893593
Change-Id: Ic3a3612fc557b1f35b4632cfab553b4492ec6222
Reviewed-on: https://chromium-review.googlesource.com/c/1382734
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58399}
2018-12-20 13:20:29 +00:00
Igor Sheludko
a4993b73c5 [testrunner] Add mode detection for v8_enable_pointer_compression
Tbr: machenbach@chromium.org
Bug: v8:8562
Change-Id: If2242c6271d400e7c4c031e10871b06438c2636d
Reviewed-on: https://chromium-review.googlesource.com/c/1384088
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58373}
2018-12-19 16:59:46 +00:00
Sergiy Belozorov
9fa98c139f [tools] Use device_utils.HealthyDevices to handle USB bus issues
TBR=machenbach@chromium.org

No-Try: true
Bug: chromium:893593
Change-Id: Ia1512e55426dc64f260b074dd964cc7dcc822dcf
Reviewed-on: https://chromium-review.googlesource.com/c/1382457
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58330}
2018-12-18 15:52:13 +00:00
Tamer Tas
2a7a827f68 [tools] skipping the debug mode in status files imply skipping in dcheck_always_on as well
R=machenbach@chromium.org
CC=yangguo@chromium.org

Bug: v8:8491
Change-Id: I8379825c194e588da582a3000201eea75b59140a
Reviewed-on: https://chromium-review.googlesource.com/c/1371826
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58243}
2018-12-14 14:26:49 +00:00
Ross McIlroy
d37d767b92 [Test] Add --stress-flush-bytecode to gc-stress tester.
Also disables --stress-flush-bytecode on some mjsunit tests which fail
when bytecode flushing is stressed due to test invariants.

Bug=v8:8395

Change-Id: If627910214b3c266e7776340ba182829148e8289
Reviewed-on: https://chromium-review.googlesource.com/c/1372071
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58230}
2018-12-13 18:37:56 +00:00
Sergiy Belozorov
996cf21954 [tools] Generate additional variant for FAIL_PHASE_ONLY tests
The additional variant does not use wrapper disabling phase tests and negated
outcome processor. This allows to ensure that tests marked FAIL_PHASE_ONLY, do
actually fail without it.

R=machenbach@chromium.org

Bug: v8:8467
Change-Id: I66e07bd7107520872cc013bf0f33fdc6664baf56
Reviewed-on: https://chromium-review.googlesource.com/c/1361164
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58211}
2018-12-13 11:28:57 +00:00
Sergiy Byelozyorov
948cf2f642 [tools] Add FAIL_PHASE_ONLY status file modifier for test262 tests
R=machenbach@chromium.org, mathias@chromium.org

Bug: v8:8467
Change-Id: I7369fa993d0762cf9c3c6f55361d074d067a0eb5
Reviewed-on: https://chromium-review.googlesource.com/c/1338347
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57734}
2018-11-22 12:43:05 +00:00
Michael Achenbach
ab18455532 [test] Add feature to keep running skipped tests.
This commit adds a --run-skipped flag to the test runner that will
bypass the 'SKIP' status.

Bug: v8:8485
Change-Id: Iac012bdaf2de6b0f8e44ed3a65bc9330709527bb
Reviewed-on: https://chromium-review.googlesource.com/c/1346490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Sigurd Schneider <sigurds@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57710}
2018-11-22 08:12:21 +00:00
Sergiy Byelozyorov
281a320fa2 Revert "[tools] Re-land: Add retries when trying to discover the device"
This reverts commit f401cd4b2c.

Reason for revert: did not help

Original change's description:
> [tools] Re-land: Add retries when trying to discover the device
> 
> This is an experimental change that may help mitigate the issue.
> 
> TBR=machenbach@chromium.org
> 
> No-Try: true
> No-Tree-Checks: true
> Bug: chromium:893593
> Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
> Reviewed-on: https://chromium-review.googlesource.com/c/1339401
> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57558}

TBR=machenbach@chromium.org,sergiyb@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: chromium:893593
Change-Id: I30c529a627d1e6fa52099939c5c209110e9d0eb9
Reviewed-on: https://chromium-review.googlesource.com/c/1342931
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57650}
2018-11-20 15:30:58 +00:00
Sergiy Byelozyorov
f401cd4b2c [tools] Re-land: Add retries when trying to discover the device
This is an experimental change that may help mitigate the issue.

TBR=machenbach@chromium.org

No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
Reviewed-on: https://chromium-review.googlesource.com/c/1339401
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57558}
2018-11-16 07:03:37 +00:00
Sergiy Byelozyorov
02f1529b1b Revert "[tools] Add retries when trying to discover the device"
This reverts commit f1741bdd2b.

Reason for revert: forgot to import 'time' module

Original change's description:
> [tools] Add retries when trying to discover the device
> 
> This is an experimental change that may help mitigate the issue.
> 
> TBR=machenbach@chromium.org
> 
> No-Try: true
> No-Tree-Checks: true
> Bug: chromium:893593
> Change-Id: Ideb74a83b9937dbe917e8c7c93305d9824b48a93
> Reviewed-on: https://chromium-review.googlesource.com/c/1339419
> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57556}

TBR=machenbach@chromium.org,sergiyb@chromium.org

Change-Id: I5ae591e099f630fdb4cd63d18bfb2f1bf347f929
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:893593
Reviewed-on: https://chromium-review.googlesource.com/c/1339519
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57557}
2018-11-16 06:54:07 +00:00
Sergiy Byelozyorov
f1741bdd2b [tools] Add retries when trying to discover the device
This is an experimental change that may help mitigate the issue.

TBR=machenbach@chromium.org

No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Ideb74a83b9937dbe917e8c7c93305d9824b48a93
Reviewed-on: https://chromium-review.googlesource.com/c/1339419
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57556}
2018-11-16 06:36:32 +00:00
Michael Achenbach
1f147c50af [test] Skip flaky test on ODROIDs
Also add the ability to skip tests only when --optimize-for-size is passed.

TBR=sergiyb@chromium.org

Bug: v8:7819
Change-Id: I1dcc26ea8664d014b6c50f0d636c13bf21e26ff2
Reviewed-on: https://chromium-review.googlesource.com/c/1328945
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57396}
2018-11-09 14:35:19 +00:00
Adam Klein
764b4095d3 [testrunner] Remove last use of 'simulator' and its handling code
'simulator_run' is now used exclusively in test/*/*.status.

Change-Id: I501b7ffd19e2476f4c803ed3d25cba69d67039fa
Reviewed-on: https://chromium-review.googlesource.com/c/1318610
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57300}
2018-11-06 20:04:38 +00:00
Sergiy Byelozyorov
691dbd2fbe [tools] Stop passing ensure_logs_on_timeout since it defaults to True now
TBR=machenbach@chromium.org

Bug: chromium:899721
Change-Id: Ifdb4864e47a56cb0fe2565e44e79f3def496b138
Reviewed-on: https://chromium-review.googlesource.com/c/1317821
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57253}
2018-11-05 20:57:55 +00:00
Ross McIlroy
69138876cd [tools] Add lite_mode status file parameter.
BUG=v8:8293

Change-Id: Ic58ac512d5f32d26756e21ca929dda79f133a40e
Reviewed-on: https://chromium-review.googlesource.com/c/1290982
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56844}
2018-10-22 10:15:10 +00:00
Ben Smith
4347d0add1 Reland "[wasm] Add a new wasm-js testsuite to run js-api tests"
This is a reland of a12203c64b

Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
> 
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
> 
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}

Change-Id: I25fcd95bfc1aee1d21da390359423e5dfed112a4
Reviewed-on: https://chromium-review.googlesource.com/c/1286952
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Ben Smith <binji@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56791}
2018-10-18 20:18:55 +00:00
Michael Achenbach
ba48855133 Revert "[test] Be more chatty when killing hanging tests"
This reverts commit fad1c1c94f.

Reason for revert: Debug printing not needed anymore.

Original change's description:
> [test] Be more chatty when killing hanging tests
> 
> Bug: v8:8292
> Change-Id: I74fd304692e90adfb694b73ecf5e7858e3b66607
> Reviewed-on: https://chromium-review.googlesource.com/c/1275814
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Maya Lekova <mslekova@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56562}

TBR=machenbach@chromium.org,sergiyb@chromium.org,mslekova@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: v8:8292
Change-Id: Ieaba51f866b93d49fe168b6370bd126752993afb
Reviewed-on: https://chromium-review.googlesource.com/c/1288632
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56774}
2018-10-18 12:04:41 +00:00
Bill Budge
5c5dd02128 Revert "[wasm] Add a new wasm-js testsuite to run js-api tests"
This reverts commit a12203c64b.

Reason for revert: Breaks isolate_tests

https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux%20-%20builder/36777

Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
> 
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
> 
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}

TBR=binji@chromium.org,machenbach@chromium.org,yangguo@chromium.org,ahaas@chromium.org,clemensh@chromium.org,mathias@chromium.org

Change-Id: I2edd0ca94cb5990322571879c81671fa835f3ecd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/1286526
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56746}
2018-10-17 17:30:45 +00:00
Ben Smith
a12203c64b [wasm] Add a new wasm-js testsuite to run js-api tests
These changes were necessary to run with the new style of jsapi tests
introduced in https://github.com/WebAssembly/spec/pull/883.

Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
Reviewed-on: https://chromium-review.googlesource.com/c/1277724
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56745}
2018-10-17 17:07:53 +00:00
Michael Achenbach
c57508c477 Revert "[test] Temporary debug output for hanging tests"
This reverts commit c45bc57d21.

Reason for revert: Seen output once, that is enough.

Original change's description:
> [test] Temporary debug output for hanging tests
> 
> TBR=sigurds@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8292
> Change-Id: Ic03e2884ef645231807762f6b0210e69c5f9ac74
> Reviewed-on: https://chromium-review.googlesource.com/c/1282604
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56674}

TBR=machenbach@chromium.org,sigurds@chromium.org,sergiyb@chromium.org

Change-Id: Id5a2414682a9795998b86369368276e95c11394c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8292
Reviewed-on: https://chromium-review.googlesource.com/c/1283112
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56683}
2018-10-16 10:41:36 +00:00
Sigurd Schneider
a4e820df48 [tools] Add verify_csa statusfile param
Change-Id: Ibacdc7b41a96fc7c7d4ff44552fe6d86ec2ef1aa
Bug: v8:8312
Reviewed-on: https://chromium-review.googlesource.com/c/1282605
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56679}
2018-10-16 09:48:05 +00:00
Michael Achenbach
c45bc57d21 [test] Temporary debug output for hanging tests
TBR=sigurds@chromium.org,sergiyb@chromium.org

Bug: v8:8292
Change-Id: Ic03e2884ef645231807762f6b0210e69c5f9ac74
Reviewed-on: https://chromium-review.googlesource.com/c/1282604
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56674}
2018-10-16 09:00:33 +00:00
Sergiy Byelozyorov
6fdf6e228e [tools] Use a flag to ensure that we keep logs on timeout
R=machenbach@chromium.org

Bug: chromium:891314
Change-Id: Iaab09ac543b0928f6a81275f529df73949693c83
Reviewed-on: https://chromium-review.googlesource.com/c/1276425
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56630}
2018-10-15 12:02:31 +00:00
Michael Achenbach
fad1c1c94f [test] Be more chatty when killing hanging tests
Bug: v8:8292
Change-Id: I74fd304692e90adfb694b73ecf5e7858e3b66607
Reviewed-on: https://chromium-review.googlesource.com/c/1275814
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56562}
2018-10-11 12:17:42 +00:00
Sigurd Schneider
f805b43d9a [tools] Add embedded_builtins statusfile param
R=machenbach@chromium.org, sigurds@chromium.org

Bug: v8:6666
Change-Id: I9ac0fc393a9f9a66bebb72abd54a429e95aa078a
Reviewed-on: https://chromium-review.googlesource.com/c/1270840
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sigurd Schneider <sigurds@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56510}
2018-10-10 09:48:43 +00:00
Michael Achenbach
007c674467 [test] Add task-delay-stress fuzzer configs
TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:8278
Change-Id: I000c31366f2b3d894208665ddb6cef49216099a6
Reviewed-on: https://chromium-review.googlesource.com/c/1273097
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56508}
2018-10-10 09:12:14 +00:00
Michael Achenbach
eb7c56c8a7 [test] Fix resource regexp for Android
The regexp for finding resources to be pushed to Android devices was
too lax. On empty strings it tried to check for more resources on a
directory and hung.

The last test262 roll contains tests with empty imports that started
hanging in this way.

TBR=neis@chromium.org
NOTRY=true

Bug: v8:7834
Change-Id: Ie58f1b18bdd99b7b40c1fb39b25e2f481932e0f3
Reviewed-on: https://chromium-review.googlesource.com/c/1270579
Reviewed-by: Georg Neis <neis@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56465}
2018-10-09 08:35:50 +00:00
Sergiy Byelozyorov
1e25524841 [tools] Implement --exit-after-n-failures functionality via a flag
R=machenbach@chromium.org

Bug: v8:8239
Change-Id: I2451230f92fa6ad66ce6446f97efaa7fafd04e12
Reviewed-on: https://chromium-review.googlesource.com/1251524
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56337}
2018-10-02 14:18:48 +00:00
Sergiy Byelozyorov
1a236fb899 [tools] Print command output on timeout
R=machenbach@chromium.org

Bug: chromium:872257
Change-Id: I6eb1f70e0d44c76e18eb4f6f936c7f2cf0380444
Reviewed-on: https://chromium-review.googlesource.com/1217942
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55834}
2018-09-12 15:38:39 +00:00
Michael Achenbach
e3a0b362db [test] Fix test-listing command on Android
The cctest and unittests suites call the test executable to list the
tests, which requires pushing the executables to the device on Android.

NOTRY=true
TBR=sergiyb@chromium.org

Bug: chromium:866862
Change-Id: I318dff7af2b2de3b8642ec4b6ba30b602da808bb
Reviewed-on: https://chromium-review.googlesource.com/1213202
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55723}
2018-09-07 12:16:58 +00:00
Michael Achenbach
0dba4b907b Reland "[test] Increase coverage of d8_default test suites"
This is a reland of 8ac91f6c6a

Skips failing tests on gc stress and fixes predictable testing.

Original change's description:
> [test] Increase coverage of d8_default test suites
>
> NOTRY=true
>
> Bug: v8:7285,v8:8140
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I9cb216de302bc787189f8f12f5b254909b0f5773
> Reviewed-on: https://chromium-review.googlesource.com/1208496
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55682}

Bug: v8:7285, v8:8140, v8:8141
Change-Id: Ia7a437b874d5c8712f6def30382404e527145610
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Cq-Include-Trybots: luci.v8.try:v8_android_arm64_n5x_rel_ng
Cq-Include-Trybots: luci.v8.try:v8_linux_optional_rel_ng
Cq-Include-Trybots: luci.v8.try:v8_linux_gc_stress_dbg
Cq-Include-Trybots: luci.v8.try:v8_mac64_gc_stress_dbg
Reviewed-on: https://chromium-review.googlesource.com/1209762
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55701}
2018-09-07 07:53:29 +00:00
Michael Achenbach
6a41625000 Revert "[test] Increase coverage of d8_default test suites"
This reverts commit 8ac91f6c6a.

Reason for revert:
Some actual failures on Mac and debugger:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Mac64%20GC%20Stress/2914

Failing message tests need to be skipped for predictable testing:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux%20-%20predictable/18967

Original change's description:
> [test] Increase coverage of d8_default test suites
> 
> NOTRY=true
> 
> Bug: v8:7285,v8:8140
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I9cb216de302bc787189f8f12f5b254909b0f5773
> Reviewed-on: https://chromium-review.googlesource.com/1208496
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55682}

TBR=rmcilroy@chromium.org,machenbach@chromium.org,sergiyb@chromium.org

Change-Id: I701abe28317028acbf65769674f15517020d3496
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7285, v8:8140
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/1209347
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55683}
2018-09-06 12:00:54 +00:00
Michael Achenbach
8ac91f6c6a [test] Increase coverage of d8_default test suites
NOTRY=true

Bug: v8:7285,v8:8140
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I9cb216de302bc787189f8f12f5b254909b0f5773
Reviewed-on: https://chromium-review.googlesource.com/1208496
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55682}
2018-09-06 11:36:13 +00:00
Michael Achenbach
217cced963 [test] Enable Andoid testing on more JS test suites
Bug: chromium:866862
Change-Id: Ib6c89d88344c2348de155fd54eafa03cf18ea88d
Reviewed-on: https://chromium-review.googlesource.com/1206631
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55674}
2018-09-06 07:26:22 +00:00
Michael Achenbach
beb7778d87 [test] Fix resource-fetching logic for imported files
Path names in import/export statements are relative to the file they are in.

This fixes the logic and unblocks using the messages test suite on Android,
which has cases importing files from mjsunit, which import more files from
there.

Bug: chromium:866862
Change-Id: I8d2ff645f69b67fbdaf4a622d06308e55298b0ce
Reviewed-on: https://chromium-review.googlesource.com/1206570
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55643}
2018-09-05 10:27:07 +00:00
Michael Achenbach
6206a3e362 [test] Share resource-fetching logic with all d8 test cases
This shares logic for finding additional resources in JS source code.
Previously the logic was implemented for mjsunit, now it will be used
across all d8-based test cases.

This'll enable adding those test suites for Android testing.

Bug: chromium:866862
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I7c89ba141483aaf692a03c0e168edb61bbb7b010
Reviewed-on: https://chromium-review.googlesource.com/1193873
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55503}
2018-08-29 17:51:43 +00:00
Sergiy Byelozyorov
0e0157a42c [tools] Upload adb logcat to logdog when running tests
R=machenbach@chromium.org

No-Try: true
Bug: chromium:872257
Change-Id: I4de1a9dfccbb0d123c1c42c31a1f697e628623fa
Reviewed-on: https://chromium-review.googlesource.com/1171224
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55067}
2018-08-11 09:08:14 +00:00
Michael Achenbach
08e317da42 Reland "[test] Pass suite environment to Android test runner"
This is a reland of 18c90f4860

Original change's description:
> [test] Pass suite environment to Android test runner
>
> TBR=sergiyb@chromium.org
>
> Bug: v8:8046
> Change-Id: I201f7a7ee119628e14c864602a2d3cbadfc4052b
> Reviewed-on: https://chromium-review.googlesource.com/1170779
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55051}

NOTRY=true

Bug: v8:8046
Change-Id: I73923a2ae532bb7d773c87fd77844b4e655b568c
Reviewed-on: https://chromium-review.googlesource.com/1171382
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55064}
2018-08-10 19:02:43 +00:00
Michael Achenbach
b2537f219d Reland "[test] Add logic to run tests on Android"
This is a reland of 4c0943424c

Original change's description:
> [test] Add logic to run tests on Android
> 
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
> 
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
> 
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
> 
> Some test cases need an extra resource section for specifying required
> files.
> 
> Remaining failing tests are marked in the status files for later
> triage.
> 
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}

Bug: chromium:866862
Change-Id: Icf7e04c75d4abeab7254d10ba21240e46b0022ae
Reviewed-on: https://chromium-review.googlesource.com/1170643
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55059}
2018-08-10 17:56:12 +00:00
Sergiy Byelozyorov
2071051e4b Revert "[test] Add logic to run tests on Android"
This reverts commit 4c0943424c.

Reason for revert: Unfortunately this broke all perf builders.

Original change's description:
> [test] Add logic to run tests on Android
> 
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
> 
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
> 
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
> 
> Some test cases need an extra resource section for specifying required
> files.
> 
> Remaining failing tests are marked in the status files for later
> triage.
> 
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}

TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org

Change-Id: If80129810586b709dab762c9b5724888e15daec2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:866862
Reviewed-on: https://chromium-review.googlesource.com/1170962
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55058}
2018-08-10 16:16:16 +00:00
Sergiy Byelozyorov
36e5b7b35a Revert "[test] Pass suite environment to Android test runner"
This reverts commit 18c90f4860.

Reason for revert: needs to be reverted prior to https://crrev.com/c/1170962

Original change's description:
> [test] Pass suite environment to Android test runner
> 
> TBR=sergiyb@chromium.org
> 
> Bug: v8:8046
> Change-Id: I201f7a7ee119628e14c864602a2d3cbadfc4052b
> Reviewed-on: https://chromium-review.googlesource.com/1170779
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55051}

TBR=machenbach@chromium.org,sergiyb@chromium.org

Change-Id: I2c874305251b8b73f3bc0aaaf31a81ef30d6c15d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8046
Reviewed-on: https://chromium-review.googlesource.com/1171002
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55056}
2018-08-10 16:09:52 +00:00
Michael Achenbach
18c90f4860 [test] Pass suite environment to Android test runner
TBR=sergiyb@chromium.org

Bug: v8:8046
Change-Id: I201f7a7ee119628e14c864602a2d3cbadfc4052b
Reviewed-on: https://chromium-review.googlesource.com/1170779
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55051}
2018-08-10 13:35:18 +00:00
Michael Achenbach
4c0943424c [test] Add logic to run tests on Android
This adds a new command abstraction for running commands on Android
using dockered devices on swarming.

The new abstraction handles pushing all required files to the device.
The logic used for pushing and running is reused from the perf runner.

This adds only the mjsunit test suite. Others will be handled in
follow up CLs. The suite logic is enhanced with auto-detection of files
to be pushed to devices, for e.g. load or import statements.

Some test cases need an extra resource section for specifying required
files.

Remaining failing tests are marked in the status files for later
triage.

Bug: chromium:866862
Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
Reviewed-on: https://chromium-review.googlesource.com/1150153
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55041}
2018-08-10 10:11:53 +00:00
Michael Achenbach
de80e94c9a [test] Refactoring - extract Android driver logic from perf runner
This prepares for reusing the driver logic for correctness testing.

NOTRY=true

Bug: chromium:866862
Change-Id: If901ca8552cf48f29e240ed5b52ea7e722e0e1e4
Reviewed-on: https://chromium-review.googlesource.com/1163608
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54961}
2018-08-08 09:12:18 +00:00
Jakob Kummerow
f15cb8fe54 [tests] Make processing of test filters faster
The test driver compares command-line arguments against test names
it finds on disk. Using Python's "fnmatch" for this nicely handles
wildcards, but is relatively slow. For given test names that don't
contain any '*', we can use string equality testing, which is much
faster.

Example: the time to evaluate

 tools/run-tests.py --arch x64 --mode release \
   $(grep 'object-spread' -l -r test/test262/data/test/ | \
       sed -E 's|\.js$||' | \
       sed -E 's|^test/test262/data/test/|test262/|')

goes from "I gave up and killed the process after several minutes"
to a couple of seconds with this patch.

Change-Id: I9ec404b7516afd801fe6126347f6dff533d1977c
Reviewed-on: https://chromium-review.googlesource.com/1149196
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54704}
2018-07-25 20:48:05 +00:00
Michael Achenbach
2acc4d440b [test] Add reusable negative-test output processor
Bug: chromium:846711
Change-Id: Ic51e31a48106e495c3dc7d75ef4c97fa82071a64
Reviewed-on: https://chromium-review.googlesource.com/1148057
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54633}
2018-07-24 09:50:18 +00:00
Clemens Hammacher
76ac6372fa [wasm] Prepare enabling liftoff by default on ia32 and x64
Before flipping the flag, some tests need to be adapted. This CL
prepares these tests, such that the flag flip CL really just flips a
flag.

R=titzer@chromium.org, hablich@chromium.org

Bug: v8:6600, chromium:787421
Change-Id: I8030df69cda5f3fb81354350a37f65c0d1c669bd
Reviewed-on: https://chromium-review.googlesource.com/1110363
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53959}
2018-06-22 11:06:26 +00:00
Michael Achenbach
eb086bb6b2 [test] Change precedence of statusfile flags
Bug: v8:7812
Change-Id: Ia5a9c68e0aeb9d993ca2cd5f855ff39e172890e8
Reviewed-on: https://chromium-review.googlesource.com/1090282
Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53601}
2018-06-08 08:09:22 +00:00
Michael Achenbach
c401af670e [test] Enable loading suites from non-default test root
Bug: chromium:846711
Change-Id: I04e448c90f557f5ec23feae5989ece89e3c88dbc
Reviewed-on: https://chromium-review.googlesource.com/1073453
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53508}
2018-06-05 07:06:08 +00:00
Michael Lippautz
020d387f6a [testing] Add gc_stats variant
This is to allow testing object stats collection.

No-try: true
Bug: v8:7760
Change-Id: I1e5318f18bfb3799b90c0ee0aec6df87a794b32c
Reviewed-on: https://chromium-review.googlesource.com/1065478
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53274}
2018-05-22 07:19:20 +00:00
Michael Achenbach
3904290f63 [test] Pass --no-turbo-verify-allocation in nosnap builds
This is to speed up slow nosnap runs. Allocation verification is covered by
running mksnapshot on other builders.

Change-Id: I9c286f7dd4abac9cf1be45be4a483a7b36d09f8a
Reviewed-on: https://chromium-review.googlesource.com/1059113
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53234}
2018-05-17 13:55:02 +00:00
Eric Holk
4d4a3294b9 [test] Reverse sense of wasm_traps variant
D8 enables the Wasm trap handler by default now, but we need to make sure the
older bounds check case still gets test coverage too, as bounds checks will
continue to be a supported configuration.

Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I5b0bdded6929a9b3a8480e87d038398b8d2a0fd8
Reviewed-on: https://chromium-review.googlesource.com/1048835
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53078}
2018-05-08 17:49:14 +00:00
Michael Achenbach
cccc7abfe2 [test] Suppress more false positives with numfuzz
This makes testing with random gc stress use the status file entrees
for gc fuzzing as well, since many test cases not suitable for one, are
not suitable for the other.

This also skips two more tests that rely on assert(Un)Optimized, which
is unreliable with gc fuzzing.

TBR=sigurds@chromium.org

Change-Id: I33a617b251d5cf65cf6e486d07ec55cde050b8ab
Reviewed-on: https://chromium-review.googlesource.com/1021082
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52701}
2018-04-20 07:22:11 +00:00
Michael Achenbach
c2780ac4bf [test] Fix typo in numfuzzer
NOTRY=true
TBR=sergiyb@chromium.org

Bug: chromium:830557
Change-Id: I61510297e5195eb74425c71e41c4d95f080a280d
Reviewed-on: https://chromium-review.googlesource.com/1016420
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52659}
2018-04-18 07:44:35 +00:00
Michael Achenbach
24cfa03d85 [test] Make variants flag of numfuzz compatible with test runner
Prepares for: https://crrev.com/c/1015000

NOTRY=true

Bug: chromium:830557
Change-Id: I51c12867d95de19350e0e55aaf097bbb479348dc
Reviewed-on: https://chromium-review.googlesource.com/1015004
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52657}
2018-04-18 06:32:34 +00:00
Michael Achenbach
b27ee43a0f Reland "[build] Remove legacy isolate configurations"
This is a reland of 712b66da81

Breakage is fixed on infra side by:
https://crrev.com/c/983417

Original change's description:
> [build] Remove legacy isolate configurations
>
> Bug: chromium:669910
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: Iad58563fd4bb35501493f88af83362b1206a186c
> Reviewed-on: https://chromium-review.googlesource.com/982630
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#52267}

Bug: chromium:669910
Change-Id: I6c06a1fe9587206aa4e983befb105327bfec4154
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/983573
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52273}
2018-03-28 12:36:55 +00:00
Michael Achenbach
77a71f85c1 Revert "[build] Remove legacy isolate configurations"
This reverts commit 712b66da81.

Reason for revert:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64%20-%20builder/builds/32049

Original change's description:
> [build] Remove legacy isolate configurations
> 
> Bug: chromium:669910
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: Iad58563fd4bb35501493f88af83362b1206a186c
> Reviewed-on: https://chromium-review.googlesource.com/982630
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#52267}

TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org,jgruber@chromium.org

Change-Id: I1955325b0b419b38d793ab205131de8de08cb50a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:669910
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/983418
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52268}
2018-03-28 10:57:28 +00:00
Michael Achenbach
712b66da81 [build] Remove legacy isolate configurations
Bug: chromium:669910
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Iad58563fd4bb35501493f88af83362b1206a186c
Reviewed-on: https://chromium-review.googlesource.com/982630
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52267}
2018-03-28 10:35:05 +00:00
jgruber
68bdf6a75c [builtins] Remove --stress-off-heap-code flag
Embedded builtins are now based off the v8_enable_embedded_builtins
gn flag instead, which conditionally defines V8_EMBEDDED_BUILTINS.

Bug: v8:6666
Change-Id: I44d40d30fce3a3ed9bbf973d46c4990ba3fade40
Reviewed-on: https://chromium-review.googlesource.com/964361
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52092}
2018-03-21 08:52:17 +00:00
Michael Starzinger
0fc6c35c3a [wasm] Remove deprecated --wasm-jit-to-native flag.
The feature in question has been enabled by default for a while and we
no longer need to maintain a configuration without it enabled. Note that
this change only removes the mechanical pieces. Further cleanup enabled
by this will be done as follow-ups.

R=clemensh@chromium.org
BUG=v8:7549

Change-Id: I90e5bcddabe74a18a4d2a88132e8dc93317bcff4
Reviewed-on: https://chromium-review.googlesource.com/958424
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51883}
2018-03-12 14:36:32 +00:00
Ross McIlroy
323ad6a732 [Compiler] Remove unused background_compile flag.
We now unconditionally both parse and compile StreamedSource on the background
thread.

BUG=v8:5203

Change-Id: I42d6fe9059bc1745da3a415d270f46cf1c08b306
Reviewed-on: https://chromium-review.googlesource.com/948854
Reviewed-by: Mythri Alle <mythria@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51758}
2018-03-06 10:42:21 +00:00
Ilija Pavlovic
d2c2b39831 [mips][mips64]: Skip tests for MIPS release 6.
For MIPS architecture release 6, following tests will be skipped:
cctest/test-run-machops/RunFloat64MulAndFloat64Add1
cctest/test-run-machops/RunFloat64MulAndFloat64Add2
cctest/test-run-machops/RunFloat64MulAndFloat64Sub1
cctest/test-run-machops/RunFloat64MulAndFloat64Sub2

TEST=
BUG=

Change-Id: Id359580b809d1387d504d98b6d25d6c112cfda78
Reviewed-on: https://chromium-review.googlesource.com/945689
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#51754}
2018-03-06 09:19:11 +00:00
jgruber
abcc28ced0 [builtins] Enable embedded builtins and add testing variants
This enables the v8_enable_embedded_builtins gn flag on non-ia32 builds
and adds a new --stress-off-heap-code test mode to fyi bots.

v8_enable_embedded_builtins=true changes accesses to constants and
external references to go through the root list in builtins code.

--stress-off-heap-code copies builtins code off-heap on isolate
creation.

A few drive-by-fixes:
- ensure that we actually inspect the correct builtin during
  isolate-independence testing.
- relax tests to decrease maintenance (now we only fail if a builtin
  should be isolate-independent but isn't).
- switch to a different off-heap-trampoline register on arm due to
  conflicts with custom stub linkages.

Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Bug: v8:6666
Change-Id: I09ad3c75cb4342f4c548ea780f275993730896c8
Reviewed-on: https://chromium-review.googlesource.com/934281
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51717}
2018-03-05 09:50:01 +00:00
Ilija Pavlovic
1fd5a25b10 [test] Add status-file variable mips_arch_variant.
Introducing a new status-file variable: mips_arch_variant. With this
variable, in status files will be possible to define selections which
are based on MIPS architecture revisions/variants.

TEST=
BUG=

Change-Id: Ifd682552db2f26be4e56dc94ad50bed063ff14c5
Reviewed-on: https://chromium-review.googlesource.com/941212
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#51647}
2018-03-01 12:11:36 +00:00
Michael Achenbach
4a2d9b1b20 [test] Abort testing immediately on interrupt and sigterm
This is a partial revert of:
https://crrev.com/c/890938 and https://crrev.com/c/893982

Before this CL, the test runner blocked on ongoing tests in order to
process their results after an internal timeout. However, the logic
required for this feature was overly complicated and prevented an
acceptable implementation for fast aborts. Furthermore, also the fuzzers
suffered from timeouts on swarming due to hanging tests.

Instead, we now abort immediately on internal timeout (used on
fuzzers), SIGINT (Ctrl-C) and SIGTERM. Ongoing tests are immediately
terminated and their results are disregarded. On SIGTERM and SIGINT,
we return with non-zero exit codes, and zero on internal timeout.

This will also properly return json output, when the external hard
timeout is reached on swarming (causes SIGTERM).

TBR=sergiyb@chromium.org

Bug: v8:7423, chromium:813065
Change-Id: Ib20f835f58a0970693bdd3b21dc5d766d8e115d8
Reviewed-on: https://chromium-review.googlesource.com/924852
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51399}
2018-02-20 18:02:25 +00:00
Michael Achenbach
667173aab8 [test] Add thread-pool-size numfuzzer
NOTRY=true

Bug: v8:7455
Change-Id: Icb82e8196bc16b4c8b0eebb3c5820e6b3d581735
Reviewed-on: https://chromium-review.googlesource.com/924309
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51336}
2018-02-16 20:49:17 +00:00
Tobias Tebbi
3df2b3ade0 [test] make test runner less passive-aggressive
Change-Id: I31d5dddd74aa8b1bcd386a13fe34449dd6933547
Reviewed-on: https://chromium-review.googlesource.com/919163
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51293}
2018-02-14 15:49:25 +00:00
Michael Achenbach
70147ff2f0 Revert "Update test configurations for code serializer"
This reverts commit 9808093ef6.

Reason for revert: 
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/19149

Original change's description:
> Update test configurations for code serializer
> 
> Update test configurations on the bots to test for serializing
> after execute (--cache=after-execute) and producing full code
> cache (--cache=full-code-cache) options. We no longer need
> to test serializing before execute (--cache=code) on the bots.
> 
> Bug: v8:7302
> Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
> Reviewed-on: https://chromium-review.googlesource.com/869931
> Commit-Queue: Mythri Alle <mythria@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#51235}

TBR=machenbach@chromium.org,yangguo@chromium.org,mythria@chromium.org

Change-Id: I47e03101804194c21b0edf418b744b0ccb66cbf6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7302
Reviewed-on: https://chromium-review.googlesource.com/913134
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51236}
2018-02-12 12:09:54 +00:00
Mythri
9808093ef6 Update test configurations for code serializer
Update test configurations on the bots to test for serializing
after execute (--cache=after-execute) and producing full code
cache (--cache=full-code-cache) options. We no longer need
to test serializing before execute (--cache=code) on the bots.

Bug: v8:7302
Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
Reviewed-on: https://chromium-review.googlesource.com/869931
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51235}
2018-02-12 11:22:54 +00:00
Michael Achenbach
d18c314a20 [test] Fix using suppressed test cases with interrupt-budget fuzzer
The suppression flag must be set before loading the tests to have an effect.
This got turned around in https://crrev.com/c/899366 and is fixed again by
this CL.

TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6917
Change-Id: Ie58465288a0d6eec9a99a23d610710de9e1cdddd
Reviewed-on: https://chromium-review.googlesource.com/911114
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51209}
2018-02-09 14:05:16 +00:00
Clemens Hammacher
b60a4c6ac8 Add wasm_no_native variant
This new variant is executed on a small number of bots as part of the
"extra" suite. It checks that the wasm-jit-to-native flag can still be
disabled if unexpected failures pop up on a release branch.

R=machenbach@chromium.org

Bug: v8:7417
Change-Id: I1658cb2f04302fa80915b59bfedd85d980742db5
Reviewed-on: https://chromium-review.googlesource.com/909213
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51204}
2018-02-09 10:19:47 +00:00
Michael Achenbach
99fad4115d [test] Forgive tests timing out on num-fuzzers
Timeouts on num-fuzzer only lead to false positives, as tests might
take unpredictably longer.

This CL forgives timeouts through a global override mechanism of the
expected outcomes. This allows to remove already existing scattered
code that allowed timeouts in some test suites only.

Bug: v8:6917
Change-Id: Ib131765d360e335789c1952bc6793ed051e016ea
Reviewed-on: https://chromium-review.googlesource.com/908454
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51188}
2018-02-08 23:08:05 +00:00
Michal Majewski
3a0372f9f1 [test] Organize flags into sections and share more code
Bug: v8:6917
Change-Id: I4267900b6beed44eeae3df9b3b7a3e58402d6e6e
Reviewed-on: https://chromium-review.googlesource.com/899366
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51084}
2018-02-03 20:18:30 +00:00
Michael Achenbach
ac5ad35283 [test] Make it more likely to use smaller interrupt-budget on fuzzer
TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6917
Change-Id: Ie585754c84f42cd864ab3b161a1ae27f7c2d2ab5
Reviewed-on: https://chromium-review.googlesource.com/899303
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51070}
2018-02-02 14:45:17 +00:00
Michael Achenbach
f4411a32cd [test] Properly load mjsunit.js on endurance fuzzer
Load mjsunit.js inside the realm as otherwise the functions are not
available in the realm's scope.

This also prints timestamps after each test to easier track down slow
tests.

We also pass --omit-quit to not stop too early.

This also adds the ability to skip certain tests for endurance
fuzzing and skips some tests with known problems.

TBR=ulan@chromium.org,hpayer@chromium.org

Bug: v8:6972, v8:7400
Change-Id: I44464c28bfb10c84f2e59972e7b86945a47ca3b3
Reviewed-on: https://chromium-review.googlesource.com/899008
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51066}
2018-02-02 13:55:32 +00:00
Michal Majewski
533820fe83 [test] Move loading status file to base runner
Bug: v8:6917
Change-Id: I2fb63e05336798606dee558d4adc253d5aace040
Reviewed-on: https://chromium-review.googlesource.com/897807
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51038}
2018-02-01 15:32:59 +00:00
Michal Majewski
2fe1af4787 [test] Remove dead code from suites and tests
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I6bfea537347fb236d41b31f0c6c64c108928bf28
Reviewed-on: https://chromium-review.googlesource.com/897784
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51033}
2018-02-01 14:12:49 +00:00
Michal Majewski
e64f546140 [test] Remove old code from run-tests
Only test processors code left. It enabled to move more stuff to
the base runner, like progress indicators creation.

Bug: v8:6917
Change-Id: Ie6dd211cec561a07d92bcc4431ea88eb1842c8fa
Reviewed-on: https://chromium-review.googlesource.com/897624
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51030}
2018-02-01 13:24:57 +00:00
Michal Majewski
bb9d073a2c [test] Always run with test processors
System tests updated to pass. I will cleanup the code in a
separate CL removing the old code.

Bug: v8:6917
Change-Id: Ie49efe06c43faade846e322b35c4a2068137a88b
Reviewed-on: https://chromium-review.googlesource.com/897565
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51025}
2018-02-01 10:46:32 +00:00
Michal Majewski
0d013072f7 [test] Set exit code to 0 even with remaining tests
Bug: v8:6917
Change-Id: Ibc6a7e049a17b8f9bef6b2f5cb187c11f89cf638
Reviewed-on: https://chromium-review.googlesource.com/897564
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51024}
2018-02-01 10:39:12 +00:00
Michal Majewski
804cc9f385 [test] Remove context usage from test processors
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Id5a7b41502dfd4be43496b1edb958522a8eb0c31
Reviewed-on: https://chromium-review.googlesource.com/895588
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51023}
2018-02-01 10:35:02 +00:00
Michael Achenbach
8990851521 [test] Fully fix timeout-scalefactor for numfuzzer
TBR=majeski@google.com
NOTRY=true

Bug: v8:6917
Change-Id: I8ca2297ab9641b81a91ee7f8ff594bbe4b9e369b
Reviewed-on: https://chromium-review.googlesource.com/895366
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51005}
2018-01-31 17:21:57 +00:00
Michael Achenbach
fcb5b4a501 [test] Fix timeout-scalefactor for numfuzzer
TBR=majeski@google.com
NOTRY=true

Bug: v8:6917
Change-Id: I088dc4492df6c1dda1f86fbd6d7153a46a4117e0
Reviewed-on: https://chromium-review.googlesource.com/895600
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51000}
2018-01-31 15:09:03 +00:00
Michal Majewski
dbb198af92 [test] Random seed and rerun proc moved to base runner
Bug: v8:6917
Change-Id: I543c232489e6b93f5f98ccf63eea475535d82613
Reviewed-on: https://chromium-review.googlesource.com/893566
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50998}
2018-01-31 14:35:38 +00:00
Michal Majewski
d3083b46ee [test] Move options to test config
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I254d2e545709029346f585b02a9edf91d3f27893
Reviewed-on: https://chromium-review.googlesource.com/893321
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50996}
2018-01-31 13:46:29 +00:00
Michal Majewski
35ca0a01c3 [test] Filter out incompatible tests in predictable mode
Bug: v8:6917
Change-Id: Ie18390ae0849e8d5341e4c2a7b757fae886c2986
Reviewed-on: https://chromium-review.googlesource.com/895582
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50994}
2018-01-31 13:32:41 +00:00
Michal Majewski
4ca5a577e6 [test] Stressing random seed with multiple subtests in parallel
Bug: v8:6917
Change-Id: I562680ebbc4b3140fb0e5be7cab7d5ebb658bbac
Reviewed-on: https://chromium-review.googlesource.com/893272
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50989}
2018-01-31 12:06:11 +00:00
Michal Majewski
1478c30786 [test] Return heartbeats and results during pool termination
Bug: v8:6917
Change-Id: I5cca65111141f32f8b9f241a9f482d09e1b54655
Reviewed-on: https://chromium-review.googlesource.com/893982
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50987}
2018-01-31 11:25:26 +00:00
Michal Majewski
d161bc3c14 [test] Added signal handling to test processors
Bug: v8:6917
Change-Id: If91defd11c91fd26bdbacc146992745ea772a941
Reviewed-on: https://chromium-review.googlesource.com/890938
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50985}
2018-01-31 10:39:06 +00:00
Predrag Rudic
d2ad89d846 [cctest] Skip test-run-wasm-simd if SIMD is not supported
For mips, if 'mips_arch_variant=="r6"' and if 'mips_use_msa' flag is set
to 'true', then test-run-wasm-simd tests won't be skipped for mips. It
will also force 'MIPS_SIMD' bit in CpuFeatures to be set.
ARM processors are assumed to support SIMD.

Change-Id: Iea668b97ef995ca4949ddbf2ffc734aad89d3aa3
Reviewed-on: https://chromium-review.googlesource.com/868430
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#50981}
2018-01-31 10:08:14 +00:00
Michal Majewski
5ede3cb533 Reland "[test] Random seed processor"
This is a reland of 0db74d4974.

Original change's description:
> [test] Random seed processor
> 
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
> 
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}

Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
2018-01-31 09:53:34 +00:00
Michael Achenbach
ba30988cbc Revert "[test] Random seed processor"
This reverts commit 0db74d4974.

Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1

Original change's description:
> [test] Random seed processor
> 
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
> 
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
2018-01-30 22:05:03 +00:00
Michal Majewski
0db74d4974 [test] Random seed processor
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.

Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
2018-01-30 15:50:49 +00:00
Michael Achenbach
f6ed964d18 [test] Reduce timeout with interrupt-budget fuzzer
TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6917
Change-Id: I4903ffc49b5ed475706c0dc604a92b90f6b5d2a0
Reviewed-on: https://chromium-review.googlesource.com/892866
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50955}
2018-01-30 11:52:29 +00:00
Michael Achenbach
dc210a95d3 [test] Add interrupt-budget fuzzer
This adds back an option for interrupt budget available in no-snap
builds. This also adds a fuzzer configuration for numfuzz that enables
fuzzing the interrupt budget option. A new flag --disable-analysis
allows to generally skip the fuzzer's analysis phase, which can be
chosen for interrupt budget, which doesn't support an analysis phase.

Bug: v8:6917
Change-Id: I546dd9ee41c3e0fb027108ef4606a34514f230d4
Reviewed-on: https://chromium-review.googlesource.com/885805
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50929}
2018-01-29 16:31:40 +00:00
Michael Achenbach
399101df1d [test] Wrap combined tests with a realm on gc fuzzer
Without realm-eval, the toplevel references in loaded tests are not
garbage-collected and changes in the tests can affect the global
object.

TBR=ulan@chromium.org
NOTRY=true

Bug: v8:7379, v8:7376
Change-Id: Id4d451e38fe49c6531a07d541ad3c67298b6acf2
Reviewed-on: https://chromium-review.googlesource.com/890519
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50915}
2018-01-27 14:39:26 +00:00
Michael Achenbach
68c85fe0be [test] Use new test runner by default
TBR=sergiyb@chromium.org

Bug: v8:7343
Change-Id: I0739a72f6453d682eb840b1b3650fc697c05df7d
Reviewed-on: https://chromium-review.googlesource.com/888643
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50905}
2018-01-26 14:58:23 +00:00
Michal Majewski
1e3a8c156b [test] Remove old gc and deopt fuzzers
They have been replaced with num_fuzzer.

Bug: v8:6917
Change-Id: I2f78df308cec0a58f0d665bce82503dee68fcebc
Reviewed-on: https://chromium-review.googlesource.com/888641
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50903}
2018-01-26 14:36:03 +00:00
Michal Majewski
385611cbba Reland "[test] Keep the order of variants"
This is a reland of 8f4407fbd6.

Original change's description:
> [test] Keep the order of variants
> 
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}

Bug: v8:6917
Change-Id: I9d4c617ccec6db8929af3095f2f27d8964c5d6b8
Reviewed-on: https://chromium-review.googlesource.com/888582
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50901}
2018-01-26 14:15:53 +00:00
Michael Achenbach
874eadd1ea Revert "[test] Keep the order of variants"
This reverts commit 8f4407fbd6.

Reason for revert: Failing in old test runner. See comment.

Original change's description:
> [test] Keep the order of variants
> 
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: Ia8608053e080c70d2ac09d7921a6abb6cd0e338f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/888749
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50895}
2018-01-26 13:04:29 +00:00
Michael Achenbach
04d95171f4 Revert "[test] Let fuzzer total timeout also stop the execution loop"
This reverts commit f4ebbb3fd4.

Reason for revert: Suspected to make things worse than better, e.g.:
https://chromium-swarm.appspot.com/task?id=3b4b2f864304f010&refresh=10&show_raw=1
and
https://chromium-swarm.appspot.com/task?id=3b4b2f8045da5510&refresh=10&show_raw=1

Original change's description:
> [test] Let fuzzer total timeout also stop the execution loop
> 
> TBR=sergiyb@chromium.org
> 
> Bug: v8:6917
> Change-Id: I5bc8f49dc01d98949e3efab01192c663de8027bf
> Reviewed-on: https://chromium-review.googlesource.com/888578
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50887}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: Ib9f530348594e361d491e827aa03a38f41da9f1a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/888519
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50894}
2018-01-26 13:02:50 +00:00
Michael Achenbach
f4ebbb3fd4 [test] Let fuzzer total timeout also stop the execution loop
TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: I5bc8f49dc01d98949e3efab01192c663de8027bf
Reviewed-on: https://chromium-review.googlesource.com/888578
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50887}
2018-01-26 11:44:38 +00:00
Michael Achenbach
e19bc613ca [test] Tolerate timeout on numfuzz with combined tests
A timeout result of randomly combined tests is not a useful indicator.
Therefore we ignore those now. This also prevents timeouts from being
re-run.

TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6917
Change-Id: I8e5eb93e2f13067ca5270fb152c3dc1eda1bb3c2
Reviewed-on: https://chromium-review.googlesource.com/888559
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50879}
2018-01-26 08:04:15 +00:00
Michal Majewski
8f4407fbd6 [test] Keep the order of variants
Bug: v8:6917
Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
Reviewed-on: https://chromium-review.googlesource.com/886841
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50874}
2018-01-25 17:52:25 +00:00
Michal Majewski
084b0e763e [test] Support predictable with test processors
Bug: v8:6917
Change-Id: I36b7ef75d34f9dfee20cba09d55063d816334293
Reviewed-on: https://chromium-review.googlesource.com/887021
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50872}
2018-01-25 16:07:55 +00:00
Michal Majewski
88c8bf2e71 [test] Add combine tests option to numfuzz
Bug: v8:6917
Change-Id: I3ba4ca3df8bac400c248fa16c58fcba3497da806
Reviewed-on: https://chromium-review.googlesource.com/881167
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50871}
2018-01-25 15:50:24 +00:00
Michal Majewski
7557be5a06 Reland "[test] Move timeout control to timeout processor"
This is a reland of 4de2be999d.

Original change's description:
> [test] Move timeout control to timeout processor
> 
> Bug: v8:6917
> Change-Id: I03be38be952f0d59eb20fa98102ef09ca795de40
> Reviewed-on: https://chromium-review.googlesource.com/883446
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50848}

Bug: v8:6917
Change-Id: I52798af84991d4815910e9da5f4837329cdb3c96
Reviewed-on: https://chromium-review.googlesource.com/885765
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50870}
2018-01-25 14:09:44 +00:00
Michael Achenbach
b6cd8ebcb9 [test] Roll out new test runner to remaining bots
TBR=sergiyb@chromium.org

Bug: v8:7343
Change-Id: I24ec4d9a58596cca22ca34a48b5a7297ed24b132
Reviewed-on: https://chromium-review.googlesource.com/886341
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50866}
2018-01-25 10:32:29 +00:00
Michal Majewski
bc7d52410f Reland "[test] Move timeout control to timeout processor"
This is a reland of 4de2be999d.

Original change's description:
> [test] Move timeout control to timeout processor
>
> Bug: v8:6917
> Change-Id: I03be38be952f0d59eb20fa98102ef09ca795de40
> Reviewed-on: https://chromium-review.googlesource.com/883446
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50848}

Bug: v8:6917
Change-Id: I7c83bf1cb8965a149b06c1dbc138b5d097f5275d
Reviewed-on: https://chromium-review.googlesource.com/883284
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50854}
2018-01-24 20:24:48 +00:00
Michael Achenbach
c077ff54ae Revert "[test] Move timeout control to timeout processor"
This reverts commit 4de2be999d.

Reason for revert: Testing stops too early now, e.g. here after 2 min:
https://build.chromium.org/p/client.v8.clusterfuzz/builders/V8%20NumFuzz/builds/32

Original change's description:
> [test] Move timeout control to timeout processor
> 
> Bug: v8:6917
> Change-Id: I03be38be952f0d59eb20fa98102ef09ca795de40
> Reviewed-on: https://chromium-review.googlesource.com/883446
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50848}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I6a925866476c69b3b50aa08e99facca0eaaa396b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/884082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50851}
2018-01-24 19:30:17 +00:00
Michal Majewski
4de2be999d [test] Move timeout control to timeout processor
Bug: v8:6917
Change-Id: I03be38be952f0d59eb20fa98102ef09ca795de40
Reviewed-on: https://chromium-review.googlesource.com/883446
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50848}
2018-01-24 19:02:37 +00:00
Michael Achenbach
ee4afcbf39 [test] Roll out new test runner to all testing with arch x64
This will affect all manual test runs with x64. Most bots on x64 already
migrated.

TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:7343
Change-Id: I87f46f1848a813c0b320b3e9901481b9232025a5
Reviewed-on: https://chromium-review.googlesource.com/883101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50825}
2018-01-24 09:06:23 +00:00
Michael Achenbach
ed93f01462 [test] Prepare roll-out of new test runner to local runs
This adds a tri-state flag --infra-staging and --no-infra-staging, which
can be used to explicitly opt in or out of the staging test runner.

If not specified, a new architecture whitelist will enable roll-out per
arch for manual (none CI) runs.

We'll start whitelisting archs in follow ups.

Bug: v8:7343
Change-Id: I1228e48969fd379f5c231a2b8fad4afc01da94c0
Reviewed-on: https://chromium-review.googlesource.com/881169
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50812}
2018-01-23 16:37:47 +00:00
Michael Achenbach
3fa26f4d1b [test] Roll out new test runner to arm64 bots
TBR=sergiyb@chromium.org

Bug: v8:7343
Change-Id: Id17a931e00eda0bf018b5f1cb1cd6bac516ec26d
Reviewed-on: https://chromium-review.googlesource.com/881482
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50811}
2018-01-23 16:20:47 +00:00
Michal Majewski
432668ed32 [test] Add option to disable analysis phase
Introduce option to run fuzzer processor without analysis phase.
It will be used in fuzzing combined tests.

Bug: v8:6917
Change-Id: Ic96d6b8c5a35c81da80340555bdd75c0d518cb5a
Reviewed-on: https://chromium-review.googlesource.com/880948
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50810}
2018-01-23 16:15:58 +00:00
Michael Achenbach
51a58ac42b [test] Roll out new test runner to mac bots
TBR=sergiyb@chromium.org

Bug: v8:7343
Change-Id: Id2f60f248b40592607a0bf8f74e1169866e317d7
Reviewed-on: https://chromium-review.googlesource.com/880947
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50803}
2018-01-23 14:23:21 +00:00
Michael Achenbach
6c389866af [test] Roll out new test runner to windows bots
TBR=sergiyb@chromium.org

Bug: v8:7343
Change-Id: I46d276f16a001651ad36d550df2535fb8b3ba150
Reviewed-on: https://chromium-review.googlesource.com/880584
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50791}
2018-01-23 10:06:36 +00:00
Michael Achenbach
195a74a152 [test] Remove temporary debug output
The extra output on mac was to investigate a bug that's now fixed.

TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6927
Change-Id: Iac8074c7e89a987e164f676442da1ed4f93987cc
Reviewed-on: https://chromium-review.googlesource.com/880623
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50787}
2018-01-23 09:23:45 +00:00
Michael Achenbach
bff9d3e4ca [test] Roll out new test runner to remaining linux64 bots
TBR=sergiyb@chromium.org

Bug: v8:7343
Change-Id: Iea94f5b12b48fb6c04fe25b827a30e713fdfb716
Reviewed-on: https://chromium-review.googlesource.com/880582
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50786}
2018-01-23 09:08:01 +00:00
Michal Majewski
0998eda9a4 [test] Disable reduce result on the main process
Since we're not winning anything by changing the result between
processors on the main process, reduce is noop there and result is
immutable.

Bug: v8:6917
Change-Id: Ieb282e7abd4ab31162aee6b52493a6e1b6a25109
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/878239
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50784}
2018-01-23 08:58:20 +00:00
Michael Achenbach
937d161eae [test] Rollout new test runner to linux64 bots
Bug: v8:7343
Change-Id: Ic0b89e2376fc927303a5285cc5b47a1ad4694047
Reviewed-on: https://chromium-review.googlesource.com/878642
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50779}
2018-01-23 07:32:40 +00:00
Michal Majewski
b5e6a1517e [test] Add stress deopt to num fuzzer
Bug: v8:6917
Change-Id: I9f23515de0a1ae89babe41a42ab37fb2dfb67b48
Reviewed-on: https://chromium-review.googlesource.com/876324
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50767}
2018-01-22 15:45:56 +00:00
Michael Achenbach
174485e9c8 [test] Add master/buildermap to prepare rollout of test processors
Bug: v8:7343
Change-Id: I673a490e04f7bae56199591db69b7f1c84022fc0
Reviewed-on: https://chromium-review.googlesource.com/878541
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50764}
2018-01-22 15:15:32 +00:00
Michael Achenbach
4224421622 [test] Add swarming option to numfuzz
This is added only to uniformly call all tools by the infra side.

NOTRY=true
TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: I85a14ea51abfe1bfc775fd73d2fee02bfb2fb31e
Reviewed-on: https://chromium-review.googlesource.com/878361
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50754}
2018-01-22 12:48:11 +00:00
Michael Achenbach
7497fa1ab6 [test] Add rerun and json output to numfuzz
This adds flags to numfuzz for rerunning flaky tests and for storing
json test results. With those flags added, the infra-side can call
numfuzz with the same API as the standard test-runner.

TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: I02d1cd02d90677c83f10e072383f3650c041cab1
Reviewed-on: https://chromium-review.googlesource.com/877890
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50753}
2018-01-22 12:26:36 +00:00
Michael Achenbach
9d7c40ea76 [test] Fail when no tests are run
TBR=sergiyb@chromium.org

Bug: v8:7337
Change-Id: I1732f6e587305ce4ab41a65f73e943c7eb9e1d15
Reviewed-on: https://chromium-review.googlesource.com/877760
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50747}
2018-01-22 09:57:53 +00:00
Michal Majewski
df1b44b63f [test] Fix infinite loop and fail result in the num fuzzer
1. Fix infinite loop caused by time based fuzzing
2. Shallow copy of the result to avoid dropping output
by different processor.

Bug: v8:6917
Change-Id: Icf823e853be9d3cc8dfd46ed2fb954979bf02d2f
Reviewed-on: https://chromium-review.googlesource.com/877761
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50737}
2018-01-20 20:47:14 +00:00
Michal Majewski
fb9e22123d [test] Running num fuzzer for specific time
Bug: v8:6917
Change-Id: I7576a3b8a7fb95244b241532f50759e1c88f6a5a
Reviewed-on: https://chromium-review.googlesource.com/876427
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50734}
2018-01-19 20:05:19 +00:00
Michal Majewski
50a91fe9ee [test] Fix subtest creation
Update flags before recalculating outcomes.

Bug: v8:6917
Change-Id: I5f54f7d14fd60f7e35a976e5200d3f7f0e74a5b7
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/876364
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50731}
2018-01-19 18:37:08 +00:00
Michal Majewski
61c562b026 [test] Implement gc fuzzer with test processors
Bug: v8:6917
Change-Id: I2a7ecc6897c8ccd6ed862cf2b0b484673ee359f6
Reviewed-on: https://chromium-review.googlesource.com/871310
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50728}
2018-01-19 16:58:49 +00:00
Michael Achenbach
eda125998b [test] Temporarily remove benchmarks from deopt fuzzer
The tests currently time out due to too many deopt points and hence too many
tests generated.

TBR=sergiyb@chromium.org

Bug: v8:6900
Change-Id: I0998097024a4ed9c087728bb5ef288ab17d3371e
Reviewed-on: https://chromium-review.googlesource.com/876322
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50727}
2018-01-19 16:12:58 +00:00
Michael Achenbach
d65b53cf23 [test] Add benchmarks to deopt-fuzzer and remove obsolete configs
This adds the benchmarks suite to the deopt fuzzer and removes the
obsolete deopt configs, which since a while are part of the num-fuzz
configs.

TBR=sergiyb@chromium.org

Bug: v8:6900
Change-Id: I9202e2a66a132eca5940b4444fda97816b034e54
Reviewed-on: https://chromium-review.googlesource.com/876085
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50720}
2018-01-19 13:47:06 +00:00
Michael Achenbach
8124e06993 [test] Prepare rollout of new test-runner features
TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6917
Change-Id: Ic4f9d151be887fa5795ebe62dcdd3b849cc176ab
Reviewed-on: https://chromium-review.googlesource.com/875965
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50715}
2018-01-19 13:18:36 +00:00
Michael Achenbach
a751e73725 [test] Add random-gc-stress mode to test runner
TBR=sergiyb@chromium.org

Bug: v8:6972
Change-Id: Ia4225f0a2a102ad6b59d2436d7f32aaee90458e7
Reviewed-on: https://chromium-review.googlesource.com/875962
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50710}
2018-01-19 11:14:15 +00:00
Michael Achenbach
d63c331966 [test] Add more stress options to gc fuzzer
This adds gc stress and scavenge stress to legacy gc fuzzer. Each
flag can be used in separation or combined.

TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6972
Change-Id: Id03ef96bd42b77415dd201e7f78b3b42786dd3bc
Reviewed-on: https://chromium-review.googlesource.com/874452
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50708}
2018-01-19 10:34:04 +00:00
Michael Achenbach
f2f3a6857e [test] Prepare making stress-marking mode explicit in gc fuzzer
This prepares to add the stress-marking flag on the infra side.

TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6972
Change-Id: Ibee30beadb167d06fd7965dfd3cc05fb523158cb
Reviewed-on: https://chromium-review.googlesource.com/874350
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50697}
2018-01-18 17:59:59 +00:00
Michal Majewski
9e69733b3c [test] Move shard methods to the base runner
Bug: v8:6917
Change-Id: I0b81ebfe289b459e30ad85a4a62ed244cbd20b65
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/870123
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50657}
2018-01-17 14:29:48 +00:00
Michal Majewski
daed2eb723 [test] Move command from test to the result
Bug: v8:6917
Change-Id: I54f908609fadd88bb23bf9fc566d1e2f3ff5e18e
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/870353
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50653}
2018-01-17 12:59:54 +00:00
Michal Majewski
0bfc1b2562 [test] Do not pass unnecessary results
Bug: v8:6917
Change-Id: I1a355bdfe3f873091a7d7c32a937a533a7d8b3d4
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/867053
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50650}
2018-01-17 11:32:12 +00:00
Michal Majewski
96d06a6351 [test] Fix args parsing and name filter
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng;luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I14c7e0a140bfe828ee8199606db36cab12be14ff
Reviewed-on: https://chromium-review.googlesource.com/868135
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50649}
2018-01-17 10:32:12 +00:00
Michal Majewski
4ac4e3581f [test] Run gc fuzzer on benchmarks test suite
Bug: v8:6917
Change-Id: I0ee15a41b6028bdbf8448f066b0bc47a049ca587
Reviewed-on: https://chromium-review.googlesource.com/868154
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50647}
2018-01-17 09:59:15 +00:00
Michal Majewski
00ac7641a6 [test] Drop starting message from indicators
Since test processors create tests dynamically we cannot simply
count how many tests will be run. Instead we count only base tests
that we've loaded, before creating variants.

Bug: v8:6917
Change-Id: Ibc5b9a73f6afad423572afa575f477ca661a99d5
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/868290
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50642}
2018-01-17 09:31:19 +00:00
Michal Majewski
d53c4aa065 [test] Implement shard processor
Bug: v8:6917
Change-Id: I5b77e7445ca3a8eb5692659e94d3b8266479b415
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/866866
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50615}
2018-01-16 11:33:03 +00:00
Michal Majewski
3bcc6b3e0e [test] Filter tests based on cmd line processor
Bug: v8:6917
Change-Id: I7fa8f1857f338551dd7acd1b25eb7e9feb376576
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/866720
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50602}
2018-01-16 08:50:57 +00:00
Michal Majewski
767deb504a [test] Move loading test suites to the base runner
Bug: v8:6917
Change-Id: I5ad7f9f28be5eb0d8cdc424c0ed5cb41623212e5
Reviewed-on: https://chromium-review.googlesource.com/866505
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50601}
2018-01-16 08:47:52 +00:00
Michael Achenbach
997d5a6a5c [test] Add code_serializer testing variant
Also sort all variants in configuration mapping.

Bug: v8:7285
Change-Id: I2d3032b911e48c34557e47e50345fb29da7bbe85
Reviewed-on: https://chromium-review.googlesource.com/866502
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50584}
2018-01-15 15:04:07 +00:00
Michael Achenbach
56fa0a535b [test] Remove deprecated FAST_VARIANTS modifier
Bug: v8:7264
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I0c07c92955c0b6c145c7b72e5014f828eaf59c9a
Reviewed-on: https://chromium-review.googlesource.com/865903
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50582}
2018-01-15 14:30:47 +00:00
Michal Majewski
bddfee9822 [test] Add special result for rerun and json indicator.
Bug: v8:6917
Change-Id: I5136f183bd1728a1ab90a9ebb2560d978e17ef28
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/863623
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50576}
2018-01-15 12:54:23 +00:00
Michal Majewski
60c17bf983 [test] Simplify passing results between test processors.
Bug: v8:6917
Change-Id: Id73e4892a0d1b3b9c5bdd70ccc136e7bd2edf360
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/863603
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50565}
2018-01-15 08:51:40 +00:00
Adam Klein
1fad273e1d Revert "[test] Add more d8-based test suites to d8_default"
This reverts commit 2de796fc57.

Reason for revert: Fails tests on Linux dbg:

https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/18708

Original change's description:
> [test] Add more d8-based test suites to d8_default
> 
> This will run those suites with gc stress, code serializer, and verify-
> predictable mode.
> 
> TBR=sergiyb@chromium.org
> 
> Bug: v8:7285
> Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
> Reviewed-on: https://chromium-review.googlesource.com/864922
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50555}

TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I8d769829bdb96820e57b1cefcf92fb6976a1031d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7285
Reviewed-on: https://chromium-review.googlesource.com/865334
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50556}
2018-01-12 20:08:25 +00:00
Michael Achenbach
2de796fc57 [test] Add more d8-based test suites to d8_default
This will run those suites with gc stress, code serializer, and verify-
predictable mode.

TBR=sergiyb@chromium.org

Bug: v8:7285
Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
Reviewed-on: https://chromium-review.googlesource.com/864922
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50555}
2018-01-12 19:18:24 +00:00
Michael Achenbach
df342b8ab1 [test] Add a d8_default test suite collection
This adds a reduced version of bot_default with only d8-based test suites.
Those can be used for gc stress testing, fuzzing and predictable mode.

This also adds a missing explicit build rule for the debugger test suite.
It's working as it's part of the isolate file, but incremental
re-isolation might break on code changes.

Running the tests is still commented out to allow the infra-side to
land. Afterwards, we'll activate the extra testing as a V8 CL
commenting the lines back in.

Bug: v8:7285
Change-Id: I7b758c212f0c4ae3d2f79beea5ec597cee479565
Reviewed-on: https://chromium-review.googlesource.com/864045
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50545}
2018-01-12 15:18:04 +00:00
Michal Majewski
29b1c388e7 [test] Introduce stress sampling allocation profiler mode
Bug: v8:7209
Change-Id: Ia10479d7eea6ef2f352d008e2f4b74e2394ab79b
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/861623
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ali Ijaz Sheikh <ofrobots@google.com>
Cr-Commit-Position: refs/heads/master@{#50527}
2018-01-12 11:34:16 +00:00
Michal Majewski
7f01c9303f [test] Implement variant generators as processors
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng;luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Ib5bfdf4d6fee6102f62c7334a1b22146f1a1fc5b
Reviewed-on: https://chromium-review.googlesource.com/857376
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50525}
2018-01-12 10:39:20 +00:00
Michael Starzinger
4d09583ad9 [test] Restore "stress_incremental_marking" variant.
This removes the explicit --write-protect-code-memory flag from the list
of flags in the "stress_incremental_marking". The feature is enabled by
default by now and no longer needs explicit testing.

R=ulan@chromium.org
BUG=v8:6792

Change-Id: I5d6ba21dff261488bbe1b0148ce204bf78d57334
Reviewed-on: https://chromium-review.googlesource.com/860661
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50500}
2018-01-11 12:14:14 +00:00
Michal Majewski
463dbab3ec [test] Implemented status file filter as a processor
Bug: v8:6917
Change-Id: I4b10091a40372e1aa614ac26452e20ed481ab686
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/856498
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50482}
2018-01-10 16:08:46 +00:00
Michal Majewski
53a367ece4 [test] Use analysis flag in gc fuzzer.
Bug: v8:6972
Change-Id: I0f3f933f14b26771b73daa053c17ce4cf8906bd7
Reviewed-on: https://chromium-review.googlesource.com/819635
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50459}
2018-01-09 17:51:19 +00:00
Michal Majewski
cfd43ee713 [test] Added test processor documentation.
Bug: v8:6917
Change-Id: I6ecfab76e6d2ee0b4ab86380e8cfbb68df07599e
Reviewed-on: https://chromium-review.googlesource.com/852295
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50457}
2018-01-09 17:05:13 +00:00
Michal Majewski
ed619fa6c6 [test] Make variant names unique
Temporary workaround to avoid the same variant name for multiple
variants of the same test.

Bug: v8:6917
Change-Id: I9a25dcaf81d35da0dc2617c089cb4811c2a958cb
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/852833
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50428}
2018-01-09 11:18:40 +00:00
Michal Majewski
501413b9b9 [test] Implemented indicators as processors
Added simple system tests for different progress indicators.

Bug: v8:6917
Change-Id: I906ddfd06e82cc19d3b2210e09457456be00309b
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/852495
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50406}
2018-01-08 13:08:40 +00:00
Michal Majewski
da3416f53a [test] Introduce test processors
Test processors can be enabled with --infra-staging flag.

Rerunning tests, execution and verbose progress indicator already
work as test processors.

Bug: v8:6917
Change-Id: I40fc42db94dbc8629e8000a3d363030045532fe3
Reviewed-on: https://chromium-review.googlesource.com/850398
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50399}
2018-01-08 11:27:47 +00:00
Michael Achenbach
48d436b2ed [test] Add "trusted" testing variant to a subset of bots
Bug: chromium:798982
Change-Id: I096e2df5a5f8d038c980bc9f575857256f0a01df
Reviewed-on: https://chromium-review.googlesource.com/852073
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50378}
2018-01-05 10:36:31 +00:00
Michal Majewski
9a3773e665 [test] Fix predictable mode with output processors
Bug: v8:6917
Change-Id: I7bc5d09ff19a9cf7cb4bee68ca6cf34f5cfa8b19
Reviewed-on: https://chromium-review.googlesource.com/850874
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50365}
2018-01-04 19:16:45 +00:00
Michal Majewski
9f7d440e98 [test] Check output on the worker process.
I added additional exception logging in the execution.py since
errors in processing results were really difficult to debug.

There is a problem on Windows with class serialization when
it comes from dynamically loaded module. To fix it I moved all
output processors to the tools/testrunner/outproc/ and import
them in test/*/testcfg.py.

Bug: v8:6917
Change-Id: Ida604641d659b006e91faf1d56a37769ec47f5f3
Reviewed-on: https://chromium-review.googlesource.com/842784
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50361}
2018-01-04 14:12:31 +00:00
Michael Achenbach
c1e9bc02f8 Revert "Revert "[test] Move has unexpected output to outproc.""
This reverts commit 1685b5d27a.

Reason for revert: Was probably caused by infra change:
https://crrev.com/c/845781

Original change's description:
> Revert "[test] Move has unexpected output to outproc."
> 
> This reverts commit 71605b3ea4.
> 
> Reason for revert: Seems to break static-initializers step:
> https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
> 
> Original change's description:
> > [test] Move has unexpected output to outproc.
> > 
> > Expected outcomes optimized to serialize [PASS] as None.
> > 
> > Keeping expected outcomes inside output processors should be
> > optimized in the future. Few possible optimizations:
> > - separate classes for tests that are expected to PASS - done as
> > an example in mozilla test suite.
> > - cache output processors inside testcase.
> > - share output processors between copies of the same test - needs
> > some updates to the create_variant to update outproc only if
> > expected outcomes changed.
> > 
> > Bug: v8:6917
> > Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> > Reviewed-on: https://chromium-review.googlesource.com/843025
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#50347}
> 
> TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
> 
> Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6917
> Reviewed-on: https://chromium-review.googlesource.com/849572
> Reviewed-by: Bill Budge <bbudge@chromium.org>
> Commit-Queue: Bill Budge <bbudge@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50348}

TBR=bbudge@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I7a522b6487de6e96985d223524533493eb9171f0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/848975
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50350}
2018-01-04 07:39:21 +00:00
Bill Budge
1685b5d27a Revert "[test] Move has unexpected output to outproc."
This reverts commit 71605b3ea4.

Reason for revert: Seems to break static-initializers step:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156

Original change's description:
> [test] Move has unexpected output to outproc.
> 
> Expected outcomes optimized to serialize [PASS] as None.
> 
> Keeping expected outcomes inside output processors should be
> optimized in the future. Few possible optimizations:
> - separate classes for tests that are expected to PASS - done as
> an example in mozilla test suite.
> - cache output processors inside testcase.
> - share output processors between copies of the same test - needs
> some updates to the create_variant to update outproc only if
> expected outcomes changed.
> 
> Bug: v8:6917
> Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> Reviewed-on: https://chromium-review.googlesource.com/843025
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50347}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/849572
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50348}
2018-01-03 23:01:11 +00:00
Michal Majewski
71605b3ea4 [test] Move has unexpected output to outproc.
Expected outcomes optimized to serialize [PASS] as None.

Keeping expected outcomes inside output processors should be
optimized in the future. Few possible optimizations:
- separate classes for tests that are expected to PASS - done as
an example in mozilla test suite.
- cache output processors inside testcase.
- share output processors between copies of the same test - needs
some updates to the create_variant to update outproc only if
expected outcomes changed.

Bug: v8:6917
Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
Reviewed-on: https://chromium-review.googlesource.com/843025
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50347}
2018-01-03 22:19:49 +00:00
Michael Achenbach
e8912359b8 [test] Refactor commands into OS-specific versions
This seperates OS-specific code of the Command class into a Posix and a
WindowsCommand to simplify scattered OS checks.

This also removes some temporary mac debug output that's obsolete after
https://crbug.com/v8/6927 got resolved.

TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: Iaab3b527ce556dfba797a164ae58e8dd358eb56f
Reviewed-on: https://chromium-review.googlesource.com/847000
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50346}
2018-01-03 19:38:28 +00:00
Michael Achenbach
5a8e1f3f1b [test] Make coverage in test-runner tests work with multiprocessing
This replaces multiprocessing with threading in tests to make python
coverage work.

TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: Idff763dfefa4a7fc782133d94089b3a5b00a194d
Reviewed-on: https://chromium-review.googlesource.com/844735
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50344}
2018-01-03 17:37:05 +00:00
Michael Achenbach
13485a6991 [test] Add basic test-runner system tests
This tests some high-level scenarios of the V8 test runner, with
fake executable, test-suite extension and build configs.

The runners are slightly modified to be easier testable. Args
are passed from the tests now and the V8 root folder can be faked
by the tests.

We support coverage if python coverage 4.0 is installed. Otherwise
we run without it.

Bug: v8:6917
Change-Id: Ib149fd88027cbdc3382bcaea2d82020582f79d2d
Reviewed-on: https://chromium-review.googlesource.com/831506
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50310}
2017-12-27 18:03:49 +00:00
Michael Achenbach
460af93123 [test] Enable slow-path variant on a subset of bots
Prepared by:
https://crrev.com/c/798412

Bug: v8:7120
Change-Id: I28a00f175673209e6d5f662e30a5448236e7a7c9
Reviewed-on: https://chromium-review.googlesource.com/800612
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50287}
2017-12-21 22:05:09 +00:00
Michael Achenbach
efdbc429bb [test] Enable staging test runner changes on fyi bots
Bug: v8:6917
Change-Id: Ica9b590d194da305416826eeed9f49963b8d97ca
Reviewed-on: https://chromium-review.googlesource.com/839776
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50281}
2017-12-21 17:32:09 +00:00
Michal Majewski
a130438e13 [test] Removed old output parsing functions.
All get outcome logic moved out from testsuite which makes output
processors serializable. Output processors that are not global
are stored inside testcase to share them with all test copies.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I625466b2b00565d885556f96cab03d8d16fa7ff4
Reviewed-on: https://chromium-review.googlesource.com/837069
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50275}
2017-12-21 15:25:33 +00:00
Michal Majewski
848487ad61 [test] Output processor for message
Bug: v8:6917
Change-Id: I51ec0be7ea961d3c81e03675cc65d03ecf526c3d
Reviewed-on: https://chromium-review.googlesource.com/836988
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50268}
2017-12-21 14:38:28 +00:00
Michal Majewski
b0db2dc6ae [test] Output processors for inspector and webkit
Bug: v8:6917
Change-Id: I81bff2190766b3ccbc5da43ff2f3105c6c95da67
Reviewed-on: https://chromium-review.googlesource.com/836557
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50263}
2017-12-21 12:55:35 +00:00
Michal Majewski
d705e15857 [test] Prepare initial version of output processor
First CL moving parsing output to a separate class.
Following CLs will move (and deduplicate) code from
testcases by sharing common features like comparing
stdout to the expected output.

Bug: v8:6917
Change-Id: I1174a985949a0b946cefbf7e7c91e8815cbb4f15
Reviewed-on: https://chromium-review.googlesource.com/834130
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50228}
2017-12-20 11:24:07 +00:00
Michal Majewski
56b2b3be5e [test] Fix --report output.
Fix report mode and make it use testcase properties so statusfile
outcomes can be private.

Bug: v8:6917
Change-Id: Id38e89e0ba427c3bbb7ad12ba93e02beb7e46219
Reviewed-on: https://chromium-review.googlesource.com/833909
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50209}
2017-12-19 16:35:39 +00:00
Michal Majewski
5e6fd798f9 [test] Fixed target_name in json progress indicator
Pass shell name instead of an absolute path.

Bug: v8:796166
Change-Id: Ia9472e893fd2cb3fde2a94997f3e9daf30da06ea
Reviewed-on: https://chromium-review.googlesource.com/833917
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50207}
2017-12-19 15:23:54 +00:00
Michal Majewski
4695e97905 [test] Move getting outcomes to the statusfile
Bug: v8:6917
Change-Id: I175fa426546f2f3775a35f1094dfb19e06b2185d
Reviewed-on: https://chromium-review.googlesource.com/832394
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50203}
2017-12-19 14:35:16 +00:00
Michal Majewski
e0c4321479 [test] Prepare new API for statusfile parsing.
First step in moving all statusfile logic into statusfile.py.

Introduce StatusFile object that will be used for storing and managing
outcomes.

Bug: v8:6917
Change-Id: I024f9b1d029830345149422a08a8905e92545252
Reviewed-on: https://chromium-review.googlesource.com/832433
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50202}
2017-12-19 14:26:26 +00:00
Michal Majewski
c0173165ee [test] Update testsuite unittests.
Fix unittests since they were incompatible with the
new testcase and testsuite API.

Bug: v8:6917
Change-Id: I917bf58e21402e0b90bc91c0483ade0e7c90bdd6
Reviewed-on: https://chromium-review.googlesource.com/832392
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50201}
2017-12-19 14:13:56 +00:00
Michal Majewski
1f60466659 [test] Store outcomes in the testcase
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: Ia52d4bedbeff5b93915ef69a2dc78f6d92669061
Reviewed-on: https://chromium-review.googlesource.com/832467
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50200}
2017-12-19 14:10:06 +00:00
Michal Majewski
03165cb93b [test] Store expected outcomes in the testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I69d15801b79bf7aa846582367e9f3037b6612431
Reviewed-on: https://chromium-review.googlesource.com/829033
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50163}
2017-12-18 14:51:20 +00:00
Michael Achenbach
efb260cb47 [test] Fix wrapper order in verify-predictable mode
If a command prefix should be used in predictable mode, it should
wrap around d8 and not around the predictable wrapper.

NOTRY=true
TBR=majeski@google.com

Bug: v8:7166
Change-Id: I16f33090c647ea4f6f040c6194908df6500b82a3
Reviewed-on: https://chromium-review.googlesource.com/832206
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50160}
2017-12-18 13:41:50 +00:00
Michal Majewski
9ab1c76adc [test] Fix tests expected to fail count.
Bug: v8:6917
Change-Id: I45ee0277bd243559746f9bc9396d1a8900c891cc
Reviewed-on: https://chromium-review.googlesource.com/832026
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50159}
2017-12-18 13:24:40 +00:00
Michal Majewski
867c39f22c [test] Remove duration from testcase.
Bug: v8:6917
Change-Id: Ifc3dac7a82fa67dda1f8b166fbd3f76a123ffba2
Reviewed-on: https://chromium-review.googlesource.com/824365
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50122}
2017-12-15 08:10:40 +00:00
Michael Achenbach
efc4b9bac5 [test] Fix verify-predictable outcome check
This moves the error-code check for verify-predictable mode to the API
method checking for status file outcomes, overwriting the default
behavior.

This is resembling the behavior prior to:
https://chromium-review.googlesource.com/c/808971/

Otherwise, the status file outcomes will expect some negative tests
to fail in the mozilla test suite, which pass in predictable mode.

Now, negative tests are simply not supported.

Bug: v8:7166
Change-Id: I1d4bcaf66cb54c5fbb217dd9091b88ecc5b0e456
Reviewed-on: https://chromium-review.googlesource.com/817741
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50110}
2017-12-14 14:20:56 +00:00
Sergiy Byelozyorov
599324d03e [tools] Pass output to GetOutcome function
R=majeski@google.com
TBR=machenbach@chromium.org

This is breaking some test runs, e.g.
https://chromium-swarm.appspot.com/task?id=3a69c31ccb964d10&refresh=10&show_raw=1.

Bug: v8:6917
Change-Id: I61c12eafdf12a1bee2bf08de75f26b0d44fd0056
Reviewed-on: https://chromium-review.googlesource.com/825122
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50089}
2017-12-13 19:41:16 +00:00
Michal Majewski
8cd7617195 [test] Ignore rename coverage data in execution time.
Bug: v8:6917
Change-Id: I88fdb12c39311531600c0715fff5abe5da051295
Reviewed-on: https://chromium-review.googlesource.com/824271
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50079}
2017-12-13 15:04:51 +00:00
Michal Majewski
af192c8905 [test] Remove output from testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I5ca840db75766413659cb96fd28b922712040cec
Reviewed-on: https://chromium-review.googlesource.com/823843
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50078}
2017-12-13 15:00:52 +00:00
Michal Majewski
c8102945af [test] Code cleanup in testcfg.py
- All testcase/testsuite/variant generator subclasses renamed to
  just TestCase/TestSuite/VariantGenerator since they're private
  implementation.
- All `testcase` variables renamed to `test` to not conflict with
  a module name.
- No more two statements in the same line.
- Removed some unused testsuite methods.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I7710f3419f738a5f9ddca73765dd2cad2e35b952
Reviewed-on: https://chromium-review.googlesource.com/823964
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50076}
2017-12-13 13:36:20 +00:00
Michal Majewski
08a05c2bff [test] Move data access from suite to test.
- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
2017-12-13 09:14:47 +00:00
Michal Majewski
4f4654a013 [test] Deduplicate parsing source file flags.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I12d62e28b8e22820d4358d0166fa5db5e09b8bc3
Reviewed-on: https://chromium-review.googlesource.com/819630
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50033}
2017-12-12 13:12:18 +00:00
Michal Majewski
222076d927 [test] Return analysis exit code for no tests.
NOTRY=true

Bug: v8:6917
Change-Id: I5f1edec76f3e67909b7d910187ab2d0bc774d10b
Reviewed-on: https://chromium-review.googlesource.com/822490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50032}
2017-12-12 13:11:13 +00:00
Michal Majewski
ccd15eaf2c [test] Prepare test262 sources during init.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: Iad6b429cfd855c435528468396c880f7733776d9
Reviewed-on: https://chromium-review.googlesource.com/819551
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50024}
2017-12-12 11:39:57 +00:00
Michal Majewski
22eac2e987 [test] Code cleanup in test suites.
Some refactor moved from
https://chromium-review.googlesource.com/c/v8/v8/+/798331.

Bug: v8:6917
Change-Id: I8cae6cfca7a0d7d8e234052c0ab0bfe252355e60
Reviewed-on: https://chromium-review.googlesource.com/819550
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50020}
2017-12-12 08:54:52 +00:00
Michal Majewski
a91d043dd4 [test] Move GoogleTestSuite to a separate testcfg.
Since we have only unittests that are under GoogleTestSuite there
is no need to keep it as a default suite and we can make it
specific for unittests.

Bug: v8:6917
Change-Id: Ie2d57342773f228dea72184ab0f2abfc9d2daa70
Reviewed-on: https://chromium-review.googlesource.com/819253
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50011}
2017-12-11 20:57:07 +00:00
Michael Achenbach
325a56cdd1 [test] Move predictable testing into a python wrapper
This moves the verify-predictable logic from the test runner into
a python wrapper script.

This revealed two more tests that don't print allocations, which are
now skipped.

Bug: v8:7166, v8:7177
Change-Id: Ie4a541cb2a20900414ffe1caf4b3fccc4a5edb52
Reviewed-on: https://chromium-review.googlesource.com/808971
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49964}
2017-12-08 14:46:10 +00:00
Mircea Trofin
1048a6b4ea [wasm] Fix wasm-jit-to-native + W^X GC heap interaction issues
- in certain cases, we need both modification scopes because we may
mutate JS functions, even in the jit-to-native case - e.g. JS-to-wasm
wrappers
- added handling for wasm-to-wasm wrappers in the context of lazy
compilation.

Bug: v8:7105
Change-Id: I085c14e03ef0b08d040998f2207abf7bc3fff01c
Reviewed-on: https://chromium-review.googlesource.com/811285
Commit-Queue: Mircea Trofin <mtrofin@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49947}
2017-12-08 02:21:46 +00:00
Michael Starzinger
02d201bfdd Enable --write-protect-code-memory by default.
R=hablich@chromium.org
BUG=v8:6792

Change-Id: Ib3bb3f84f9acec37a37803f460f5c8cdc3669867
Reviewed-on: https://chromium-review.googlesource.com/792937
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49886}
2017-12-06 10:14:58 +00:00
Michal Majewski
453a9f1b95 [test] Create commands after sharding.
Bug: v8:6917
Change-Id: I3a79a7f273df566035cee6a0e22be2e02d67d946
Reviewed-on: https://chromium-review.googlesource.com/808505
Commit-Queue: Michał Majewski <majeski@google.com>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49865}
2017-12-05 13:48:42 +00:00
Michal Majewski
68c5d01ad6 Reland "Reland "[test] Creating command before execution phase.""
This is a reland of 3b06511052
Original change's description:
> Reland "[test] Creating command before execution phase."
> 
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> > 
> > Immutable command class with shell, flags and
> > environment.
> > 
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> > 
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
> 
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}

Bug: v8:6917
Change-Id: Ia39010a0a0f63537ad12490dfab17897d70d4930
Reviewed-on: https://chromium-review.googlesource.com/806034
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49830}
2017-12-04 13:40:29 +00:00
Michal Majewski
8a6c6f5880 Reland "Reland "[test] Creating command before execution phase.""
This is a reland of 3b06511052
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}

Bug: v8:6917
Change-Id: I981994224e493bee4c9435cb80772b6e2ad8fbb1
Reviewed-on: https://chromium-review.googlesource.com/805336
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49827}
2017-12-04 11:42:50 +00:00
Michael Achenbach
d3a8dec257 Revert "Reland "[test] Creating command before execution phase.""
This reverts commit 3b06511052.

Reason for revert: Broke fuzzers:
https://build.chromium.org/p/client.v8.clusterfuzz/builders/V8%20Deopt%20Fuzzer/builds/384

Original change's description:
> Reland "[test] Creating command before execution phase."
> 
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> > 
> > Immutable command class with shell, flags and
> > environment.
> > 
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> > 
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
> 
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: v8:6917
Change-Id: I4938642c4396366be1e13daf6998c4b8538b688b
Reviewed-on: https://chromium-review.googlesource.com/804254
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49805}
2017-12-01 20:44:50 +00:00
Michal Majewski
3b06511052 Reland "[test] Creating command before execution phase."
This is a reland of 98cc9e862f
Original change's description:
> [test] Creating command before execution phase.
> 
> Immutable command class with shell, flags and
> environment.
> 
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
> 
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}

Bug: v8:6917
Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
Reviewed-on: https://chromium-review.googlesource.com/800370
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49756}
2017-11-30 13:47:40 +00:00
Michael Achenbach
d8b369d2be Revert "[test] Creating command before execution phase."
This reverts commit 98cc9e862f.

Reason for revert: Breaks test isolation:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20builder/builds/29746

Original change's description:
> [test] Creating command before execution phase.
> 
> Immutable command class with shell, flags and
> environment.
> 
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
> 
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I44b99468d18fd093833f4185dad067a9eeaf2bc1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/800292
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49747}
2017-11-30 12:32:31 +00:00
Michal Majewski
98cc9e862f [test] Creating command before execution phase.
Immutable command class with shell, flags and
environment.

Command creation moved from worker to the main
process. Because of that there is no need to send
test cases beyond process boundaries and load test
suites in worker processes.

Bug: v8:6917
Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
Reviewed-on: https://chromium-review.googlesource.com/791710
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49746}
2017-11-30 12:30:09 +00:00
Michael Achenbach
df659b17ed [test] Refactor variant flags
This derives two global variables to ease adding new variants.

Bug: chromium:788104
Change-Id: Ibe933dc0999f6dfa0ae8bf2afa0a5de8e44aefca
Reviewed-on: https://chromium-review.googlesource.com/799790
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49741}
2017-11-30 10:41:57 +00:00
Michael Lippautz
c99c9c3e96 [testing] Add minor MC variant
No-try: true
Bug: v8:788113
Change-Id: I6e9c659cc5658f4da361552936551adb594f56d9
Reviewed-on: https://chromium-review.googlesource.com/799872
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49740}
2017-11-30 10:02:57 +00:00
Mircea Trofin
b03b1bd9a8 Revert "Revert "[wasm] JIT using WasmCodeManager""
This reverts commit b301203e5a.

Reason for revert: Fixed issues on arm.

Original change's description:
> Revert "[wasm] JIT using WasmCodeManager"
> 
> This reverts commit d4c8393c1c.
> 
> Reason for revert: Breaks ARM hardware:
> https://build.chromium.org/p/client.v8.ports/builders/V8%20Arm%20-%20debug/builds/5268
> 
> Original change's description:
> > [wasm] JIT using WasmCodeManager
> > 
> > This is the first step towards wasm code sharing. This CL moves wasm
> > code generation outside the JavaScript GC heap using the previously -
> > introduced WasmCodeManager (all this, behind the --wasm-jit-to-native
> > flag).
> > 
> > See design document: go/wasm-on-native-heap-stage-1
> > 
> > This CL doesn't change other wasm architectural invariants. We still
> > have per-Isolate wasm code generation, and per-wasm module instance
> > code specialization.
> > 
> > Bug:v8:6876
> > 
> > Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
> > Change-Id: I1e08cecad75f93fb081545c31228a4568be276d3
> > Reviewed-on: https://chromium-review.googlesource.com/674086
> > Reviewed-by: Ben Titzer <titzer@chromium.org>
> > Reviewed-by: Eric Holk <eholk@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49689}
> 
> TBR=bradnelson@chromium.org,titzer@chromium.org,mtrofin@chromium.org,eholk@chromium.org
> 
> Change-Id: I89af1ea5decd841bc12cd2ceaf74d32bc4433885
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6876
> Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
> Reviewed-on: https://chromium-review.googlesource.com/794690
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49691}

TBR=bradnelson@chromium.org,machenbach@chromium.org,titzer@chromium.org,mtrofin@chromium.org,eholk@chromium.org

Change-Id: I1b07638d1bb2ba0664305b4b2dcfc1342dc8444f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6876
Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/794434
Commit-Queue: Mircea Trofin <mtrofin@chromium.org>
Reviewed-by: Mircea Trofin <mtrofin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49692}
2017-11-28 22:27:13 +00:00
Michael Achenbach
b301203e5a Revert "[wasm] JIT using WasmCodeManager"
This reverts commit d4c8393c1c.

Reason for revert: Breaks ARM hardware:
https://build.chromium.org/p/client.v8.ports/builders/V8%20Arm%20-%20debug/builds/5268

Original change's description:
> [wasm] JIT using WasmCodeManager
> 
> This is the first step towards wasm code sharing. This CL moves wasm
> code generation outside the JavaScript GC heap using the previously -
> introduced WasmCodeManager (all this, behind the --wasm-jit-to-native
> flag).
> 
> See design document: go/wasm-on-native-heap-stage-1
> 
> This CL doesn't change other wasm architectural invariants. We still
> have per-Isolate wasm code generation, and per-wasm module instance
> code specialization.
> 
> Bug:v8:6876
> 
> Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
> Change-Id: I1e08cecad75f93fb081545c31228a4568be276d3
> Reviewed-on: https://chromium-review.googlesource.com/674086
> Reviewed-by: Ben Titzer <titzer@chromium.org>
> Reviewed-by: Eric Holk <eholk@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49689}

TBR=bradnelson@chromium.org,titzer@chromium.org,mtrofin@chromium.org,eholk@chromium.org

Change-Id: I89af1ea5decd841bc12cd2ceaf74d32bc4433885
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6876
Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/794690
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49691}
2017-11-28 21:58:44 +00:00
Mircea Trofin
d4c8393c1c [wasm] JIT using WasmCodeManager
This is the first step towards wasm code sharing. This CL moves wasm
code generation outside the JavaScript GC heap using the previously -
introduced WasmCodeManager (all this, behind the --wasm-jit-to-native
flag).

See design document: go/wasm-on-native-heap-stage-1

This CL doesn't change other wasm architectural invariants. We still
have per-Isolate wasm code generation, and per-wasm module instance
code specialization.

Bug:v8:6876

Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
Change-Id: I1e08cecad75f93fb081545c31228a4568be276d3
Reviewed-on: https://chromium-review.googlesource.com/674086
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49689}
2017-11-28 20:39:25 +00:00
Michael Achenbach
a2aff67f69 [test] Simplify flags for configuring testing variants.
This deprecates --exhaustive-variants and --no-variants flags to
simplify code configuring variants.

Simplification after:
https://crrev.com/c/789831

Bug: chromium:788104
Change-Id: Ie77d48eca083e7721e02d34bc1e9aa1b1f0d0202
Reviewed-on: https://chromium-review.googlesource.com/789836
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49633}
2017-11-27 13:10:26 +00:00
Michal Majewski
6056d32b38 [test] Extract expected outcomes. Simpler FAIL_* handling.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I864de452bacb9e34fa1bc70722bf4c2fa3de4204
Reviewed-on: https://chromium-review.googlesource.com/782723
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49614}
2017-11-24 09:23:03 +00:00
Michal Majewski
7a24b61387 [test] Move wildcards checking to statusfile presubmit.
Bug: v8:6917
Change-Id: Ia2ff836fc5b8bba42d9abe74c2387c26a63ad048
Reviewed-on: https://chromium-review.googlesource.com/782499
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49601}
2017-11-23 09:33:02 +00:00
Michal Majewski
491fc3e97e [test] Remove PASS_OR_FAIL and TIMEOUT statusfile outcomes.
Bug: v8:6917
Change-Id: I767e5b0fa2bfef73feb3e0bc568c00d22424afa8
Reviewed-on: https://chromium-review.googlesource.com/782560
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49578}
2017-11-22 14:17:58 +00:00
Michal Majewski
e06fd5703c [test] Refactor and add more comments to statusfile parsing.
Bug: v8:6917
Change-Id: I867f8c92454d5f10a5f22002361bf8cc053dc174
Reviewed-on: https://chromium-review.googlesource.com/782119
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49576}
2017-11-22 14:07:48 +00:00
Michal Majewski
0f2223c894 [test] Refactor getting test outcomes from statusfile.
Filtering by status file split to four parts:
1. Getting outcomes - reads both variant dependent and
independent outcomes, no more need to do it twice.
2. Checking unused rules - has a switch to check only variant
dependent/independent rules.
3. Reading flags - if outcome starts with '--' it is treated as a flag.
4. Actual filtering.

Outcomes removed from the testcase object, can be accessed
by call to its testsuite.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I35762f891010ddda926250452b88656047433daa
Reviewed-on: https://chromium-review.googlesource.com/775160
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49547}
2017-11-21 15:00:24 +00:00
Clemens Hammacher
412f1568f2 Reland "[test] Add Liftoff variant"
This is a reland of 712fa67554.

Original change's description:
> [test] Add Liftoff variant
>
> Add a variant for testing the current state of the Liftoff
> implementation.
> This variant will only run on a subset of the bots, just like the
> --future variant.
>
> R=machenbach@chromium.org, hablich@chromium.org
>
> Bug: v8:7088, v8:6600
> Change-Id: If49fad3a8ed579356504b821a787326754f24e78
> Reviewed-on: https://chromium-review.googlesource.com/779420
> Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49504}

TBR=machenbach@chromium.org
CC=hablich@chromium.org

Bug: v8:7088, v8:6600
Change-Id: Ieb20020f07c70acaa64bb421763a41aa163a261b
Reviewed-on: https://chromium-review.googlesource.com/781499
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49531}
2017-11-21 12:10:53 +00:00
Michael Achenbach
7e2033eb48 [test] Run the future variant on a subset of bots.
TBR=sergiyb@chromium.org

Bug: chromium:786938,v8:7106
Change-Id: Id60354be899187e4c09d53434ad70019c857f9e9
Reviewed-on: https://chromium-review.googlesource.com/779725
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49511}
2017-11-20 21:05:59 +00:00
Michael Achenbach
f93b14b0df [test] Prepare activating future variant on bots
TBR=sergiyb@chromium.org

Bug: v8:7106,chromium:786938
Change-Id: Iddd573013cb9322d1df06038f10f0e49a6497110
Reviewed-on: https://chromium-review.googlesource.com/779721
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49510}
2017-11-20 18:22:21 +00:00
Michael Achenbach
b5bdb762ac Revert "[test] Add Liftoff variant"
This reverts commit 712fa67554.

Reason for revert: Infra side change was reverted. Please reland
after it stays in for a bot cycle...

Original change's description:
> [test] Add Liftoff variant
> 
> Add a variant for testing the current state of the Liftoff
> implementation.
> This variant will only run on a subset of the bots, just like the
> --future variant.
> 
> R=​machenbach@chromium.org, hablich@chromium.org
> 
> Bug: v8:7088, v8:6600
> Change-Id: If49fad3a8ed579356504b821a787326754f24e78
> Reviewed-on: https://chromium-review.googlesource.com/779420
> Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49504}

TBR=machenbach@chromium.org,hablich@chromium.org,clemensh@chromium.org

Change-Id: Ib6b2e79cea5d9f99f8933c72bbb9d9dddbd6ae07
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7088, v8:6600
Reviewed-on: https://chromium-review.googlesource.com/779719
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49507}
2017-11-20 17:35:58 +00:00
Clemens Hammacher
712fa67554 [test] Add Liftoff variant
Add a variant for testing the current state of the Liftoff
implementation.
This variant will only run on a subset of the bots, just like the
--future variant.

R=machenbach@chromium.org, hablich@chromium.org

Bug: v8:7088, v8:6600
Change-Id: If49fad3a8ed579356504b821a787326754f24e78
Reviewed-on: https://chromium-review.googlesource.com/779420
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49504}
2017-11-20 17:30:51 +00:00
Michal Majewski
5bd4e2d893 [test] Change wildcards to prefix rules.
Test suite contract changes:
- support * only at the end of the rule.
- loading status file is mandatory before filtering by status file.

Bug: v8:6917
Change-Id: Ia345ebfa7827c50f13f20e5cb7489e62c53f3357
Reviewed-on: https://chromium-review.googlesource.com/779185
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49496}
2017-11-20 14:25:04 +00:00
Michal Majewski
657e726a0d [test] Common statusfile flag format for mozilla and test262.
Bug: v8:6917
Change-Id: Ida8594caead9119b7b5dad6209017e2eae9cd3aa
Reviewed-on: https://chromium-review.googlesource.com/776799
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49495}
2017-11-20 14:20:24 +00:00
Michael Achenbach
209d379083 [test] Enable future variant as part of the extra suite set
Bug: chromium:786938
Change-Id: Ib8041c3cfe2237922824d783ebf8f0bb4d967a53
Reviewed-on: https://chromium-review.googlesource.com/779259
Reviewed-by: Michael Hablich <hablich@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49493}
2017-11-20 13:43:34 +00:00
Michal Majewski
430e03b3c6 Remove shell info from testcase.
Bug: v8:6917
Change-Id: Ic50ed8aca2ef6b6e60eae194cf46c2264a416657
Reviewed-on: https://chromium-review.googlesource.com/774265
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49417}
2017-11-16 15:42:26 +00:00
Michal Majewski
0a12eaea45 Remove env field from testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I09fb05ac6d4b9b1223118494ce2c89e3ab5de109
Reviewed-on: https://chromium-review.googlesource.com/771870
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49415}
2017-11-16 15:26:06 +00:00
Michal Majewski
243eaf8b06 [test] Change stress compaction flag to boolean value.
Bug: v8:6972
Change-Id: I116d48f22045cf42cf2123297458640b551d37da
Reviewed-on: https://chromium-review.googlesource.com/768868
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49408}
2017-11-16 12:03:28 +00:00
Ross McIlroy
2f5d90a816 [d8] Add a stress-background-compile mode
Adds a stress-background-compile mode which runs compilation on a background
thread at the same time as compiling on the main thread to flush out races.
This is added as one of the variants, removing stress_asm_validation since this
is no longer a useful variant.

BUG=v8:5203

Change-Id: I73dd9f21d9b6da4a4cb39c05061aa24116e6c5ad
Reviewed-on: https://chromium-review.googlesource.com/741746
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49376}
2017-11-15 11:08:21 +00:00
Michal Majewski
d8c40807cb Deprecate distribution options in gc fuzzer.
Bug: v8:6917
Change-Id: I4fb315ddf9999a3e7e2306201e5ace69b4e0be4b
Reviewed-on: https://chromium-review.googlesource.com/768713
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49353}
2017-11-14 13:56:44 +00:00
Michal Majewski
a3f983a4ef Use new stress marking flag in gc fuzzer. Skip failed tests.
Bug: v8:6917
Change-Id: I41324b8bf1827c43a652fa5f6b83731eaa17776d
Reviewed-on: https://chromium-review.googlesource.com/766887
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49341}
2017-11-14 08:31:58 +00:00
Michal Majewski
392ab01d61 Introduce status file variable gc_fuzzer.
Bug: v8:6917
Change-Id: Ib983e3dfa8fac25fd494fb24e245bfd0caadd7fd
Reviewed-on: https://chromium-review.googlesource.com/766187
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49326}
2017-11-13 14:33:10 +00:00
Michal Majewski
5faaa2ebc1 Temporary GC fuzzer.
Temporary implementation of GC fuzzer based on the deopt fuzzer.
It will be merged with the deopt fuzzer to create more general numfuzzer.

Webkit test suite updated to ignore trace_incremental_marking flag output.

Bug: v8:6917
Change-Id: I053ae32f3cd3f74ba8b71fc5bdfc7264f54dca11
Reviewed-on: https://chromium-review.googlesource.com/758437
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49296}
2017-11-10 14:23:38 +00:00
Myles Borins
02c84d4192 [gyp] Fix string escaping for GYP on Windows
Fixes .gyp scaffolding for building on Windows.

Patch from Refael Ackermann <refack@gmail.com>.
5528afd073

Bug=v8:7061

Change-Id: I4faaf3f488b6725942746d74838ef7ce73b1e8d0
Reviewed-on: https://chromium-review.googlesource.com/761477
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49280}
2017-11-09 20:38:41 +00:00
Michal Majewski
616be20ebc Remove test runner's valgrind option.
Bug: v8:6917
Change-Id: Id13b0d84ee255d12ecaf4a30479e7c7173462d5c
Reviewed-on: https://chromium-review.googlesource.com/760360
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49274}
2017-11-09 15:40:51 +00:00
Michael Starzinger
2328e4d196 [test] Add testing of --write-protect-code-memory feature.
This adds test coverage for write-protected code pages by adding the
feature to the existing "stress_incremental_marking" variant. Since this
is considered a temporary state, the variant has not been renamed.

R=machenbach@chromium.org
BUG=v8:6792

Change-Id: I62fa5aa15bf5309b12fbc269fcf81069845755f9
Reviewed-on: https://chromium-review.googlesource.com/753388
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49267}
2017-11-09 12:25:35 +00:00
Michal Majewski
a9c908eb40 Remove test runner's network mode.
Bug: v8:6917
Change-Id: I7dae0715264cdf9f963f2454b101f6260d8493ff
Reviewed-on: https://chromium-review.googlesource.com/758837
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49256}
2017-11-09 09:21:05 +00:00
Michal Majewski
80cb266c75 Deprecate buildbot option and discover build directory automatically.
Bug: v8:6917
Change-Id: I0dc20f84257b501d217e00cb29b34dd2a985ecf9
Reviewed-on: https://chromium-review.googlesource.com/737834
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49189}
2017-11-07 13:18:53 +00:00
Michael Achenbach
67d543ec31 [test] Fix sanitizer options for OOM and leak detection
Bug: chromium:726584
Change-Id: I49daa3e7de4f792db9908885ab94a87661950a4e
Reviewed-on: https://chromium-review.googlesource.com/732654
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49186}
2017-11-07 12:59:25 +00:00
Michal Majewski
694a18781c Deprecated shell dir option.
Bug: v8:6917
Change-Id: Ia80de1d0430efb0c5f41a79267ba9146c5be42ce
Reviewed-on: https://chromium-review.googlesource.com/751462
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49089}
2017-11-02 19:03:15 +00:00
Michal Majewski
e053582a55 Removed shell option from testrunner.
Bug: v8:6917
Change-Id: I95355d445ae08a500a3210314dc893dfaddf209d
Reviewed-on: https://chromium-review.googlesource.com/750808
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49085}
2017-11-02 18:09:30 +00:00
Michal Majewski
fd1fa35721 Reland "Setting up the environment for testrunner refactored"
This is a reland of 697b2c09a5
Original change's description:
> Setting up the environment for testrunner refactored
> 
> Bug: v8:6917
> Change-Id: Ibc3c738ef807d37d8b76f440d9765c4d0405c021
> Reviewed-on: https://chromium-review.googlesource.com/735421
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48923}

Bug: v8:6917
Change-Id: I07918814ee8277eca96bcb2d330e6b08f6a0505d
Reviewed-on: https://chromium-review.googlesource.com/741621
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49078}
2017-11-02 13:55:25 +00:00
Michael Achenbach
0a35d6f985 [test] Let flags from test case have precedence
The status-file flags and the flags from the test case's source code
must always overwrite extra flags set by bots.

Bug: v8:6924
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I0e2aabb69da7cfb8ba6c1c79bd3851462071a6ac
Reviewed-on: https://chromium-review.googlesource.com/732656
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49001}
2017-10-27 13:22:39 +00:00
Michal Majewski
0b9acc20e2 Reland "Reland "Reuse arch/mode discovery in deopt fuzzer""
This is a reland of 5442e8b224
Original change's description:
> Reland "Reuse arch/mode discovery in deopt fuzzer"
> 
> This is a reland of a24c7c9a52
> Original change's description:
> > Reuse arch/mode discovery in deopt fuzzer
> > 
> > Bug: v8:6917
> > Change-Id: I1b7169c8702c8649812b17579d38d64de676ed60
> > Reviewed-on: https://chromium-review.googlesource.com/723420
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#48838}
> 
> Bug: v8:6917
> Change-Id: I03b2c288257d44c2df9d0fa6cf4750d1c5719d59
> Reviewed-on: https://chromium-review.googlesource.com/735719
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48916}

Bug: v8:6917
Change-Id: I81419466d359b786c2a1b3a9aad3a86ed136f6f0
Reviewed-on: https://chromium-review.googlesource.com/738252
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#48997}
2017-10-27 09:53:51 +00:00
Michał Majewski
5b8702548f Revert "Reland "Reuse arch/mode discovery in deopt fuzzer""
This reverts commit 5442e8b224.

Reason for revert: https://crbug.com/v8/7006

Original change's description:
> Reland "Reuse arch/mode discovery in deopt fuzzer"
> 
> This is a reland of a24c7c9a52
> Original change's description:
> > Reuse arch/mode discovery in deopt fuzzer
> > 
> > Bug: v8:6917
> > Change-Id: I1b7169c8702c8649812b17579d38d64de676ed60
> > Reviewed-on: https://chromium-review.googlesource.com/723420
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#48838}
> 
> Bug: v8:6917
> Change-Id: I03b2c288257d44c2df9d0fa6cf4750d1c5719d59
> Reviewed-on: https://chromium-review.googlesource.com/735719
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48916}

TBR=machenbach@chromium.org,majeski@google.com

Change-Id: Ibbc16bb7115c8e8b93bd8f39065742d3c2d7739f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917, v8:7006
Reviewed-on: https://chromium-review.googlesource.com/737807
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48960}
2017-10-26 09:51:18 +00:00
Michał Majewski
0ff84265c6 Revert "Setting up the environment for testrunner refactored"
This reverts commit 697b2c09a5.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> Setting up the environment for testrunner refactored
> 
> Bug: v8:6917
> Change-Id: Ibc3c738ef807d37d8b76f440d9765c4d0405c021
> Reviewed-on: https://chromium-review.googlesource.com/735421
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48923}

TBR=machenbach@chromium.org,majeski@google.com

Change-Id: Iafb29f1f13decc6e6ed00ed6e815008fcde1dc44
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/737635
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48959}
2017-10-26 09:47:38 +00:00
Michal Majewski
697b2c09a5 Setting up the environment for testrunner refactored
Bug: v8:6917
Change-Id: Ibc3c738ef807d37d8b76f440d9765c4d0405c021
Reviewed-on: https://chromium-review.googlesource.com/735421
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48923}
2017-10-25 12:13:45 +00:00
Michal Majewski
5442e8b224 Reland "Reuse arch/mode discovery in deopt fuzzer"
This is a reland of a24c7c9a52
Original change's description:
> Reuse arch/mode discovery in deopt fuzzer
> 
> Bug: v8:6917
> Change-Id: I1b7169c8702c8649812b17579d38d64de676ed60
> Reviewed-on: https://chromium-review.googlesource.com/723420
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48838}

Bug: v8:6917
Change-Id: I03b2c288257d44c2df9d0fa6cf4750d1c5719d59
Reviewed-on: https://chromium-review.googlesource.com/735719
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48916}
2017-10-25 10:53:16 +00:00
Michael Achenbach
0e272905c6 Revert "Reuse arch/mode discovery in deopt fuzzer"
This reverts commit a24c7c9a52.

Reason for revert: Breaks some users' workflow where
--arch and --mode are used to determine the build path.

Original change's description:
> Reuse arch/mode discovery in deopt fuzzer
> 
> Bug: v8:6917
> Change-Id: I1b7169c8702c8649812b17579d38d64de676ed60
> Reviewed-on: https://chromium-review.googlesource.com/723420
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48838}

TBR=machenbach@chromium.org,majeski@google.com

Change-Id: If1f6e67eee17f11ec7547943ce084d18242f6875
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/735442
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48869}
2017-10-24 11:24:57 +00:00
Michael Achenbach
39e1299094 Revert "Deprecated option --shell removed"
This reverts commit f681e53042.

Reason for revert: Need to revert:
https://chromium-review.googlesource.com/c/v8/v8/+/723420

Original change's description:
> Deprecated option --shell removed
> 
> Bug: v8:6917
> Change-Id: I4262bc98019a3af0fc2bad7ef12b29ba5d4ea5ea
> Reviewed-on: https://chromium-review.googlesource.com/734241
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#48855}

TBR=machenbach@chromium.org,majeski@google.com

Change-Id: I8a0065883d41f5011934cdf04263e76801f1910e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/735150
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48868}
2017-10-24 11:23:37 +00:00
Michal Majewski
f681e53042 Deprecated option --shell removed
Bug: v8:6917
Change-Id: I4262bc98019a3af0fc2bad7ef12b29ba5d4ea5ea
Reviewed-on: https://chromium-review.googlesource.com/734241
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#48855}
2017-10-24 07:40:02 +00:00
Michal Majewski
a24c7c9a52 Reuse arch/mode discovery in deopt fuzzer
Bug: v8:6917
Change-Id: I1b7169c8702c8649812b17579d38d64de676ed60
Reviewed-on: https://chromium-review.googlesource.com/723420
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48838}
2017-10-23 16:12:37 +00:00
Michael Achenbach
efe438c5ed [test] Fix isolate for auto-detect in gyp
TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: I96cedba176cd3c35ca39bb2d4604956cded114fc
Reviewed-on: https://chromium-review.googlesource.com/728023
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48740}
2017-10-19 11:52:11 +00:00
Michael Achenbach
cdf5f2b0d0 [test] Fix auto-detect cfi
TBR=sergiyb@chromium.org
NOTRY=true

Bug: v8:6917
Change-Id: Ia417ab5cf59a6c2894de67d5519394629b6ab1b9
Reviewed-on: https://chromium-review.googlesource.com/727936
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48735}
2017-10-19 10:27:31 +00:00
Michael Achenbach
8c98e07c82 [test] Auto-detect verify-predictable builds
Bug: v8:6917
Change-Id: Ia768c9aaf71e70d1376ae21a35fd539a7315b0cd
Reviewed-on: https://chromium-review.googlesource.com/725802
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48717}
2017-10-19 07:19:19 +00:00
Michael Achenbach
96c5e2df06 [test] Enable auto-detection of test flags in gyp
This ports the build_config json from GN to GYP to prepare deprecating
tedious flags passing to the test runner.

This also removes two unused GN flags that only hold temporary values.

Bug: v8:6917
Change-Id: I976185f1541277dc5c9bfbaa7578f35c19dd254c
Reviewed-on: https://chromium-review.googlesource.com/725706
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48715}
2017-10-19 07:02:38 +00:00
Michal Majewski
f5c6daf186 Deprecate multiple arch/mode options in the testrunner
Bug: v8:6917
Change-Id: I45a3a683388341d61989473ebd0b761a1ad8a9e3
Reviewed-on: https://chromium-review.googlesource.com/721659
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48676}
2017-10-18 10:58:36 +00:00
Michael Achenbach
49bdec3470 Revert "Revert "[test] Fix win-asan symbolizer path""
This reverts commit 4054cf278f.

Reason for revert: Just exposes existing issues.

Original change's description:
> Revert "[test] Fix win-asan symbolizer path"
> 
> This reverts commit 135576ffb6.
> 
> Reason for revert: V8 Win32 ASAN failures: https://build.chromium.org/p/client.v8/builders/V8%20Win32%20ASAN/builds/73
> 
> It appears these failures were lurking there already, but were hidden because of the bug this CL fixed. Opened https://crbug.com/v8/6953 about these issues.
> 
> Original change's description:
> > [test] Fix win-asan symbolizer path
> > 
> > This makes the symbolizer path relative, as the absolute paths contain
> > a drive letter + colon on windows. The colon is confused by the
> > sanitizer as an option separator.
> > 
> > The test driver changes the cwd to the V8 root dir in each
> > invocation.
> > 
> > Bug: chromium:726584
> > Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
> > Reviewed-on: https://chromium-review.googlesource.com/721124
> > Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#48652}
> 
> TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,etienneb@chromium.org
> 
> Change-Id: Ic78527950f6a239a03658e042d7244c9781d05db
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: chromium:726584
> Reviewed-on: https://chromium-review.googlesource.com/723825
> Reviewed-by: Eric Holk <eholk@chromium.org>
> Commit-Queue: Eric Holk <eholk@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48653}

TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,eholk@chromium.org,sergiyb@chromium.org,etienneb@chromium.org

Change-Id: I8ea3b1d74ece09bed4758522f51cbee56a7792e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:726584
Reviewed-on: https://chromium-review.googlesource.com/725319
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48662}
2017-10-18 06:54:46 +00:00
Eric Holk
4054cf278f Revert "[test] Fix win-asan symbolizer path"
This reverts commit 135576ffb6.

Reason for revert: V8 Win32 ASAN failures: https://build.chromium.org/p/client.v8/builders/V8%20Win32%20ASAN/builds/73

It appears these failures were lurking there already, but were hidden because of the bug this CL fixed. Opened https://crbug.com/v8/6953 about these issues.

Original change's description:
> [test] Fix win-asan symbolizer path
> 
> This makes the symbolizer path relative, as the absolute paths contain
> a drive letter + colon on windows. The colon is confused by the
> sanitizer as an option separator.
> 
> The test driver changes the cwd to the V8 root dir in each
> invocation.
> 
> Bug: chromium:726584
> Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
> Reviewed-on: https://chromium-review.googlesource.com/721124
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48652}

TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,etienneb@chromium.org

Change-Id: Ic78527950f6a239a03658e042d7244c9781d05db
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:726584
Reviewed-on: https://chromium-review.googlesource.com/723825
Reviewed-by: Eric Holk <eholk@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48653}
2017-10-17 19:42:12 +00:00
Michael Achenbach
135576ffb6 [test] Fix win-asan symbolizer path
This makes the symbolizer path relative, as the absolute paths contain
a drive letter + colon on windows. The colon is confused by the
sanitizer as an option separator.

The test driver changes the cwd to the V8 root dir in each
invocation.

Bug: chromium:726584
Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
Reviewed-on: https://chromium-review.googlesource.com/721124
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48652}
2017-10-17 18:01:14 +00:00
Ben Smith
f7fbc8ed46 Fix test runner with "--arch-and-mode" flag
Change-Id: I77b6eb5e949f8b37fa76d2a161d1ec2c1117ab04
Reviewed-on: https://chromium-review.googlesource.com/722078
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Ben Smith <binji@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48616}
2017-10-16 23:00:41 +00:00
Michal Majewski
8e23c4b7f6 Build options moved to the BaseTestRunner.
Bug: v8:6917
Change-Id: Ife79f1d3c60f591c00bb1ecd340a9f1b31b02039
Reviewed-on: https://chromium-review.googlesource.com/720808
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#48598}
2017-10-16 14:35:41 +00:00
Michael Achenbach
089dd7d244 [test] Add output for investigating win-asan failures
Also fix asan environment for testing on windows.

TBR=sergiyb@chromium.org

Bug: chromium:726584
Change-Id: Ic9e6afa714f4757ad1b0f2ebfa742e742e1c04b9
Reviewed-on: https://chromium-review.googlesource.com/720811
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48583}
2017-10-16 10:30:27 +00:00
Michael Achenbach
bc70017d21 [test] Print processes in test-driver on mac for investigation
TBR=ulan@chromium.org

Bug: v8:6927
Change-Id: I6e76cc3e967b9136ed50612afb9159a404ccbfad
Reviewed-on: https://chromium-review.googlesource.com/718599
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48538}
2017-10-13 12:17:06 +00:00
Michal Majewski
d5be2832e3 TestRunner initial refactor
Preparation for sharing more code between deopt fuzzer and normal test runner.

Bug: v8:6917
Change-Id: Id8022c7597544ffddf5d1b0a0d73bb4a9d163fc3
Reviewed-on: https://chromium-review.googlesource.com/715417
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48521}
2017-10-13 08:52:28 +00:00
Michael Achenbach
c1f7694b5d [test] Improve printing for investigating hung tests
NOTRY=true
TBR=ulan@chromium.org

Bug: v8:6927
Change-Id: I84e239c94ed6b545544bc3db30689b8ff1b158ae
Reviewed-on: https://chromium-review.googlesource.com/718098
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48519}
2017-10-13 08:24:36 +00:00