Commit Graph

48 Commits

Author SHA1 Message Date
Liviu Rau
b3477fdd01 [test] Refactor testrunner (4)
- Removed duplication and unnecessary indirection from all suites testcfgs.
 - Introduced a more comprehensive context to cover both command context and other platform specific concerns.
 - Propagated above context to TestLoader to allow for test counting command execution on all platforms.
 - Wrapped original pool with another class to give it a new interface and allow injecting different implementations in the future.
 - Consolidated progress indicators under a single processor in the pipeline.
 - Consolidated result retention requirements calculation outside of pipeline chain.
 - Refactored LoaderProc and got it under tests.
 - Added some more tests for the standard runner.
 - Extracted BuildConfig class.


Bug: v8:12785
Change-Id: I87be040e91f792a983662bb5a10d55b36a14ea7f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3701595
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/main@{#81770}
2022-07-18 09:52:24 +00:00
Simon Zünd
29c4cf0e3c [inspector] Move restart-frame test helpers into protocol-test.js
R=bmeurer@chromium.org

Bug: chromium:1303521
Change-Id: Iff7247fda94037ff4f9d37f334d386eb4e63ce62
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3595819
Commit-Queue: Simon Zünd <szuend@chromium.org>
Auto-Submit: Simon Zünd <szuend@chromium.org>
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
Commit-Queue: Benedikt Meurer <bmeurer@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80038}
2022-04-20 08:02:34 +00:00
Simon Zünd
ad0677129b [inspector] Add disabled tests for 'Restart frame' 1/2
Doc: https://bit.ly/revive-restart-frame
Context: https://crrev.com/c/3582395 (whole feature)

This CL adds the first batch of inspector tests for the upcoming
"Restart frame" feature. Landing the tests upfront allows us to
better discuss the proposed API as well as think early about
corner cases we should test.

The tests check for the functionality of `Debugger.restartFrame`, as
well as the newly added parameter `canBeRestarted` in
the `Debugger.paused` event.

Bug: chromium:1303521
Change-Id: Ibda6d8b6110fce893e0844f8902fbd5d901ae01d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3585946
Reviewed-by: Jaroslav Sevcik <jarin@chromium.org>
Reviewed-by: Kim-Anh Tran <kimanh@chromium.org>
Commit-Queue: Simon Zünd <szuend@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80013}
2022-04-19 09:00:17 +00:00
Tamer Tas
82e29d8af8 [test] Load missing file to Android devices for inspector test suite
R=rmcilroy@chromium.org,machenbach@chromium.org,jkummerow@chromium.org

Bug: chromium:1099623
Change-Id: I7f79549a5de171dd2d85c86b3c4114f8aa2250b2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2308344
Auto-Submit: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68967}
2020-07-21 14:11:11 +00:00
Kim-Anh Tran
0ad867db75 [wasm][debug] Clean up inspector tests
Pulling out common functionality related to dumping scope properties.

Bug: chromium:1093165
Change-Id: I7de377b8812b6181bac21fc0d90c416568b0d640
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2237126
Commit-Queue: Kim-Anh Tran <kimanh@chromium.org>
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68277}
2020-06-10 07:28:10 +00:00
Z Nguyen-Huu
0ecbda1408 Introduce regenerate-expected-files for run-test
Example can be inspector tests.

Bug: v8:10264
Change-Id: I996bb68d0f36920568a04f93cd8c1256a4f41a96
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2070912
Commit-Queue: Z Nguyen-Huu <duongn@microsoft.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66572}
2020-03-03 18:58:24 +00:00
Tamer Tas
df630e67fd Reland "Reland "[test] refactor testsuite configuration""
This is a reland of 81eec150f6

Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}

Bug: v8:8174, v8:8790
Change-Id: I38ab9d37bca76057441a970f26e2102e4387a857
Reviewed-on: https://chromium-review.googlesource.com/c/1454724
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59387}
2019-02-06 09:02:09 +00:00
Tamer Tas
ea4412ad33 Revert "Reland "[test] refactor testsuite configuration""
This reverts commit 81eec150f6.

Reason for revert: windows mozilla test failures

Original change's description:
> Reland "[test] refactor testsuite configuration"
> 
> This is a reland of 7f92ad0ab6
> 
> Original change's description:
> > [test] refactor testsuite configuration
> > 
> > Every testsuite configuration consist of at least 30% code duplication.
> > 
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> > 
> > This CL removes all the duplication by refactoring the common functionality.
> > 
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> > 
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> > 
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
> 
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org

Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
2019-02-05 15:20:46 +00:00
Tamer Tas
81eec150f6 Reland "[test] refactor testsuite configuration"
This is a reland of 7f92ad0ab6

Original change's description:
> [test] refactor testsuite configuration
> 
> Every testsuite configuration consist of at least 30% code duplication.
> 
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
> 
> This CL removes all the duplication by refactoring the common functionality.
> 
> This CL contains structural changes without any logical changes % small bug
> fixes.
> 
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
2019-02-05 14:53:05 +00:00
Tamer Tas
97068800fe Revert "[test] refactor testsuite configuration"
This reverts commit 7f92ad0ab6.

Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148

Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=​machenbach@chromium.org
> CC=​yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}

TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org

Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
2019-02-05 12:06:18 +00:00
Tamer Tas
7f92ad0ab6 [test] refactor testsuite configuration
Every testsuite configuration consist of at least 30% code duplication.

The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.

This CL removes all the duplication by refactoring the common functionality.

This CL contains structural changes without any logical changes % small bug
fixes.

R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
2019-02-05 11:37:51 +00:00
Alexey Kozyatinskiy
ebd070ec2f reland: inspector: move injected script source to native
- introduced ValueMirror interface, this interface contains methods to generate
  different protocol entities,
- introduced DebugPropertyIterator, this iterator iterates through object properties
  in the following order: exotic indices, enumerable strings, all other properties,
- removed all injected script infra, e.g. closure compiler,

R=dgozman@chromium.org
TBR=yangguo@chromium.org

Bug: chromium:595206
Change-Id: I030fdb3a80074ca6edd4749f86b39b590776ae6f
Reviewed-on: https://chromium-review.googlesource.com/c/1310056
Reviewed-by: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
Commit-Queue: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57181}
2018-10-31 17:47:58 +00:00
Aleksey Kozyatinskiy
fc5c8d7f78 Revert "inspector: move injected script source to native"
This reverts commit 7e079c660b.

Reason for revert: native implementation should be ready for navigation.

Original change's description:
> inspector: move injected script source to native
> 
> - introduced ValueMirror interface, this interface contains methods to generate
>   different protocol entities,
> - introduced DebugPropertyIterator, this iterator iterates through object properties
>   in the following order: exotic indices, enumerable strings, all other properties,
> - removed all injected script infra, e.g. closure compiler,
> 
> R=​dgozman@chromium.org
> TBR=yangguo@chromium.org
> 
> Bug: chromium:595206
> Change-Id: Idcfc04489ee52e015ad1d1d191c3474cc65e63f2
> Reviewed-on: https://chromium-review.googlesource.com/c/1308353
> Reviewed-by: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
> Commit-Queue: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57150}

TBR=dgozman@chromium.org,yangguo@chromium.org,kozyatinskiy@chromium.org

Change-Id: I8c5c61f4cfe5a66cd33eadd02ab4acec539cc3bb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:595206
Reviewed-on: https://chromium-review.googlesource.com/c/1310055
Reviewed-by: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
Commit-Queue: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57176}
2018-10-31 15:58:30 +00:00
Alexey Kozyatinskiy
7e079c660b inspector: move injected script source to native
- introduced ValueMirror interface, this interface contains methods to generate
  different protocol entities,
- introduced DebugPropertyIterator, this iterator iterates through object properties
  in the following order: exotic indices, enumerable strings, all other properties,
- removed all injected script infra, e.g. closure compiler,

R=dgozman@chromium.org
TBR=yangguo@chromium.org

Bug: chromium:595206
Change-Id: Idcfc04489ee52e015ad1d1d191c3474cc65e63f2
Reviewed-on: https://chromium-review.googlesource.com/c/1308353
Reviewed-by: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
Commit-Queue: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57150}
2018-10-30 21:30:55 +00:00
Clemens Hammacher
192bee6bac Revert "inspector: move injected script source to native"
This reverts commit 34686abe40.

Reason for revert: Compile errors on several bots, e.g. https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux%20-%20debug%20builder/33299

Original change's description:
> inspector: move injected script source to native
> 
> - introduced ValueMirror interface, this interface contains methods to generate
>   different protocol entities,
> - introduced DebugPropertyIterator, this iterator iterates through object properties
>   in the following order: exotic indices, enumerable strings, all other properties,
> - removed all injected script infra, e.g. closure compiler,
> 
> R=​dgozman@chromium.org
> TBR=yangguo@chromium.org
> 
> Bug: chromium:595206
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_headless_rel;luci.chromium.try:linux_chromium_rel_ng;master.tryserver.blink:linux_trusty_blink_rel
> Change-Id: I077c1879622aa0d9900d719b80d2ef5ba4221a22
> Reviewed-on: https://chromium-review.googlesource.com/c/1295550
> Commit-Queue: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
> Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57142}

TBR=dgozman@chromium.org,yangguo@chromium.org,kozyatinskiy@chromium.org

Change-Id: I6e4ccaf1d6b151fbc0ffe4f26daa584433321c77
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:595206
Cq-Include-Trybots: luci.chromium.try:linux_chromium_headless_rel;luci.chromium.try:linux_chromium_rel_ng;master.tryserver.blink:linux_trusty_blink_rel
Reviewed-on: https://chromium-review.googlesource.com/c/1307432
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57144}
2018-10-30 17:04:54 +00:00
Alexey Kozyatinskiy
34686abe40 inspector: move injected script source to native
- introduced ValueMirror interface, this interface contains methods to generate
  different protocol entities,
- introduced DebugPropertyIterator, this iterator iterates through object properties
  in the following order: exotic indices, enumerable strings, all other properties,
- removed all injected script infra, e.g. closure compiler,

R=dgozman@chromium.org
TBR=yangguo@chromium.org

Bug: chromium:595206
Cq-Include-Trybots: luci.chromium.try:linux_chromium_headless_rel;luci.chromium.try:linux_chromium_rel_ng;master.tryserver.blink:linux_trusty_blink_rel
Change-Id: I077c1879622aa0d9900d719b80d2ef5ba4221a22
Reviewed-on: https://chromium-review.googlesource.com/c/1295550
Commit-Queue: Aleksey Kozyatinskiy <kozyatinskiy@chromium.org>
Reviewed-by: Dmitry Gozman <dgozman@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57142}
2018-10-30 16:43:55 +00:00
Michael Achenbach
0d1b00b8c2 [test] Add non-d8 test suites to Android testing
This enables cctest, unittests, fuzzer and inspector on Android.
The cctest suite requires extra resource-fetching logic for the
bytecode-generator expectation files.

Bug: chromium:866862
Cq-Include-Trybots: luci.v8.try:v8_android_arm64_n5x_rel_ng
Change-Id: If3da853a62c047388476a7f38e32e64e2859f186
Reviewed-on: https://chromium-review.googlesource.com/1213208
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55784}
2018-09-11 11:12:21 +00:00
Michal Majewski
804cc9f385 [test] Remove context usage from test processors
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Id5a7b41502dfd4be43496b1edb958522a8eb0c31
Reviewed-on: https://chromium-review.googlesource.com/895588
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51023}
2018-02-01 10:35:02 +00:00
Michal Majewski
d3083b46ee [test] Move options to test config
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I254d2e545709029346f585b02a9edf91d3f27893
Reviewed-on: https://chromium-review.googlesource.com/893321
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50996}
2018-01-31 13:46:29 +00:00
Michal Majewski
5ede3cb533 Reland "[test] Random seed processor"
This is a reland of 0db74d4974.

Original change's description:
> [test] Random seed processor
> 
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
> 
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}

Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
2018-01-31 09:53:34 +00:00
Michael Achenbach
ba30988cbc Revert "[test] Random seed processor"
This reverts commit 0db74d4974.

Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1

Original change's description:
> [test] Random seed processor
> 
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
> 
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
2018-01-30 22:05:03 +00:00
Michal Majewski
0db74d4974 [test] Random seed processor
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.

Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
2018-01-30 15:50:49 +00:00
Michal Majewski
9f7d440e98 [test] Check output on the worker process.
I added additional exception logging in the execution.py since
errors in processing results were really difficult to debug.

There is a problem on Windows with class serialization when
it comes from dynamically loaded module. To fix it I moved all
output processors to the tools/testrunner/outproc/ and import
them in test/*/testcfg.py.

Bug: v8:6917
Change-Id: Ida604641d659b006e91faf1d56a37769ec47f5f3
Reviewed-on: https://chromium-review.googlesource.com/842784
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50361}
2018-01-04 14:12:31 +00:00
Michael Achenbach
c1e9bc02f8 Revert "Revert "[test] Move has unexpected output to outproc.""
This reverts commit 1685b5d27a.

Reason for revert: Was probably caused by infra change:
https://crrev.com/c/845781

Original change's description:
> Revert "[test] Move has unexpected output to outproc."
> 
> This reverts commit 71605b3ea4.
> 
> Reason for revert: Seems to break static-initializers step:
> https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
> 
> Original change's description:
> > [test] Move has unexpected output to outproc.
> > 
> > Expected outcomes optimized to serialize [PASS] as None.
> > 
> > Keeping expected outcomes inside output processors should be
> > optimized in the future. Few possible optimizations:
> > - separate classes for tests that are expected to PASS - done as
> > an example in mozilla test suite.
> > - cache output processors inside testcase.
> > - share output processors between copies of the same test - needs
> > some updates to the create_variant to update outproc only if
> > expected outcomes changed.
> > 
> > Bug: v8:6917
> > Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> > Reviewed-on: https://chromium-review.googlesource.com/843025
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#50347}
> 
> TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
> 
> Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6917
> Reviewed-on: https://chromium-review.googlesource.com/849572
> Reviewed-by: Bill Budge <bbudge@chromium.org>
> Commit-Queue: Bill Budge <bbudge@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50348}

TBR=bbudge@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I7a522b6487de6e96985d223524533493eb9171f0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/848975
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50350}
2018-01-04 07:39:21 +00:00
Bill Budge
1685b5d27a Revert "[test] Move has unexpected output to outproc."
This reverts commit 71605b3ea4.

Reason for revert: Seems to break static-initializers step:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156

Original change's description:
> [test] Move has unexpected output to outproc.
> 
> Expected outcomes optimized to serialize [PASS] as None.
> 
> Keeping expected outcomes inside output processors should be
> optimized in the future. Few possible optimizations:
> - separate classes for tests that are expected to PASS - done as
> an example in mozilla test suite.
> - cache output processors inside testcase.
> - share output processors between copies of the same test - needs
> some updates to the create_variant to update outproc only if
> expected outcomes changed.
> 
> Bug: v8:6917
> Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> Reviewed-on: https://chromium-review.googlesource.com/843025
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50347}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/849572
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50348}
2018-01-03 23:01:11 +00:00
Michal Majewski
71605b3ea4 [test] Move has unexpected output to outproc.
Expected outcomes optimized to serialize [PASS] as None.

Keeping expected outcomes inside output processors should be
optimized in the future. Few possible optimizations:
- separate classes for tests that are expected to PASS - done as
an example in mozilla test suite.
- cache output processors inside testcase.
- share output processors between copies of the same test - needs
some updates to the create_variant to update outproc only if
expected outcomes changed.

Bug: v8:6917
Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
Reviewed-on: https://chromium-review.googlesource.com/843025
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50347}
2018-01-03 22:19:49 +00:00
Michal Majewski
a130438e13 [test] Removed old output parsing functions.
All get outcome logic moved out from testsuite which makes output
processors serializable. Output processors that are not global
are stored inside testcase to share them with all test copies.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I625466b2b00565d885556f96cab03d8d16fa7ff4
Reviewed-on: https://chromium-review.googlesource.com/837069
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50275}
2017-12-21 15:25:33 +00:00
Michal Majewski
b0db2dc6ae [test] Output processors for inspector and webkit
Bug: v8:6917
Change-Id: I81bff2190766b3ccbc5da43ff2f3105c6c95da67
Reviewed-on: https://chromium-review.googlesource.com/836557
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50263}
2017-12-21 12:55:35 +00:00
Michal Majewski
5e6fd798f9 [test] Fixed target_name in json progress indicator
Pass shell name instead of an absolute path.

Bug: v8:796166
Change-Id: Ia9472e893fd2cb3fde2a94997f3e9daf30da06ea
Reviewed-on: https://chromium-review.googlesource.com/833917
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50207}
2017-12-19 15:23:54 +00:00
Michal Majewski
03165cb93b [test] Store expected outcomes in the testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I69d15801b79bf7aa846582367e9f3037b6612431
Reviewed-on: https://chromium-review.googlesource.com/829033
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50163}
2017-12-18 14:51:20 +00:00
Michal Majewski
af192c8905 [test] Remove output from testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I5ca840db75766413659cb96fd28b922712040cec
Reviewed-on: https://chromium-review.googlesource.com/823843
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50078}
2017-12-13 15:00:52 +00:00
Michal Majewski
c8102945af [test] Code cleanup in testcfg.py
- All testcase/testsuite/variant generator subclasses renamed to
  just TestCase/TestSuite/VariantGenerator since they're private
  implementation.
- All `testcase` variables renamed to `test` to not conflict with
  a module name.
- No more two statements in the same line.
- Removed some unused testsuite methods.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I7710f3419f738a5f9ddca73765dd2cad2e35b952
Reviewed-on: https://chromium-review.googlesource.com/823964
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50076}
2017-12-13 13:36:20 +00:00
Michal Majewski
08a05c2bff [test] Move data access from suite to test.
- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
2017-12-13 09:14:47 +00:00
Michal Majewski
4f4654a013 [test] Deduplicate parsing source file flags.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I12d62e28b8e22820d4358d0166fa5db5e09b8bc3
Reviewed-on: https://chromium-review.googlesource.com/819630
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50033}
2017-12-12 13:12:18 +00:00
Michal Majewski
430e03b3c6 Remove shell info from testcase.
Bug: v8:6917
Change-Id: Ic50ed8aca2ef6b6e60eae194cf46c2264a416657
Reviewed-on: https://chromium-review.googlesource.com/774265
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49417}
2017-11-16 15:42:26 +00:00
Michal Majewski
0a12eaea45 Remove env field from testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I09fb05ac6d4b9b1223118494ce2c89e3ab5de109
Reviewed-on: https://chromium-review.googlesource.com/771870
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49415}
2017-11-16 15:26:06 +00:00
Michael Achenbach
0a35d6f985 [test] Let flags from test case have precedence
The status-file flags and the flags from the test case's source code
must always overwrite extra flags set by bots.

Bug: v8:6924
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I0e2aabb69da7cfb8ba6c1c79bd3851462071a6ac
Reviewed-on: https://chromium-review.googlesource.com/732656
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49001}
2017-10-27 13:22:39 +00:00
yangguo
aaaaa80f02 [inspector] always include user scripts in the snapshot.
V8 can bundle user scripts in the start up snapshot. These are
shared across contexts, and do not work well context groups.

R=kozyatinskiy@chromium.org
BUG=v8:6274

Review-Url: https://codereview.chromium.org/2836623002
Cr-Original-Commit-Position: refs/heads/master@{#44847}
Committed: 9685cfd310
Review-Url: https://codereview.chromium.org/2836623002
Cr-Commit-Position: refs/heads/master@{#44897}
2017-04-26 15:13:14 +00:00
yangguo
f385b747eb Revert of [inspector] always include user scripts in the snapshot. (patchset #3 id:40001 of https://codereview.chromium.org/2836623002/ )
Reason for revert:
So that https://codereview.chromium.org/2841993002/ can be reverted.

Original issue's description:
> [inspector] always include user scripts in the snapshot.
>
> V8 can bundle user scripts in the start up snapshot. These are
> shared across contexts, and do not work well context groups.
>
> R=kozyatinskiy@chromium.org
> BUG=v8:6274
>
> Review-Url: https://codereview.chromium.org/2836623002
> Cr-Commit-Position: refs/heads/master@{#44847}
> Committed: 9685cfd310

TBR=kozyatinskiy@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=v8:6274

Review-Url: https://codereview.chromium.org/2840923002
Cr-Commit-Position: refs/heads/master@{#44868}
2017-04-25 19:37:33 +00:00
yangguo
9685cfd310 [inspector] always include user scripts in the snapshot.
V8 can bundle user scripts in the start up snapshot. These are
shared across contexts, and do not work well context groups.

R=kozyatinskiy@chromium.org
BUG=v8:6274

Review-Url: https://codereview.chromium.org/2836623002
Cr-Commit-Position: refs/heads/master@{#44847}
2017-04-25 12:08:19 +00:00
yangguo
891bbe2c85 [inspector] correctly test more variants.
This flushed out a number of bugs.

To reproduce, remove the inspector.status file entries, build with GN,
and run `tools/run-tests.py --gn --exhaustive-variants inspector`.

R=mstarzinger@chromium.org
BUG=v8:6165,v8:6166,v8:6167,v8:6168,v8:6170,v8:6171

Review-Url: https://codereview.chromium.org/2777413005
Cr-Commit-Position: refs/heads/master@{#44242}
2017-03-29 18:08:51 +00:00
kozyatinskiy
bbeb6dc15d [inspector] added master test for break locations
BUG=none
R=dgozman@chromium.org

Review-Url: https://codereview.chromium.org/2710903003
Cr-Commit-Position: refs/heads/master@{#43459}
2017-02-27 20:20:39 +00:00
kozyatinskiy
59eb62d483 [inspector] use BREAK_POSITION_ALIGNED for breakpoints
With inline breakpoints DevTools are ready for break position aligned breakpoints instead of statement aligned.

BUG=chromium:695236
R=dgozman@chromium.org,yangguo@chromium.org

Review-Url: https://codereview.chromium.org/2709263002
Cr-Original-Commit-Position: refs/heads/master@{#43385}
Committed: 2fed7a0090
Review-Url: https://codereview.chromium.org/2709263002
Cr-Commit-Position: refs/heads/master@{#43400}
2017-02-24 02:01:26 +00:00
machenbach
bdd23acd92 Revert of [inspector] use BREAK_POSITION_ALIGNED for breakpoints (patchset #7 id:120001 of https://codereview.chromium.org/2709263002/ )
Reason for revert:
breaks layout tests:
https://build.chromium.org/p/client.v8.fyi/builders/V8-Blink%20Linux%2064/builds/13747

which blocks rolling. See also:
https://github.com/v8/v8/wiki/Blink-layout-tests

Original issue's description:
> [inspector] use BREAK_POSITION_ALIGNED for breakpoints
>
> With inline breakpoints DevTools are ready for break position aligned breakpoints instead of statement aligned.
>
> BUG=chromium:695236
> R=dgozman@chromium.org,yangguo@chromium.org
>
> Review-Url: https://codereview.chromium.org/2709263002
> Cr-Commit-Position: refs/heads/master@{#43385}
> Committed: 2fed7a0090

TBR=dgozman@chromium.org,yangguo@chromium.org,kozyatinskiy@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:695236

Review-Url: https://codereview.chromium.org/2709313003
Cr-Commit-Position: refs/heads/master@{#43395}
2017-02-23 20:27:08 +00:00
kozyatinskiy
2fed7a0090 [inspector] use BREAK_POSITION_ALIGNED for breakpoints
With inline breakpoints DevTools are ready for break position aligned breakpoints instead of statement aligned.

BUG=chromium:695236
R=dgozman@chromium.org,yangguo@chromium.org

Review-Url: https://codereview.chromium.org/2709263002
Cr-Commit-Position: refs/heads/master@{#43385}
2017-02-23 00:05:26 +00:00
kozyatinskiy
d1191e1b8a [inspector] added inspector test runner [part 4]
- added inspector test suite definition in testcfg.py
- added JS infrastructure for tests in protocol-test.js

BUG=chromium:635948
R=dgozman@chromium.org,alph@chromium.org

Committed: https://crrev.com/4a5f5d0991656a6aa45b0021a0f5ba6a7ceabe15
Review-Url: https://codereview.chromium.org/2370743003
Cr-Original-Commit-Position: refs/heads/master@{#39895}
Cr-Commit-Position: refs/heads/master@{#39930}
2016-10-02 19:41:28 +00:00
machenbach
ee0d69910b Revert "[inspector] added inspector test runner [part 3-5]"
Revert "[inspector] added inspector test runner [part 3]"

This reverts commit f3f9f4448d.

Revert "[inspector] added inspector test runner [part 4]"

This reverts commit 4a5f5d0991.

Revert "[inspector] added inspector test runner [part 5]"

This reverts commit 9ddbdab195.

Reverting this in order to revert parts 1-2 which block the roll:
https://codereview.chromium.org/2379053003/

BUG=chromium:635948
TBR=kozyatinskiy@chromium.org,
NOTRY=true

Review-Url: https://codereview.chromium.org/2379303002
Cr-Commit-Position: refs/heads/master@{#39908}
2016-09-30 09:25:31 +00:00
kozyatinskiy
4a5f5d0991 [inspector] added inspector test runner [part 4]
- added inspector test suite definition in testcfg.py
- added JS infrastructure for tests in protocol-test.js

BUG=chromium:635948
R=dgozman@chromium.org,alph@chromium.org

Review-Url: https://codereview.chromium.org/2370743003
Cr-Commit-Position: refs/heads/master@{#39895}
2016-09-30 01:35:42 +00:00