Commit Graph

22 Commits

Author SHA1 Message Date
Michael Achenbach
cdfadf4a99 Reland "[test] Print hanging tests on linux on test-runner termination"
This is a reland of 3fc9663159

The private method on_event in timeout proc is now renamed to be truly
private.

Original change's description:
> [test] Print hanging tests on linux on test-runner termination
>
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
>
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}

Bug: v8:8292
Change-Id: Ibad1172666d6f4d2c07884a54edfe9d6499b57fe
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735318
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63072}
2019-08-05 11:50:18 +00:00
Michael Achenbach
cfdbab36d9 Revert "[test] Print hanging tests on linux on test-runner termination"
This reverts commit 3fc9663159.

Reason for revert: _on_event clashes with a the same function in timeout proc.

Original change's description:
> [test] Print hanging tests on linux on test-runner termination
> 
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
> 
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: Ic29013e92c77a9905e06ee13a2449e033285eb36
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8292
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735317
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63066}
2019-08-05 10:52:53 +00:00
Michael Achenbach
3fc9663159 [test] Print hanging tests on linux on test-runner termination
This will print the list of processes still running before and after
joining workers during termination. This will help debugging hanging
tests during flake-bisect or with num-fuzzer, which both terminate
on total timeout and currently still sometimes hang without printing
processes.

Bug: v8:8292
Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63065}
2019-08-05 10:21:48 +00:00
Tamer Tas
d852ae6f2b [testrunner] remove recursive result calls in chain processors
Procs return the result by increasing recursion through result_for.

This CL eliminates that mechanism from the Processor interface and uses boolen
return values for sending tests to signal success or the failure to load the
test into the execution queue.

R=machenbach@chromium.org
CC=​​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174,v8:8731
Change-Id: I073a86ca84bcf88da11132b90013d4c8455bc61e
Reviewed-on: https://chromium-review.googlesource.com/c/1439239
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59201}
2019-01-30 13:04:07 +00:00
Michael Achenbach
1c0b17b03f Revert "[testrunner] load tests concurrently into test execution processor"
This reverts commit 25457c60a7.

Reason for revert: https://crbug.com/v8/8731

Original change's description:
> [testrunner] load tests concurrently into test execution processor
> 
> loading every test up-front into the processing queue costs about 224MB for a
> x64 testsuite run.
> 
> This CL eliminates that overhead by utilizing generators and threading.
> 
> LoadingProc now loads test after receiving the results of the loaded tests.
> 
> R=​machenbach@chromium.org
> CC=​​yangguo@chromium.org,sergiyb@chromium.org
> 
> Bug: v8:8174
> Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
> Reviewed-on: https://chromium-review.googlesource.com/c/1420678
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59056}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: I1e074a031dced367a32a93827b9e863b0331340f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174
Reviewed-on: https://chromium-review.googlesource.com/c/1433792
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59065}
2019-01-24 13:56:04 +00:00
Tamer Tas
25457c60a7 [testrunner] load tests concurrently into test execution processor
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.

This CL eliminates that overhead by utilizing generators and threading.

LoadingProc now loads test after receiving the results of the loaded tests.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174
Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
Reviewed-on: https://chromium-review.googlesource.com/c/1420678
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59056}
2019-01-24 11:19:12 +00:00
Michael Achenbach
4a2d9b1b20 [test] Abort testing immediately on interrupt and sigterm
This is a partial revert of:
https://crrev.com/c/890938 and https://crrev.com/c/893982

Before this CL, the test runner blocked on ongoing tests in order to
process their results after an internal timeout. However, the logic
required for this feature was overly complicated and prevented an
acceptable implementation for fast aborts. Furthermore, also the fuzzers
suffered from timeouts on swarming due to hanging tests.

Instead, we now abort immediately on internal timeout (used on
fuzzers), SIGINT (Ctrl-C) and SIGTERM. Ongoing tests are immediately
terminated and their results are disregarded. On SIGTERM and SIGINT,
we return with non-zero exit codes, and zero on internal timeout.

This will also properly return json output, when the external hard
timeout is reached on swarming (causes SIGTERM).

TBR=sergiyb@chromium.org

Bug: v8:7423, chromium:813065
Change-Id: Ib20f835f58a0970693bdd3b21dc5d766d8e115d8
Reviewed-on: https://chromium-review.googlesource.com/924852
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51399}
2018-02-20 18:02:25 +00:00
Michal Majewski
804cc9f385 [test] Remove context usage from test processors
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Id5a7b41502dfd4be43496b1edb958522a8eb0c31
Reviewed-on: https://chromium-review.googlesource.com/895588
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51023}
2018-02-01 10:35:02 +00:00
Michal Majewski
d3083b46ee [test] Move options to test config
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I254d2e545709029346f585b02a9edf91d3f27893
Reviewed-on: https://chromium-review.googlesource.com/893321
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50996}
2018-01-31 13:46:29 +00:00
Michal Majewski
1478c30786 [test] Return heartbeats and results during pool termination
Bug: v8:6917
Change-Id: I5cca65111141f32f8b9f241a9f482d09e1b54655
Reviewed-on: https://chromium-review.googlesource.com/893982
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50987}
2018-01-31 11:25:26 +00:00
Michal Majewski
d161bc3c14 [test] Added signal handling to test processors
Bug: v8:6917
Change-Id: If91defd11c91fd26bdbacc146992745ea772a941
Reviewed-on: https://chromium-review.googlesource.com/890938
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50985}
2018-01-31 10:39:06 +00:00
Michael Achenbach
04d95171f4 Revert "[test] Let fuzzer total timeout also stop the execution loop"
This reverts commit f4ebbb3fd4.

Reason for revert: Suspected to make things worse than better, e.g.:
https://chromium-swarm.appspot.com/task?id=3b4b2f864304f010&refresh=10&show_raw=1
and
https://chromium-swarm.appspot.com/task?id=3b4b2f8045da5510&refresh=10&show_raw=1

Original change's description:
> [test] Let fuzzer total timeout also stop the execution loop
> 
> TBR=sergiyb@chromium.org
> 
> Bug: v8:6917
> Change-Id: I5bc8f49dc01d98949e3efab01192c663de8027bf
> Reviewed-on: https://chromium-review.googlesource.com/888578
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50887}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: Ib9f530348594e361d491e827aa03a38f41da9f1a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/888519
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50894}
2018-01-26 13:02:50 +00:00
Michael Achenbach
f4ebbb3fd4 [test] Let fuzzer total timeout also stop the execution loop
TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: I5bc8f49dc01d98949e3efab01192c663de8027bf
Reviewed-on: https://chromium-review.googlesource.com/888578
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50887}
2018-01-26 11:44:38 +00:00
Michal Majewski
084b0e763e [test] Support predictable with test processors
Bug: v8:6917
Change-Id: I36b7ef75d34f9dfee20cba09d55063d816334293
Reviewed-on: https://chromium-review.googlesource.com/887021
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50872}
2018-01-25 16:07:55 +00:00
Michal Majewski
88c8bf2e71 [test] Add combine tests option to numfuzz
Bug: v8:6917
Change-Id: I3ba4ca3df8bac400c248fa16c58fcba3497da806
Reviewed-on: https://chromium-review.googlesource.com/881167
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50871}
2018-01-25 15:50:24 +00:00
Michal Majewski
0998eda9a4 [test] Disable reduce result on the main process
Since we're not winning anything by changing the result between
processors on the main process, reduce is noop there and result is
immutable.

Bug: v8:6917
Change-Id: Ieb282e7abd4ab31162aee6b52493a6e1b6a25109
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/878239
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50784}
2018-01-23 08:58:20 +00:00
Michal Majewski
daed2eb723 [test] Move command from test to the result
Bug: v8:6917
Change-Id: I54f908609fadd88bb23bf9fc566d1e2f3ff5e18e
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/870353
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50653}
2018-01-17 12:59:54 +00:00
Michal Majewski
0bfc1b2562 [test] Do not pass unnecessary results
Bug: v8:6917
Change-Id: I1a355bdfe3f873091a7d7c32a937a533a7d8b3d4
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/867053
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50650}
2018-01-17 11:32:12 +00:00
Michal Majewski
60c17bf983 [test] Simplify passing results between test processors.
Bug: v8:6917
Change-Id: Id73e4892a0d1b3b9c5bdd70ccc136e7bd2edf360
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/863603
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50565}
2018-01-15 08:51:40 +00:00
Michal Majewski
463dbab3ec [test] Implemented status file filter as a processor
Bug: v8:6917
Change-Id: I4b10091a40372e1aa614ac26452e20ed481ab686
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/856498
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50482}
2018-01-10 16:08:46 +00:00
Michal Majewski
cfd43ee713 [test] Added test processor documentation.
Bug: v8:6917
Change-Id: I6ecfab76e6d2ee0b4ab86380e8cfbb68df07599e
Reviewed-on: https://chromium-review.googlesource.com/852295
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50457}
2018-01-09 17:05:13 +00:00
Michal Majewski
da3416f53a [test] Introduce test processors
Test processors can be enabled with --infra-staging flag.

Rerunning tests, execution and verbose progress indicator already
work as test processors.

Bug: v8:6917
Change-Id: I40fc42db94dbc8629e8000a3d363030045532fe3
Reviewed-on: https://chromium-review.googlesource.com/850398
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50399}
2018-01-08 11:27:47 +00:00