Commit Graph

302 Commits

Author SHA1 Message Date
Clemens Hammacher
76ac6372fa [wasm] Prepare enabling liftoff by default on ia32 and x64
Before flipping the flag, some tests need to be adapted. This CL
prepares these tests, such that the flag flip CL really just flips a
flag.

R=titzer@chromium.org, hablich@chromium.org

Bug: v8:6600, chromium:787421
Change-Id: I8030df69cda5f3fb81354350a37f65c0d1c669bd
Reviewed-on: https://chromium-review.googlesource.com/1110363
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53959}
2018-06-22 11:06:26 +00:00
Michael Lippautz
020d387f6a [testing] Add gc_stats variant
This is to allow testing object stats collection.

No-try: true
Bug: v8:7760
Change-Id: I1e5318f18bfb3799b90c0ee0aec6df87a794b32c
Reviewed-on: https://chromium-review.googlesource.com/1065478
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53274}
2018-05-22 07:19:20 +00:00
Eric Holk
4d4a3294b9 [test] Reverse sense of wasm_traps variant
D8 enables the Wasm trap handler by default now, but we need to make sure the
older bounds check case still gets test coverage too, as bounds checks will
continue to be a supported configuration.

Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I5b0bdded6929a9b3a8480e87d038398b8d2a0fd8
Reviewed-on: https://chromium-review.googlesource.com/1048835
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53078}
2018-05-08 17:49:14 +00:00
jgruber
68bdf6a75c [builtins] Remove --stress-off-heap-code flag
Embedded builtins are now based off the v8_enable_embedded_builtins
gn flag instead, which conditionally defines V8_EMBEDDED_BUILTINS.

Bug: v8:6666
Change-Id: I44d40d30fce3a3ed9bbf973d46c4990ba3fade40
Reviewed-on: https://chromium-review.googlesource.com/964361
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52092}
2018-03-21 08:52:17 +00:00
Michael Starzinger
0fc6c35c3a [wasm] Remove deprecated --wasm-jit-to-native flag.
The feature in question has been enabled by default for a while and we
no longer need to maintain a configuration without it enabled. Note that
this change only removes the mechanical pieces. Further cleanup enabled
by this will be done as follow-ups.

R=clemensh@chromium.org
BUG=v8:7549

Change-Id: I90e5bcddabe74a18a4d2a88132e8dc93317bcff4
Reviewed-on: https://chromium-review.googlesource.com/958424
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51883}
2018-03-12 14:36:32 +00:00
Ross McIlroy
323ad6a732 [Compiler] Remove unused background_compile flag.
We now unconditionally both parse and compile StreamedSource on the background
thread.

BUG=v8:5203

Change-Id: I42d6fe9059bc1745da3a415d270f46cf1c08b306
Reviewed-on: https://chromium-review.googlesource.com/948854
Reviewed-by: Mythri Alle <mythria@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51758}
2018-03-06 10:42:21 +00:00
Ilija Pavlovic
d2c2b39831 [mips][mips64]: Skip tests for MIPS release 6.
For MIPS architecture release 6, following tests will be skipped:
cctest/test-run-machops/RunFloat64MulAndFloat64Add1
cctest/test-run-machops/RunFloat64MulAndFloat64Add2
cctest/test-run-machops/RunFloat64MulAndFloat64Sub1
cctest/test-run-machops/RunFloat64MulAndFloat64Sub2

TEST=
BUG=

Change-Id: Id359580b809d1387d504d98b6d25d6c112cfda78
Reviewed-on: https://chromium-review.googlesource.com/945689
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#51754}
2018-03-06 09:19:11 +00:00
jgruber
abcc28ced0 [builtins] Enable embedded builtins and add testing variants
This enables the v8_enable_embedded_builtins gn flag on non-ia32 builds
and adds a new --stress-off-heap-code test mode to fyi bots.

v8_enable_embedded_builtins=true changes accesses to constants and
external references to go through the root list in builtins code.

--stress-off-heap-code copies builtins code off-heap on isolate
creation.

A few drive-by-fixes:
- ensure that we actually inspect the correct builtin during
  isolate-independence testing.
- relax tests to decrease maintenance (now we only fail if a builtin
  should be isolate-independent but isn't).
- switch to a different off-heap-trampoline register on arm due to
  conflicts with custom stub linkages.

Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Bug: v8:6666
Change-Id: I09ad3c75cb4342f4c548ea780f275993730896c8
Reviewed-on: https://chromium-review.googlesource.com/934281
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51717}
2018-03-05 09:50:01 +00:00
Michael Achenbach
4a2d9b1b20 [test] Abort testing immediately on interrupt and sigterm
This is a partial revert of:
https://crrev.com/c/890938 and https://crrev.com/c/893982

Before this CL, the test runner blocked on ongoing tests in order to
process their results after an internal timeout. However, the logic
required for this feature was overly complicated and prevented an
acceptable implementation for fast aborts. Furthermore, also the fuzzers
suffered from timeouts on swarming due to hanging tests.

Instead, we now abort immediately on internal timeout (used on
fuzzers), SIGINT (Ctrl-C) and SIGTERM. Ongoing tests are immediately
terminated and their results are disregarded. On SIGTERM and SIGINT,
we return with non-zero exit codes, and zero on internal timeout.

This will also properly return json output, when the external hard
timeout is reached on swarming (causes SIGTERM).

TBR=sergiyb@chromium.org

Bug: v8:7423, chromium:813065
Change-Id: Ib20f835f58a0970693bdd3b21dc5d766d8e115d8
Reviewed-on: https://chromium-review.googlesource.com/924852
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51399}
2018-02-20 18:02:25 +00:00
Michael Achenbach
70147ff2f0 Revert "Update test configurations for code serializer"
This reverts commit 9808093ef6.

Reason for revert: 
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/19149

Original change's description:
> Update test configurations for code serializer
> 
> Update test configurations on the bots to test for serializing
> after execute (--cache=after-execute) and producing full code
> cache (--cache=full-code-cache) options. We no longer need
> to test serializing before execute (--cache=code) on the bots.
> 
> Bug: v8:7302
> Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
> Reviewed-on: https://chromium-review.googlesource.com/869931
> Commit-Queue: Mythri Alle <mythria@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#51235}

TBR=machenbach@chromium.org,yangguo@chromium.org,mythria@chromium.org

Change-Id: I47e03101804194c21b0edf418b744b0ccb66cbf6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7302
Reviewed-on: https://chromium-review.googlesource.com/913134
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51236}
2018-02-12 12:09:54 +00:00
Mythri
9808093ef6 Update test configurations for code serializer
Update test configurations on the bots to test for serializing
after execute (--cache=after-execute) and producing full code
cache (--cache=full-code-cache) options. We no longer need
to test serializing before execute (--cache=code) on the bots.

Bug: v8:7302
Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
Reviewed-on: https://chromium-review.googlesource.com/869931
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51235}
2018-02-12 11:22:54 +00:00
Clemens Hammacher
b60a4c6ac8 Add wasm_no_native variant
This new variant is executed on a small number of bots as part of the
"extra" suite. It checks that the wasm-jit-to-native flag can still be
disabled if unexpected failures pop up on a release branch.

R=machenbach@chromium.org

Bug: v8:7417
Change-Id: I1658cb2f04302fa80915b59bfedd85d980742db5
Reviewed-on: https://chromium-review.googlesource.com/909213
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51204}
2018-02-09 10:19:47 +00:00
Michal Majewski
533820fe83 [test] Move loading status file to base runner
Bug: v8:6917
Change-Id: I2fb63e05336798606dee558d4adc253d5aace040
Reviewed-on: https://chromium-review.googlesource.com/897807
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51038}
2018-02-01 15:32:59 +00:00
Michal Majewski
2fe1af4787 [test] Remove dead code from suites and tests
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I6bfea537347fb236d41b31f0c6c64c108928bf28
Reviewed-on: https://chromium-review.googlesource.com/897784
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51033}
2018-02-01 14:12:49 +00:00
Michal Majewski
e64f546140 [test] Remove old code from run-tests
Only test processors code left. It enabled to move more stuff to
the base runner, like progress indicators creation.

Bug: v8:6917
Change-Id: Ie6dd211cec561a07d92bcc4431ea88eb1842c8fa
Reviewed-on: https://chromium-review.googlesource.com/897624
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51030}
2018-02-01 13:24:57 +00:00
Michal Majewski
804cc9f385 [test] Remove context usage from test processors
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Id5a7b41502dfd4be43496b1edb958522a8eb0c31
Reviewed-on: https://chromium-review.googlesource.com/895588
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51023}
2018-02-01 10:35:02 +00:00
Michal Majewski
1478c30786 [test] Return heartbeats and results during pool termination
Bug: v8:6917
Change-Id: I5cca65111141f32f8b9f241a9f482d09e1b54655
Reviewed-on: https://chromium-review.googlesource.com/893982
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50987}
2018-01-31 11:25:26 +00:00
Michal Majewski
d161bc3c14 [test] Added signal handling to test processors
Bug: v8:6917
Change-Id: If91defd11c91fd26bdbacc146992745ea772a941
Reviewed-on: https://chromium-review.googlesource.com/890938
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50985}
2018-01-31 10:39:06 +00:00
Michal Majewski
5ede3cb533 Reland "[test] Random seed processor"
This is a reland of 0db74d4974.

Original change's description:
> [test] Random seed processor
> 
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
> 
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}

Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
2018-01-31 09:53:34 +00:00
Michael Achenbach
ba30988cbc Revert "[test] Random seed processor"
This reverts commit 0db74d4974.

Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1

Original change's description:
> [test] Random seed processor
> 
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
> 
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
2018-01-30 22:05:03 +00:00
Michal Majewski
0db74d4974 [test] Random seed processor
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.

Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
2018-01-30 15:50:49 +00:00
Michael Achenbach
dc210a95d3 [test] Add interrupt-budget fuzzer
This adds back an option for interrupt budget available in no-snap
builds. This also adds a fuzzer configuration for numfuzz that enables
fuzzing the interrupt budget option. A new flag --disable-analysis
allows to generally skip the fuzzer's analysis phase, which can be
chosen for interrupt budget, which doesn't support an analysis phase.

Bug: v8:6917
Change-Id: I546dd9ee41c3e0fb027108ef4606a34514f230d4
Reviewed-on: https://chromium-review.googlesource.com/885805
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50929}
2018-01-29 16:31:40 +00:00
Michal Majewski
385611cbba Reland "[test] Keep the order of variants"
This is a reland of 8f4407fbd6.

Original change's description:
> [test] Keep the order of variants
> 
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}

Bug: v8:6917
Change-Id: I9d4c617ccec6db8929af3095f2f27d8964c5d6b8
Reviewed-on: https://chromium-review.googlesource.com/888582
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50901}
2018-01-26 14:15:53 +00:00
Michael Achenbach
874eadd1ea Revert "[test] Keep the order of variants"
This reverts commit 8f4407fbd6.

Reason for revert: Failing in old test runner. See comment.

Original change's description:
> [test] Keep the order of variants
> 
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: Ia8608053e080c70d2ac09d7921a6abb6cd0e338f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/888749
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50895}
2018-01-26 13:04:29 +00:00
Michal Majewski
8f4407fbd6 [test] Keep the order of variants
Bug: v8:6917
Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
Reviewed-on: https://chromium-review.googlesource.com/886841
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50874}
2018-01-25 17:52:25 +00:00
Michal Majewski
88c8bf2e71 [test] Add combine tests option to numfuzz
Bug: v8:6917
Change-Id: I3ba4ca3df8bac400c248fa16c58fcba3497da806
Reviewed-on: https://chromium-review.googlesource.com/881167
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50871}
2018-01-25 15:50:24 +00:00
Michael Achenbach
997d5a6a5c [test] Add code_serializer testing variant
Also sort all variants in configuration mapping.

Bug: v8:7285
Change-Id: I2d3032b911e48c34557e47e50345fb29da7bbe85
Reviewed-on: https://chromium-review.googlesource.com/866502
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50584}
2018-01-15 15:04:07 +00:00
Michael Achenbach
56fa0a535b [test] Remove deprecated FAST_VARIANTS modifier
Bug: v8:7264
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I0c07c92955c0b6c145c7b72e5014f828eaf59c9a
Reviewed-on: https://chromium-review.googlesource.com/865903
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50582}
2018-01-15 14:30:47 +00:00
Michal Majewski
29b1c388e7 [test] Introduce stress sampling allocation profiler mode
Bug: v8:7209
Change-Id: Ia10479d7eea6ef2f352d008e2f4b74e2394ab79b
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/861623
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ali Ijaz Sheikh <ofrobots@google.com>
Cr-Commit-Position: refs/heads/master@{#50527}
2018-01-12 11:34:16 +00:00
Michal Majewski
7f01c9303f [test] Implement variant generators as processors
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng;luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Ib5bfdf4d6fee6102f62c7334a1b22146f1a1fc5b
Reviewed-on: https://chromium-review.googlesource.com/857376
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50525}
2018-01-12 10:39:20 +00:00
Michael Starzinger
4d09583ad9 [test] Restore "stress_incremental_marking" variant.
This removes the explicit --write-protect-code-memory flag from the list
of flags in the "stress_incremental_marking". The feature is enabled by
default by now and no longer needs explicit testing.

R=ulan@chromium.org
BUG=v8:6792

Change-Id: I5d6ba21dff261488bbe1b0148ce204bf78d57334
Reviewed-on: https://chromium-review.googlesource.com/860661
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50500}
2018-01-11 12:14:14 +00:00
Michal Majewski
463dbab3ec [test] Implemented status file filter as a processor
Bug: v8:6917
Change-Id: I4b10091a40372e1aa614ac26452e20ed481ab686
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/856498
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50482}
2018-01-10 16:08:46 +00:00
Michal Majewski
501413b9b9 [test] Implemented indicators as processors
Added simple system tests for different progress indicators.

Bug: v8:6917
Change-Id: I906ddfd06e82cc19d3b2210e09457456be00309b
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/852495
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50406}
2018-01-08 13:08:40 +00:00
Michal Majewski
da3416f53a [test] Introduce test processors
Test processors can be enabled with --infra-staging flag.

Rerunning tests, execution and verbose progress indicator already
work as test processors.

Bug: v8:6917
Change-Id: I40fc42db94dbc8629e8000a3d363030045532fe3
Reviewed-on: https://chromium-review.googlesource.com/850398
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50399}
2018-01-08 11:27:47 +00:00
Michael Achenbach
48d436b2ed [test] Add "trusted" testing variant to a subset of bots
Bug: chromium:798982
Change-Id: I096e2df5a5f8d038c980bc9f575857256f0a01df
Reviewed-on: https://chromium-review.googlesource.com/852073
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50378}
2018-01-05 10:36:31 +00:00
Michal Majewski
9f7d440e98 [test] Check output on the worker process.
I added additional exception logging in the execution.py since
errors in processing results were really difficult to debug.

There is a problem on Windows with class serialization when
it comes from dynamically loaded module. To fix it I moved all
output processors to the tools/testrunner/outproc/ and import
them in test/*/testcfg.py.

Bug: v8:6917
Change-Id: Ida604641d659b006e91faf1d56a37769ec47f5f3
Reviewed-on: https://chromium-review.googlesource.com/842784
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50361}
2018-01-04 14:12:31 +00:00
Michael Achenbach
c1e9bc02f8 Revert "Revert "[test] Move has unexpected output to outproc.""
This reverts commit 1685b5d27a.

Reason for revert: Was probably caused by infra change:
https://crrev.com/c/845781

Original change's description:
> Revert "[test] Move has unexpected output to outproc."
> 
> This reverts commit 71605b3ea4.
> 
> Reason for revert: Seems to break static-initializers step:
> https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
> 
> Original change's description:
> > [test] Move has unexpected output to outproc.
> > 
> > Expected outcomes optimized to serialize [PASS] as None.
> > 
> > Keeping expected outcomes inside output processors should be
> > optimized in the future. Few possible optimizations:
> > - separate classes for tests that are expected to PASS - done as
> > an example in mozilla test suite.
> > - cache output processors inside testcase.
> > - share output processors between copies of the same test - needs
> > some updates to the create_variant to update outproc only if
> > expected outcomes changed.
> > 
> > Bug: v8:6917
> > Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> > Reviewed-on: https://chromium-review.googlesource.com/843025
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#50347}
> 
> TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
> 
> Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6917
> Reviewed-on: https://chromium-review.googlesource.com/849572
> Reviewed-by: Bill Budge <bbudge@chromium.org>
> Commit-Queue: Bill Budge <bbudge@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50348}

TBR=bbudge@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: I7a522b6487de6e96985d223524533493eb9171f0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/848975
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50350}
2018-01-04 07:39:21 +00:00
Bill Budge
1685b5d27a Revert "[test] Move has unexpected output to outproc."
This reverts commit 71605b3ea4.

Reason for revert: Seems to break static-initializers step:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156

Original change's description:
> [test] Move has unexpected output to outproc.
> 
> Expected outcomes optimized to serialize [PASS] as None.
> 
> Keeping expected outcomes inside output processors should be
> optimized in the future. Few possible optimizations:
> - separate classes for tests that are expected to PASS - done as
> an example in mozilla test suite.
> - cache output processors inside testcase.
> - share output processors between copies of the same test - needs
> some updates to the create_variant to update outproc only if
> expected outcomes changed.
> 
> Bug: v8:6917
> Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> Reviewed-on: https://chromium-review.googlesource.com/843025
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50347}

TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com

Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/849572
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50348}
2018-01-03 23:01:11 +00:00
Michal Majewski
71605b3ea4 [test] Move has unexpected output to outproc.
Expected outcomes optimized to serialize [PASS] as None.

Keeping expected outcomes inside output processors should be
optimized in the future. Few possible optimizations:
- separate classes for tests that are expected to PASS - done as
an example in mozilla test suite.
- cache output processors inside testcase.
- share output processors between copies of the same test - needs
some updates to the create_variant to update outproc only if
expected outcomes changed.

Bug: v8:6917
Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
Reviewed-on: https://chromium-review.googlesource.com/843025
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50347}
2018-01-03 22:19:49 +00:00
Michael Achenbach
e8912359b8 [test] Refactor commands into OS-specific versions
This seperates OS-specific code of the Command class into a Posix and a
WindowsCommand to simplify scattered OS checks.

This also removes some temporary mac debug output that's obsolete after
https://crbug.com/v8/6927 got resolved.

TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: Iaab3b527ce556dfba797a164ae58e8dd358eb56f
Reviewed-on: https://chromium-review.googlesource.com/847000
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50346}
2018-01-03 19:38:28 +00:00
Michael Achenbach
5a8e1f3f1b [test] Make coverage in test-runner tests work with multiprocessing
This replaces multiprocessing with threading in tests to make python
coverage work.

TBR=sergiyb@chromium.org

Bug: v8:6917
Change-Id: Idff763dfefa4a7fc782133d94089b3a5b00a194d
Reviewed-on: https://chromium-review.googlesource.com/844735
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50344}
2018-01-03 17:37:05 +00:00
Michael Achenbach
460af93123 [test] Enable slow-path variant on a subset of bots
Prepared by:
https://crrev.com/c/798412

Bug: v8:7120
Change-Id: I28a00f175673209e6d5f662e30a5448236e7a7c9
Reviewed-on: https://chromium-review.googlesource.com/800612
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50287}
2017-12-21 22:05:09 +00:00
Michael Achenbach
efdbc429bb [test] Enable staging test runner changes on fyi bots
Bug: v8:6917
Change-Id: Ica9b590d194da305416826eeed9f49963b8d97ca
Reviewed-on: https://chromium-review.googlesource.com/839776
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50281}
2017-12-21 17:32:09 +00:00
Michal Majewski
a130438e13 [test] Removed old output parsing functions.
All get outcome logic moved out from testsuite which makes output
processors serializable. Output processors that are not global
are stored inside testcase to share them with all test copies.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I625466b2b00565d885556f96cab03d8d16fa7ff4
Reviewed-on: https://chromium-review.googlesource.com/837069
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50275}
2017-12-21 15:25:33 +00:00
Michal Majewski
d705e15857 [test] Prepare initial version of output processor
First CL moving parsing output to a separate class.
Following CLs will move (and deduplicate) code from
testcases by sharing common features like comparing
stdout to the expected output.

Bug: v8:6917
Change-Id: I1174a985949a0b946cefbf7e7c91e8815cbb4f15
Reviewed-on: https://chromium-review.googlesource.com/834130
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50228}
2017-12-20 11:24:07 +00:00
Michal Majewski
56b2b3be5e [test] Fix --report output.
Fix report mode and make it use testcase properties so statusfile
outcomes can be private.

Bug: v8:6917
Change-Id: Id38e89e0ba427c3bbb7ad12ba93e02beb7e46219
Reviewed-on: https://chromium-review.googlesource.com/833909
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50209}
2017-12-19 16:35:39 +00:00
Michal Majewski
5e6fd798f9 [test] Fixed target_name in json progress indicator
Pass shell name instead of an absolute path.

Bug: v8:796166
Change-Id: Ia9472e893fd2cb3fde2a94997f3e9daf30da06ea
Reviewed-on: https://chromium-review.googlesource.com/833917
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50207}
2017-12-19 15:23:54 +00:00
Michal Majewski
4695e97905 [test] Move getting outcomes to the statusfile
Bug: v8:6917
Change-Id: I175fa426546f2f3775a35f1094dfb19e06b2185d
Reviewed-on: https://chromium-review.googlesource.com/832394
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50203}
2017-12-19 14:35:16 +00:00
Michal Majewski
e0c4321479 [test] Prepare new API for statusfile parsing.
First step in moving all statusfile logic into statusfile.py.

Introduce StatusFile object that will be used for storing and managing
outcomes.

Bug: v8:6917
Change-Id: I024f9b1d029830345149422a08a8905e92545252
Reviewed-on: https://chromium-review.googlesource.com/832433
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50202}
2017-12-19 14:26:26 +00:00
Michal Majewski
c0173165ee [test] Update testsuite unittests.
Fix unittests since they were incompatible with the
new testcase and testsuite API.

Bug: v8:6917
Change-Id: I917bf58e21402e0b90bc91c0483ade0e7c90bdd6
Reviewed-on: https://chromium-review.googlesource.com/832392
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50201}
2017-12-19 14:13:56 +00:00