Commit Graph

106 Commits

Author SHA1 Message Date
Clemens Backes
14ad5cd194 [foozzie] Test the --turbo-force-mid-tier-regalloc flag
This flag will force the usage of the mid-tier register allocator, which
is otherwise only used for huge functions, hence does not get much
fuzzer coverage.

R=machenbach@chromium.org

Bug: v8:12330
Change-Id: I30b602b54adf858a2a3aaf73d8828875c1f6df7a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3234492
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77500}
2021-10-21 14:02:05 +00:00
Victor Gomes
5bd6753230 [tools] Port testrunner/testproc/filter.py to PY3
Bug: chromium:1245634
Change-Id: I48eb590fa8c75fe2eded6b85dab6680efb751fd0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3199873
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Auto-Submit: Victor Gomes <victorgomes@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77193}
2021-10-01 12:15:15 +00:00
Al Muthanna Athamina
010d43d2de [infra] Make no-fail the default for numfuzz builders
Remove infra-staging from numfuzz builders and add
no-fail instead. Also remove the staging builders.

Bug: v8:11826
Cq-Include-Trybots: luci.v8.try:v8_numfuzz_dbg_ng,v8_numfuzz_ng,v8_numfuzz_tsan_ng
Change-Id: Icd7bc6275b5c78e38f0e75b5d133b3b7056f333e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3182233
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77142}
2021-09-29 15:23:56 +00:00
Michael Achenbach
65f8948a08 [test] Neuter stack-size for more tests incompatible with stack-size fuzzing
Also bump the minimum size determined by manually testing with:
mjsunit/harmony/modules-import-13

Bug: v8:12197
Change-Id: Ic21d42547ba14cd927d158388ad4bc6f1891f45c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3147453
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#76723}
2021-09-08 11:21:14 +00:00
Michael Achenbach
991fc40e0c [test] Add --stress-flush-code to numfuzz experiments
No-Try: true
Bug: v8:12172
Change-Id: I4bace3c91e71e4529ac8d75df3297036c6ca9756
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3141588
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#76687}
2021-09-07 08:41:44 +00:00
Al Muthanna Athamina
f35bb272db Add PASS expectations in NumFuzz behind infra-staging
Bug: v8:11826
Cq-Include-Trybots: luci.v8.try:v8_numfuzz_dbg_ng,v8_numfuzz_ng,v8_numfuzz_tsan_ng
Change-Id: I1e77562cd86aa5b3a3a8c51e7bb4d37079b450d8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3118947
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#76534}
2021-08-27 13:23:49 +00:00
Al Muthanna Athamina
1f83d4d8f4 Add stack size option to fuzzer
Bug: v8:11826
Change-Id: I30a582351305e1548948086709fc9ba0b0a91bd8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3069286
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Cr-Commit-Position: refs/heads/master@{#76345}
2021-08-17 12:47:44 +00:00
Clemens Backes
9740901ae1 [testrunner] Fix test selection on Windows
This makes flako usable on Windows using standard test names.
A workaround to bisecting to failures before this CL is to replace the
test_name by something like "mjsunit/regress\regress-1138075", i.e.
using this on the command line to trigger flako:
-p 'test_name="mjsunit/regress\\regress-1138075"'

R=liviurau@chromium.org

Bug: v8:9218
Change-Id: I37596efcaeca780eeacb27c2841fe8302ddb1e49
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3081610
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#76222}
2021-08-11 09:25:23 +00:00
Michael Achenbach
f370d60711 [test] Add verbose output of flaky tests
The verbose output shown on bots didn't print the first failing result
of a flaky test before. Now the result line shows all results and
the details in the end show the output of the first failure.

Previously it was confusing as it seemed that the json results and
the test runner output differed.

We now print PASS in all caps like the other statuses. A test for
this case already existed and the output is now updated.

Bug: v8:8434
Change-Id: I473ec392e0028bf64b3da53d4b37446ffcd17277
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2919670
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74792}
2021-05-26 16:22:59 +00:00
Michael Achenbach
ee56a9863e [test] Run heavy tests sequentially
This adds a new status file indicator "HEAVY" to mark tests with high
resource demands. There will be other tests running in parallel,
but only a limited number of other heavy tests. The limit is
controlled with a new parameter --max-heavy-tests and defaults to 1.

The change also marks a variety of tests as heavy that recently had
flaky timeouts. Heavy also implies slow, hence heavy tests are
executed at the beginning with a higher timeout like other slow tests.

The implementation is encapsulated in the test-processor chain. A
new processor buffers heavy tests in a queue and adds buffered tests
only if other heavy tests have ended their computation.

Bug: v8:5861
Change-Id: I89648ad0030271a3a5af588ecc9c43285b728d6d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2905767
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74712}
2021-05-21 12:59:49 +00:00
Michael Achenbach
1a9c9f0fc0 [test] Stress concurrent inlining on numfuzz
This will give extra coverage of the mode on the numfuzz tsan bot.

No-Try: true
Bug: chromium:11655
Change-Id: Ifcc4ad304116fa1b0331cc48e41c07405193d514
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2862809
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74316}
2021-05-03 06:45:13 +00:00
Michael Achenbach
76ce5bbb1d [test] Temporarily skip no-sse flags on flag fuzzer.
No-Try: true
Bug: v8:11442
Change-Id: I856470d95cf0b70e98e09a0ce4e8d80377e7a92a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2691052
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Nico Hartmann <nicohartmann@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#72706}
2021-02-12 15:14:48 +00:00
Michael Achenbach
373a9a8cfc [test] Switch to flattened json output
This flattens the json output to one result record as a dict. In
the past several records with different arch/mode combinations
could be run, but this is deprecated since several releases.

We also drop storing the arch/mode information in the record as it
isn't used on the infra side for anything.

This was prepared on the infra side by:
https://crrev.com/c/2453562

Bug: chromium:1132088
Change-Id: I944514dc00a671e7671bcdbcaa3a72407476d7ad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2456987
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70402}
2020-10-08 13:05:11 +00:00
Tobias Tebbi
e3f33f5241 [flags][testrunner] inform about failing flag expectations
Bug: v8:10577

Change-Id: I28aaec30b73def5034294f71bb6bda466172b4ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2400978
Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70055}
2020-09-22 14:02:28 +00:00
Tobias Tebbi
0832a1093d Reland^5 "[flags] warn about contradictory flags"
This is a reland of 2000aea58a
Changes compared to last reland:
- Add rule in variants.py for --enable_experimental_regexp_engine.
- Make sure --abort-on-contradictory-flags works as well as --fuzzing
  to disable the checking for fuzzers, including for d8 flags.

Original change's description:
> Reland^4 "[flags] warn about contradictory flags"
>
> This is a reland of 0ba115e6a9
> Changes compared to last reland:
> - Fix Python code trying to write to expected_outcomes, which is now a
>   computed property.
> - Fix remaining place in d8.cc that ignored the --fuzzing flag.
> - Expect flag contradictions for --cache in code_serializer variant.
>
> Original change's description:
> > Reland^3 "[flags] warn about contradictory flags"
> >
> > Changes:
> > - Also allow second parameter influenced by --cache to be reassigned.
> > - Fix --stress-opt to only --always-opt in the last iteration as before.
> >
> > Original change's description:
> > > Reland^2 "[flags] warn about contradictory flags"
> > >
> > > This is a reland of d8f8a7e210
> > > Change compared to last reland:
> > > - Do not check for d8 flag contradictions in the presence of --fuzzing
> > > - Allow identical re-declaration of --cache=*
> > >
> > > Original change's description:
> > > > Reland "[flags] warn about contradictory flags"
> > > >
> > > > This is a reland of b8f9166664
> > > > Difference to previous CL: Additional functionality to specify
> > > > incompatible flags based on GN variables and extra-flags, used
> > > > to fix the issues that came up on the waterfall.
> > > >
> > > > This also changes the rules regarding repeated flags: While
> > > > explicitly repeated flags are allowed for boolean values as long
> > > > as they are identical, repeated flags or explicit flags in the
> > > > presence of an active implication are disallowed for non-boolean
> > > > flags. The latter simplifies specifying conflict rules in
> > > > variants.py. Otherwise a rule like
> > > >
> > > > INCOMPATIBLE_FLAGS_PER_EXTRA_FLAG = {
> > > >   "--gc-interval=*": ["--gc-interval=*"],
> > > > }
> > > >
> > > > wouldn't work because specifying the same GC interval twice
> > > > wouldn't actually count as a conflict. This was an issue with
> > > > test/mjsunit/wasm/gc-buffer.js, which specifies
> > > > --gc-interval=500 exactly like the extra flag by the stress bot.
> > > >
> > > > Also, this now expands contradictory flags checking to d8 flags
> > > > for consistency.
> > > >
> > > > Original change's description:
> > > > > [flags] warn about contradictory flags
> > > > >
> > > > > Design Doc: https://docs.google.com/document/d/1lkvu8crkK7Ei39qjkPCFijpNyxWXsOktG9GB-7K34jM/
> > > > >
> > > > > Bug: v8:10577
> > > > > Change-Id: Ib9cfdffa401c48c895bf31caed5ee03545beddab
> > > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2154792
> > > > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > > > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > > > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > > > Cr-Commit-Position: refs/heads/master@{#68168}
> > > >
> > > > Bug: v8:10577
> > > > Change-Id: I268e590ee18a535b13dee14eeb15ddd0a9ee8341
> > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2235115
> > > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > > Cr-Commit-Position: refs/heads/master@{#68989}
> > >
> > > Bug: v8:10577
> > > Change-Id: I31d2794d4f9ff630f3444210100c64d67d881276
> > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2339464
> > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > Cr-Commit-Position: refs/heads/master@{#69339}
> >
> > Bug: v8:10577
> > Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng
> > Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_isolates_rel_ng
> > Change-Id: I4a69dc57a102782cb453144323e3752ac8278624
> > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2352770
> > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
> > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#69433}
>
> Change-Id: Ib6d2aeb495210f581ac671221c265df58e8e5e70
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2398640
> Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#69954}

Bug: v8:10577
TBR: clemensb@chromium.org, tmrts@chromium.org
Change-Id: Iab2d32cdcc2648934fc52255ccf3ae3ec9ca4d9b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2416386
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70000}
2020-09-18 15:45:00 +00:00
Bill Budge
a0e38f3186 Revert "Reland^4 "[flags] warn about contradictory flags""
This reverts commit 2000aea58a.

Reason for revert: Breaks NumFuzz.

Original change's description:
> Reland^4 "[flags] warn about contradictory flags"
> 
> This is a reland of 0ba115e6a9
> Changes compared to last reland:
> - Fix Python code trying to write to expected_outcomes, which is now a
>   computed property.
> - Fix remaining place in d8.cc that ignored the --fuzzing flag.
> - Expect flag contradictions for --cache in code_serializer variant.
> 
> Original change's description:
> > Reland^3 "[flags] warn about contradictory flags"
> >
> > Changes:
> > - Also allow second parameter influenced by --cache to be reassigned.
> > - Fix --stress-opt to only --always-opt in the last iteration as before.
> >
> > Original change's description:
> > > Reland^2 "[flags] warn about contradictory flags"
> > >
> > > This is a reland of d8f8a7e210
> > > Change compared to last reland:
> > > - Do not check for d8 flag contradictions in the presence of --fuzzing
> > > - Allow identical re-declaration of --cache=*
> > >
> > > Original change's description:
> > > > Reland "[flags] warn about contradictory flags"
> > > >
> > > > This is a reland of b8f9166664
> > > > Difference to previous CL: Additional functionality to specify
> > > > incompatible flags based on GN variables and extra-flags, used
> > > > to fix the issues that came up on the waterfall.
> > > >
> > > > This also changes the rules regarding repeated flags: While
> > > > explicitly repeated flags are allowed for boolean values as long
> > > > as they are identical, repeated flags or explicit flags in the
> > > > presence of an active implication are disallowed for non-boolean
> > > > flags. The latter simplifies specifying conflict rules in
> > > > variants.py. Otherwise a rule like
> > > >
> > > > INCOMPATIBLE_FLAGS_PER_EXTRA_FLAG = {
> > > >   "--gc-interval=*": ["--gc-interval=*"],
> > > > }
> > > >
> > > > wouldn't work because specifying the same GC interval twice
> > > > wouldn't actually count as a conflict. This was an issue with
> > > > test/mjsunit/wasm/gc-buffer.js, which specifies
> > > > --gc-interval=500 exactly like the extra flag by the stress bot.
> > > >
> > > > Also, this now expands contradictory flags checking to d8 flags
> > > > for consistency.
> > > >
> > > > Original change's description:
> > > > > [flags] warn about contradictory flags
> > > > >
> > > > > Design Doc: https://docs.google.com/document/d/1lkvu8crkK7Ei39qjkPCFijpNyxWXsOktG9GB-7K34jM/
> > > > >
> > > > > Bug: v8:10577
> > > > > Change-Id: Ib9cfdffa401c48c895bf31caed5ee03545beddab
> > > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2154792
> > > > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > > > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > > > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > > > Cr-Commit-Position: refs/heads/master@{#68168}
> > > >
> > > > Bug: v8:10577
> > > > Change-Id: I268e590ee18a535b13dee14eeb15ddd0a9ee8341
> > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2235115
> > > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > > Cr-Commit-Position: refs/heads/master@{#68989}
> > >
> > > Bug: v8:10577
> > > Change-Id: I31d2794d4f9ff630f3444210100c64d67d881276
> > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2339464
> > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > Cr-Commit-Position: refs/heads/master@{#69339}
> >
> > Bug: v8:10577
> > Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng
> > Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_isolates_rel_ng
> > Change-Id: I4a69dc57a102782cb453144323e3752ac8278624
> > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2352770
> > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
> > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#69433}
> 
> Change-Id: Ib6d2aeb495210f581ac671221c265df58e8e5e70
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2398640
> Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#69954}

TBR=clemensb@chromium.org,tebbi@chromium.org,tmrts@chromium.org

Change-Id: I2dc80bcad9f74c29298902e01939e7e7f3336cf6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2415133
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69959}
2020-09-16 20:45:31 +00:00
Tobias Tebbi
2000aea58a Reland^4 "[flags] warn about contradictory flags"
This is a reland of 0ba115e6a9
Changes compared to last reland:
- Fix Python code trying to write to expected_outcomes, which is now a
  computed property.
- Fix remaining place in d8.cc that ignored the --fuzzing flag.
- Expect flag contradictions for --cache in code_serializer variant.

Original change's description:
> Reland^3 "[flags] warn about contradictory flags"
>
> Changes:
> - Also allow second parameter influenced by --cache to be reassigned.
> - Fix --stress-opt to only --always-opt in the last iteration as before.
>
> Original change's description:
> > Reland^2 "[flags] warn about contradictory flags"
> >
> > This is a reland of d8f8a7e210
> > Change compared to last reland:
> > - Do not check for d8 flag contradictions in the presence of --fuzzing
> > - Allow identical re-declaration of --cache=*
> >
> > Original change's description:
> > > Reland "[flags] warn about contradictory flags"
> > >
> > > This is a reland of b8f9166664
> > > Difference to previous CL: Additional functionality to specify
> > > incompatible flags based on GN variables and extra-flags, used
> > > to fix the issues that came up on the waterfall.
> > >
> > > This also changes the rules regarding repeated flags: While
> > > explicitly repeated flags are allowed for boolean values as long
> > > as they are identical, repeated flags or explicit flags in the
> > > presence of an active implication are disallowed for non-boolean
> > > flags. The latter simplifies specifying conflict rules in
> > > variants.py. Otherwise a rule like
> > >
> > > INCOMPATIBLE_FLAGS_PER_EXTRA_FLAG = {
> > >   "--gc-interval=*": ["--gc-interval=*"],
> > > }
> > >
> > > wouldn't work because specifying the same GC interval twice
> > > wouldn't actually count as a conflict. This was an issue with
> > > test/mjsunit/wasm/gc-buffer.js, which specifies
> > > --gc-interval=500 exactly like the extra flag by the stress bot.
> > >
> > > Also, this now expands contradictory flags checking to d8 flags
> > > for consistency.
> > >
> > > Original change's description:
> > > > [flags] warn about contradictory flags
> > > >
> > > > Design Doc: https://docs.google.com/document/d/1lkvu8crkK7Ei39qjkPCFijpNyxWXsOktG9GB-7K34jM/
> > > >
> > > > Bug: v8:10577
> > > > Change-Id: Ib9cfdffa401c48c895bf31caed5ee03545beddab
> > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2154792
> > > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > > Cr-Commit-Position: refs/heads/master@{#68168}
> > >
> > > Bug: v8:10577
> > > Change-Id: I268e590ee18a535b13dee14eeb15ddd0a9ee8341
> > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2235115
> > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > Cr-Commit-Position: refs/heads/master@{#68989}
> >
> > Bug: v8:10577
> > Change-Id: I31d2794d4f9ff630f3444210100c64d67d881276
> > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2339464
> > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#69339}
>
> Bug: v8:10577
> Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng
> Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_isolates_rel_ng
> Change-Id: I4a69dc57a102782cb453144323e3752ac8278624
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2352770
> Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#69433}

Change-Id: Ib6d2aeb495210f581ac671221c265df58e8e5e70
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2398640
Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69954}
2020-09-16 16:39:49 +00:00
Michael Achenbach
d8d6110bbe [test] Print less in verbose mode
I/O is quite expensive on the bots. This cuts down a bit of it by
printing slightly fewer characters per test in verbose mode.

This leads to an overall speed improvement of ~20% for large test
suites, e.g. Test262 output-collection time goes from ~2m30 to ~2m.

The averages to a 5-10% overall speed improvement for slow tryjobs.

Bug: v8:10916
Change-Id: I56dcb072af8eb32a1e09e17a05db5782c6d79315
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2414038
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69950}
2020-09-16 15:52:00 +00:00
Michael Achenbach
e0f85c04f9 [test] Clean up stray processes before running tests on swarming
Bug: v8:10680
Change-Id: I3a6055372b757fac4c5e28840536d1389e857437
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2297381
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68868}
2020-07-15 13:42:21 +00:00
Camillo Bruni
81d37159e6 [tools] Add streaming test runner
This adds a simple test runner that prints a line for every test with
the appropriate status prefix: PASS, FAIL, CRASH or TIMEOUT

Change-Id: Ic1ba78667c38cd4392af027bb6cb671b274680b7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2264098
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68570}
2020-06-29 09:42:23 +00:00
Ng Zhi An
6f296e0b54 [Py3] Get tools/testrunner/testproc closer to Py3
This modernizes python code without breaking Py2 compat.

Ran with command:

futurize --stage1 -w tools/testrunner/testpro

Manual fixup to util_unittest to modify import paths and change to
absolute imports.

Bug: v8:9871
Change-Id: I2ac29622aff5daebc9dc42145c1d96dff8258546
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2252549
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Zhi An Ng <zhin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68492}
2020-06-23 18:55:05 +00:00
Camillo Bruni
d9337dc100 [testrunner] Output unexpected passing tests
Passing tests that are marked as fail in a status file are not
immediately visible as such.

- Always show "--- FAILED ---" for failing tests
- Show "--- UNEXPECTED PASS ---" for unexpectedly passing tests

Drive-by-fixes:
- Color failures in red with --progress=color
- Color repro command in yellow with --progress=color

Change-Id: Id43ecec348dbfd4ff627ea6aa4ba458a2e5a8445
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2213434
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68131}
2020-06-03 09:16:29 +00:00
Liviu Rau
5f7bc600a6 [testing] Collect all relevant data about slow tests
Slow tests are now collected in a heap with a fixed size.
When the maximum size is reached we evict the fastest test
after adding a new test to the heap.

Bug: v8:10168
Change-Id: If3298df85d6e924451f55fe9350e293169cc849d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2106205
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67572}
2020-05-05 17:35:09 +00:00
Michael Achenbach
9491d5eaa4 [numfuzz] Temporarily don't use --budget-for-feedback-vector-allocation=0
No-Try: true
Bug: v8:10243
Change-Id: Ief8ef4b39bffbfa30abf32010eb3ae25c05e25f6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2064972
Reviewed-by: Mythri Alle <mythria@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66365}
2020-02-20 10:54:33 +00:00
Liviu Rau
ebbaf2b787 Collect 100 slow tests by default
If we do not want _all_ tests durations, it makes sense to reuse the existing slow test collection and aggregation

Bug: v8:10168
Change-Id: I500acdb799f41ee6f0fb2c57afb95e1e1830b2dd
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2064221
Auto-Submit: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66342}
2020-02-19 14:57:54 +00:00
Michael Achenbach
0c25872503 [test] Add more flags to numfuzz
This adds 3 flags to the numfuzz fuzzer depending on a probability:
--budget-for-feedback-vector-allocation=0
--interrupt-budget=100
--no-lazy-feedback-allocation

No-Try: true
Bug: v8:10204
Change-Id: I83dabcd0e3ca80bebe596d65d03b3e99d8ecbf03
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2060490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66291}
2020-02-17 15:08:27 +00:00
Liviu Rau
b9a690f38c [testing] Collect test duration
We will be able to collect test duration and later upload them in BQ.


Change-Id: Ie5610d4e872259857bf3f26ba698fa65d23058be
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2020952
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66040}
2020-01-30 10:17:06 +00:00
Michael Achenbach
a42a0b722a Reland "[test] Add more flags to numfuzz flags fuzzer"
This is a reland of ab1b511c16

The offending flags are removed.

Original change's description:
> [test] Add more flags to numfuzz flags fuzzer
>
> This adds a selection of flags to numfuzz that are already used
> for different testing variants or on clusterfuzz for
> correctness testing.
>
> No-Try: true
> Change-Id: I79745b281b001f57d2b24977f3a8e9ce3bbab2a4
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906573
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64884}

No-Try: true
Change-Id: Ie01f244147be0b0fda8cec83f48ac3f73c5a81ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910113
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64905}
2019-11-12 11:33:31 +00:00
Michael Achenbach
2750dc9ed9 Revert "[test] Add more flags to numfuzz flags fuzzer"
This reverts commit ab1b511c16.

Reason for revert: too many spurious errors

Original change's description:
> [test] Add more flags to numfuzz flags fuzzer
> 
> This adds a selection of flags to numfuzz that are already used
> for different testing variants or on clusterfuzz for
> correctness testing.
> 
> No-Try: true
> Change-Id: I79745b281b001f57d2b24977f3a8e9ce3bbab2a4
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906573
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64884}

TBR=machenbach@chromium.org,clemensb@chromium.org,almuthanna@google.com,liviurau@chromium.org

Change-Id: Iba9cfa8e6e8e2cb3b9fe0f803b07376ae55d783c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910112
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64895}
2019-11-11 17:54:33 +00:00
Michael Achenbach
fd282c820e Revert "[numfuzz] Remove jitless from flags experiment"
This reverts commit 509995d371.

Reason for revert: want to revert first CL

Original change's description:
> [numfuzz] Remove jitless from flags experiment
> 
> Produces too many spurious errors with tests that normally get
> skipped in jitless variant.
> 
> TBR=clemensb@chromium.org
> 
> No-Try: true
> Change-Id: Iddf0e39e4c454a3b17568ba17a014e8d38922052
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910107
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64889}

TBR=machenbach@chromium.org,clemensb@chromium.org,almuthanna@google.com,liviurau@chromium.org

Change-Id: I238323289fd29fec82c58795c2ad829e7c0a073a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910111
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64894}
2019-11-11 17:53:26 +00:00
Michael Achenbach
509995d371 [numfuzz] Remove jitless from flags experiment
Produces too many spurious errors with tests that normally get
skipped in jitless variant.

TBR=clemensb@chromium.org

No-Try: true
Change-Id: Iddf0e39e4c454a3b17568ba17a014e8d38922052
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910107
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64889}
2019-11-11 15:18:14 +00:00
Michael Achenbach
ab1b511c16 [test] Add more flags to numfuzz flags fuzzer
This adds a selection of flags to numfuzz that are already used
for different testing variants or on clusterfuzz for
correctness testing.

No-Try: true
Change-Id: I79745b281b001f57d2b24977f3a8e9ce3bbab2a4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906573
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64884}
2019-11-11 14:20:08 +00:00
Liviu Rau
b990d2a4ad [testing] Show exit code hex value for failing tests
When a test fails the exit code of the process is printed
in the logs in the decimal format. On Windows searching for
the actual error is pretty difficult in this format since
we are dealing with negative values errors.
We print now the hex format of the same exit value next to
the original one.

Example failure of a test:
https://ci.chromium.org/p/v8/builders/try/v8_win64_dbg/b8897355488287117696
https://logs.chromium.org/logs/v8/buildbucket/cr-buildbucket.appspot.com/8897355488287117696/+/steps/Check/0/stdout
https://logs.chromium.org/logs/v8/buildbucket/cr-buildbucket.appspot.com/8897355488287117696/+/steps/Check/0/logs/RunWasmInterpreter_Int32Const_many/0

Bug: v8:9810
Change-Id: I51b65d6105bda6516707a764dbd41fd36c9dfc80
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1903975
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64878}
2019-11-11 11:15:41 +00:00
Michael Achenbach
76c57b5630 [numfuzz] Add cpu-features extra flags to numfuzz
This combines flags switching off CPU-features randomly with all
other flags on the numfuzz fuzzers.

In a follow up we can add also other interesting flags for testing
more features.

Bug: chromium:1021463
No-Try: true
Change-Id: Ia27d1b8b82dff2bf497242b30ec0b6ef2dfd722d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1905846
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64858}
2019-11-08 13:00:41 +00:00
Michael Achenbach
09d4a426ba [tools] Retain python2 compatibility with divisions in test runner
Bug: v8:9871
Change-Id: Ib2042002c1eba72b81fcd15811a627b13e7e3281
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1866770
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64408}
2019-10-21 08:47:46 +00:00
Liviu Rau
b3270b82eb Avoid output timeout when using progress indicator ci
Using test runner with option --progress=ci can generate
output timeouts in an actual CI environment. To avoid
that we gonna write a timestamp in the standard output
at every minute.

Bug: v8:9146
Change-Id: Id2f05530956b01d9b07809e509cd0cefc0be54b2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1863196
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64311}
2019-10-16 09:36:16 +00:00
Michael Achenbach
201c949bdd [test] Remove deprecated buildbot flag
Also remove or adapt all mentioning of buildbot. The flag was removed
on the infra side here:
https://crrev.com/c/1827557

Change-Id: I8a6be80faa65f99cd42a084c804fc5bfae7f02c2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1826722
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64017}
2019-09-27 12:13:06 +00:00
Liviu Rau
580da898dd Added a new CIProgressIndicator (--progress=ci)
The new indicator behaves like verbose without
printing the names of tests that passed. Also
a new option (--ci-test-completion=/path) was
added to represent a file where we can collect
test completion messages.

Bug: v8:9146
Change-Id: I0f1bbef4036a3019b60b094687b734d3d33a5915
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1806916
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63837}
2019-09-17 13:29:47 +00:00
Michael Achenbach
cdfadf4a99 Reland "[test] Print hanging tests on linux on test-runner termination"
This is a reland of 3fc9663159

The private method on_event in timeout proc is now renamed to be truly
private.

Original change's description:
> [test] Print hanging tests on linux on test-runner termination
>
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
>
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}

Bug: v8:8292
Change-Id: Ibad1172666d6f4d2c07884a54edfe9d6499b57fe
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735318
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63072}
2019-08-05 11:50:18 +00:00
Michael Achenbach
cfdbab36d9 Revert "[test] Print hanging tests on linux on test-runner termination"
This reverts commit 3fc9663159.

Reason for revert: _on_event clashes with a the same function in timeout proc.

Original change's description:
> [test] Print hanging tests on linux on test-runner termination
> 
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
> 
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}

TBR=machenbach@chromium.org,tmrts@chromium.org

Change-Id: Ic29013e92c77a9905e06ee13a2449e033285eb36
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8292
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735317
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63066}
2019-08-05 10:52:53 +00:00
Michael Achenbach
3fc9663159 [test] Print hanging tests on linux on test-runner termination
This will print the list of processes still running before and after
joining workers during termination. This will help debugging hanging
tests during flake-bisect or with num-fuzzer, which both terminate
on total timeout and currently still sometimes hang without printing
processes.

Bug: v8:8292
Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63065}
2019-08-05 10:21:48 +00:00
Michael Achenbach
d533da3a36 [test] Be more verbose when killing hanging tests
TBR=tmrts@chromium.org

Bug: v8:9145,v8:9098
Change-Id: Ib2170819f6d9aa00174fefb6bef4b0a27c11acfc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1578503
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60956}
2019-04-23 16:48:23 +00:00
Michael Achenbach
72b28e658a [test] Dump processes when test driver hangs
This prints the current v8-specific processes on linux whenever the
test driver emits a heart beat (i.e. no output for 30 seconds).

This is to investigate the cause of currently hanging tests on linux.

Bug: v8:9145
Change-Id: I857bb6d1c5f0b0917c64cdc0aa6076c6633f9dd6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1578438
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60941}
2019-04-23 10:21:31 +00:00
Michael Achenbach
98cde3f400 [test] Expose test-framework name in test results
This will allow to distinguish between the standard runner and the num fuzzer
on the infra side when generating flako command lines. The value could be
inferred, but it'd be more confusing.

Bug: v8:8971
Change-Id: I78f5104135d1c7fd7d98bceb4b17897e79421455
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1564050
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60769}
2019-04-11 11:34:53 +00:00
Michael Achenbach
9571654d2d [test] Debug print on test runner timeout
TBR=tmrts@chromium.org
NOTRY=true

Bug: v8:9098
Change-Id: I69d9cd21ad70357a77b45e97393949f1cfbab529
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1557137
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60669}
2019-04-07 14:53:48 +00:00
Michael Achenbach
ff0b9dd30e [test] Expose variant flags in json output
For standard test failures, the variant name is enough to deduce the
full command line of the test. This is used to create the flako
command line on the infra side.

Test failures from numfuzz have additional variant_flags, calculated
by the fuzzer, which don't match a variant name. Exposing those in
the test results will enable printing a proper flako command line
on infra side for numfuzz cases.

Bug: v8:8971
Change-Id: Ie47d42a0b34037da458b474f2a9ab38f1a5d238a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1554689
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60656}
2019-04-05 13:52:09 +00:00
Tamer Tas
a6426cb888 [testrunner] remove the extra characters in progress indicator
After python3 migration, the new print usage started causing leftover character
issues.

This CL fixes the print usage.

R=clemensh@chromium.org,neis@chromium.org
CC=machenbach@chromium.org

Bug: v8:8918
Change-Id: Ibee06677c3bae3e1141579693aa16a539309a566
Reviewed-on: https://chromium-review.googlesource.com/c/1495558
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59961}
2019-03-01 09:37:37 +00:00
Clemens Hammacher
5ebb852793 [testrunner] Print relative test command
This makes the output a lot smaller and thus makes it easier to see the
relevant part of the command.

R=machenbach@chromium.org

Change-Id: I62ac7218be4a02f0270a2d88a2f69b6ced45a041
Reviewed-on: https://chromium-review.googlesource.com/c/1491597
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59931}
2019-02-28 08:13:38 +00:00
Vadim Gorbachev (bmsdave)
6cad46b4f3 Preparing v8 to use with python3 /tools/testrunner
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds

This CL was uploaded by git cl split.

Bug: v8:8594
Change-Id: I2a90aaecb270f03aed1c0fc92da1a0e2621b0eb2
Reviewed-on: https://chromium-review.googlesource.com/c/1470101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59678}
2019-02-19 09:23:11 +00:00
Tamer Tas
baeb4e324d [testrunner] enable the progress indicator
Using test generators meant that we had to remove the progress indicator since
the total number of tests weren't known before-hand.

This CL implements a progress indicator using test number estimations.

cctest and unittests progress indicator is accurate, however estimating
means the progress will terminate over 100% in big test suites and sometimes
under 100%.

R=machenbach@chromium.org
CC=​sergiyb@chromium.org,yangguo@chromium.org

Bug: v8:8769
Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
Reviewed-on: https://chromium-review.googlesource.com/c/1460471
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59538}
2019-02-12 16:03:17 +00:00