This is a reland of 3fc9663159
The private method on_event in timeout proc is now renamed to be truly
private.
Original change's description:
> [test] Print hanging tests on linux on test-runner termination
>
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
>
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}
Bug: v8:8292
Change-Id: Ibad1172666d6f4d2c07884a54edfe9d6499b57fe
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735318
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63072}
This reverts commit 3fc9663159.
Reason for revert: _on_event clashes with a the same function in timeout proc.
Original change's description:
> [test] Print hanging tests on linux on test-runner termination
>
> This will print the list of processes still running before and after
> joining workers during termination. This will help debugging hanging
> tests during flake-bisect or with num-fuzzer, which both terminate
> on total timeout and currently still sometimes hang without printing
> processes.
>
> Bug: v8:8292
> Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63065}
TBR=machenbach@chromium.org,tmrts@chromium.org
Change-Id: Ic29013e92c77a9905e06ee13a2449e033285eb36
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8292
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735317
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63066}
This will print the list of processes still running before and after
joining workers during termination. This will help debugging hanging
tests during flake-bisect or with num-fuzzer, which both terminate
on total timeout and currently still sometimes hang without printing
processes.
Bug: v8:8292
Change-Id: I124b65fa35b8d7a6aa198fcf50f2c20df94dc51a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735312
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63065}
I noticed the indentation was off in one function, but also fixed
all the other flake8 issues in this file.
Change-Id: I2303ed87da7154484a872315f8355f57621514c4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1697054
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sam Clegg <sbc@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62669}
With lazy feedback allocation and bytecode flushing we need to call
%PrepareFunctionForOptimize before we call %OptimizeFunctionOnNextCall/
%OptimizeOsr. This cl:
1. Adds an additional state in pending optimized table to check if the
optimization was triggered manually.
2. Changes the compilation pipeline to delete the entry from pending
optimized table only if the optimization was triggered through
%OptimizeFunctionOnNextCall / %OptimizeOsr.
3. Adds a check to enforce %PrepareFunctionForOptimize was called.
4. Adds a new run-time flag to only check in the d8 test runner. We
don't want this check enabled in other cases like clusterfuzz that doesn't
ensure %PrepareFunctionForOptimize is called.
Bug: v8:8394, v8:8801, v8:9183
Change-Id: I9ae2b2da812e313c746b6df0b2da864c2ed5de51
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1664810
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62653}
Instead of conditionally normalizing file paths, go back to using
the basename in the dynamically-generated regular expression
pattern.
Bug: v8:7950, v8:9395, v8:9406
Change-Id: I57d2449a8229a67b038c7fcffe36218848d9a575
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1681122
Reviewed-by: Sigurd Schneider <sigurds@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62439}
d8 treats files with the .mjs extension as modules instead of
classic scripts. Thus, the `// MODULE` pragma and its corresponding
logic in test runners can be removed in favor of explicitly adding
the extension.
Bug: v8:7950, v8:9395, v8:9406
Also-By: tmrts@chromium.org
Change-Id: Ic74328dc5c5f176bb4bdf6d74bdd4d3966279ba5
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1675958
Commit-Queue: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Auto-Submit: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62421}
testrunner assumes that each test suite has a single extension for base tests.
".mjs" extension can be used for ECMAScript modules in addition to the standard
extension ".js" we use for the base tests.
This CL generalizes the {TestLoader} to accept multiple extensions for
a single test suite.
R=mathias@chromium.orgTBR=machenbach@chromium.org
CC=gsathya@chromium.org
Bug: v8:9395
Change-Id: Ibc155f4963472fe9f989458cd839f3642ffbddea
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1675961
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Mathias Bynens <mathias@chromium.org>
Auto-Submit: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62356}
TBR=tmrts@chromium.org
Also skip some tests too slow in full debug mode.
Bug: v8:9145
Change-Id: Ied8781be26d2c1efd7720e333775da9f6d632236
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1598759
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61281}
The "stress_background_compile" variant runs on all our bots. We
combine it with testing wasm code GC (which kind of fits into
background compile stressing) to get more coverage for that. Both
features are orthogonal, so we can test both at the same time without
loosing any coverage.
R=machenbach@chromium.orgCC=rmcilroy@chromium.org
Bug: v8:8217
Change-Id: Ib17decd4869978ff98e302694fa73d70ceec120e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1588472
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61151}
This makes the gc-stress bots set the '--stress-wasm-code-gc' flag.
Note that this also implicitly enables wasm code gc on these bots.
R=mstarzinger@chromium.org, machenbach@chromium.org
Cq-Include-Trybots: luci.v8.try:v8_linux_gc_stress_dbg
Cq-Include-Trybots: luci.v8.try:v8_mac64_gc_stress_dbg
Bug: v8:8217
Change-Id: Idc4fb996f50632d5621200e138f3ddc572ed9b05
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1585721
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61089}
This prepares removing the special timeout of 200 on the bots. First
we temporarily set 200 on v8 side to remove the flag on the infra side.
Afterwards we'll remove the v8-side code, tested in CQ.
TBR=tmrts@chromium.org
Bug: v8:9145
Change-Id: I61578da02ab8c101d0d6c916106ad5a8bc6841cb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1581259
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60962}
This is part of the refactoring to allow exporting more information about test
execution to the recipes and upload this information to ChromePerf.
This fixes running secondary even after primary run fails, which will allow us
to differentiate between test and infra failures as latter ones will also affect
refbuilds and re-runs without patch.
R=machenbach@chromium.org, tmrts@chromium.org
Bug: chromium:841700
Change-Id: I29ce49d2f8c5e73158f1d41a73c51f2b35929f36
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1570006
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60946}
This prints the current v8-specific processes on linux whenever the
test driver emits a heart beat (i.e. no output for 30 seconds).
This is to investigate the cause of currently hanging tests on linux.
Bug: v8:9145
Change-Id: I857bb6d1c5f0b0917c64cdc0aa6076c6633f9dd6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1578438
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60941}
This enable test suites to check which test driver framework is used.
When using number fuzzer on mjsunit, we add a JS file that
switches off the optimization-state assertions.
Checking intrinsic state is not feasible on the number fuzzer and in
the past, we needed to skip tests on demand, which is a maintenance
burden. The main function of the fuzzer, to check for dcheck errors and
tsan issues, is retained.
Bug: v8:9127
Change-Id: I699b85d5f7c9aaed337a2130d9eddc160c059d7b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1565892
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60816}
This will allow to distinguish between the standard runner and the num fuzzer
on the infra side when generating flako command lines. The value could be
inferred, but it'd be more confusing.
Bug: v8:8971
Change-Id: I78f5104135d1c7fd7d98bceb4b17897e79421455
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1564050
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60769}
Since wasm optimizations are independent of JS optimizations, we can
just combine the "no_liftoff" variant with the existing "stress"
variant (which has the similar "--always-opt" option for JS), and add
a "Liftoff only" variant as part of "nooptimization".
This gives more coverage to find bugs like https://crrev.com/c/1543354
more easy.
R=mstarzinger@chromium.org, machenbach@chromium.org
Change-Id: I81bb22074c59dcb650a05252da43a4170cd467ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1559740
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60714}
For standard test failures, the variant name is enough to deduce the
full command line of the test. This is used to create the flako
command line on the infra side.
Test failures from numfuzz have additional variant_flags, calculated
by the fuzzer, which don't match a variant name. Exposing those in
the test results will enable printing a proper flako command line
on infra side for numfuzz cases.
Bug: v8:8971
Change-Id: Ie47d42a0b34037da458b474f2a9ab38f1a5d238a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1554689
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60656}
Reading --verify_predictable makes me sad, whereas --verify-predictable
makes me happy. This CL introduces more happiness.
R=machenbach@chromium.org
Bug: v8:8834
Change-Id: Id51a75f32e6d5a2f87aed81e058a8b6dff189758
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1550399
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60599}
The next built-in function throws a "StopIteration" when iterator is exhausted
which will crash and stop the test runner. Asking it to return a "None" instead will
fix the issue.
Change-Id: I3ca4ff8f28f359798c9faf6f1ffbbaa6bb95f156
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1524719
Commit-Queue: Milad Farazmand <miladfar@ca.ibm.com>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60294}
This makes the test runner and numfuzz share the same exit code behavior on
errors. This is needed as they also share the same infrastructure logic
to collect swarming tasks.
Bug: chromium:937228
Change-Id: I155b37c7b10dd22959a4dcf30bbd0321c452236b
Reviewed-on: https://chromium-review.googlesource.com/c/1495987
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59976}
After python3 migration, the new print usage started causing leftover character
issues.
This CL fixes the print usage.
R=clemensh@chromium.org,neis@chromium.org
CC=machenbach@chromium.org
Bug: v8:8918
Change-Id: Ibee06677c3bae3e1141579693aa16a539309a566
Reviewed-on: https://chromium-review.googlesource.com/c/1495558
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59961}
This makes the output a lot smaller and thus makes it easier to see the
relevant part of the command.
R=machenbach@chromium.org
Change-Id: I62ac7218be4a02f0270a2d88a2f69b6ced45a041
Reviewed-on: https://chromium-review.googlesource.com/c/1491597
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59931}
WebAssembly's specification tests use a timeout annotation.
This change allows the shared testrunner to use it to calculate
the testcase timeout.
Currently, the allowed timeout values are: long. Other values will
emit a warning.
Change-Id: Id7f453f5fd49854c8f53ff86ef2ec58aa0ae8748
Reviewed-on: https://chromium-review.googlesource.com/c/1480376
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sven Sauleau <ssauleau@igalia.com>
Cr-Commit-Position: refs/heads/master@{#59919}
Also enable test runner to differentiate between clang and gcc.
Bug: v8:8919
Change-Id: Icdcae0aba3644a1b1b9ddc6c037eabde27d717f7
Reviewed-on: https://chromium-review.googlesource.com/c/1491634
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59906}
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
This CL was uploaded by git cl split.
Bug: v8:8594
Change-Id: I2a90aaecb270f03aed1c0fc92da1a0e2621b0eb2
Reviewed-on: https://chromium-review.googlesource.com/c/1470101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59678}
Progress indicator calculates the percentage using the estimated amount of
tests.
When base tests produce more tests or when testrunner filters some tests, the
percentage terminates over 100% or under it.
This CL adds an informative message about how the percentage behaves.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8728
Change-Id: I91cafd2579ea1894ac347ff7483c307cd46c545d
Reviewed-on: https://chromium-review.googlesource.com/c/1477056
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59660}
This also cleans up some OWNERS files.
NOTRY=true
Change-Id: Ic49ecee02bb3e339dc4c0de4ba69f00c36c076aa
Reviewed-on: https://chromium-review.googlesource.com/c/1475470
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59622}
Using test generators meant that we had to remove the progress indicator since
the total number of tests weren't known before-hand.
This CL implements a progress indicator using test number estimations.
cctest and unittests progress indicator is accurate, however estimating
means the progress will terminate over 100% in big test suites and sometimes
under 100%.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8769
Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
Reviewed-on: https://chromium-review.googlesource.com/c/1460471
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59538}
Testrunner runs variants of a test sequentially without taking sharding into
account. A slow test with slow variants slows down the whole test run no matter
the sharding configuration.
This CL implements a test hashing algorithm and variant sharding for test
variants.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I15f8c547fa2f361fb6c53bf8d5df055d3df38d3e
Reviewed-on: https://chromium-review.googlesource.com/c/1458016
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59530}
The test name alone isn't informative alone.
This CL prints the variant name during the progress reporting.
tested by running test262 with all variants
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8728
Change-Id: I3d7d1fc6eacef3712d87404c7276fd5523bb445a
Reviewed-on: https://chromium-review.googlesource.com/c/1458156
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59437}
V8 test suites return tests by loading them into memory up-front. Up-front
loading has memory and loading-time overhead for the testrunner.
This CL converts the test-suites to load tests during the test run in parallel.
After CL, most test suites start running immediately and all test suites start
running after 3 seconds on a modern SSD.
This CL eliminates the 200 MiB memory spikes that causes memory problems in mac
minis.
Overhead of running test262 before CL:
Line # Mem usage Increment Line Contents
=================================================
264 225.760 MiB 195.8.000 MiB tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
Overhead of running test262 after CL:
Line # Mem usage Increment Line Contents
================================================
264 28.840 MiB 0.000 MiB tests = self._load_testsuite_generators(args, options)
-------------------------------------------------
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174, v8:8552, v8:8728
Change-Id: Iab540b9410239b05dc80b4a5228db25d8b8fb3b9
Reviewed-on: https://chromium-review.googlesource.com/c/1454478
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59408}
This is a reland of 81eec150f6
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
Bug: v8:8174, v8:8790
Change-Id: I38ab9d37bca76057441a970f26e2102e4387a857
Reviewed-on: https://chromium-review.googlesource.com/c/1454724
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59387}
This reverts commit 81eec150f6.
Reason for revert: windows mozilla test failures
Original change's description:
> Reland "[test] refactor testsuite configuration"
>
> This is a reland of 7f92ad0ab6
>
> Original change's description:
> > [test] refactor testsuite configuration
> >
> > Every testsuite configuration consist of at least 30% code duplication.
> >
> > The code age ranges from 10 years old to 5 years old. Implementing anything that
> > touches the testsuite code becomes a technical fight to the death.
> >
> > This CL removes all the duplication by refactoring the common functionality.
> >
> > This CL contains structural changes without any logical changes % small bug
> > fixes.
> >
> > R=machenbach@chromium.org
> > CC=yangguo@chromium.org,sergiyb@chromium.org
> >
> > Bug: v8:8174, v8:8769
> > Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> > Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#59361}
>
> Bug: v8:8174, v8:8769
> Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
> Reviewed-on: https://chromium-review.googlesource.com/c/1454485
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59369}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org
Change-Id: I8f5650b5f46be299c004e2fa8b708fa2c17a4dc2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454607
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59370}
This is a reland of 7f92ad0ab6
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
Bug: v8:8174, v8:8769
Change-Id: I8e7078cfb875ceb3777e57084e6f8dfac09693e7
Reviewed-on: https://chromium-review.googlesource.com/c/1454485
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59369}
This reverts commit 7f92ad0ab6.
Reason for revert: https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Win32/19148
Original change's description:
> [test] refactor testsuite configuration
>
> Every testsuite configuration consist of at least 30% code duplication.
>
> The code age ranges from 10 years old to 5 years old. Implementing anything that
> touches the testsuite code becomes a technical fight to the death.
>
> This CL removes all the duplication by refactoring the common functionality.
>
> This CL contains structural changes without any logical changes % small bug
> fixes.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174, v8:8769
> Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
> Reviewed-on: https://chromium-review.googlesource.com/c/1445881
> Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59361}
TBR=machenbach@chromium.org,sergiyb@chromium.org,tmrts@chromium.org,v8-reviews@chromium.org
Change-Id: I473f0d4c6b9c0239923b8c03699dbc38b7f85030
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174, v8:8769
Reviewed-on: https://chromium-review.googlesource.com/c/1454599
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59362}
Every testsuite configuration consist of at least 30% code duplication.
The code age ranges from 10 years old to 5 years old. Implementing anything that
touches the testsuite code becomes a technical fight to the death.
This CL removes all the duplication by refactoring the common functionality.
This CL contains structural changes without any logical changes % small bug
fixes.
R=machenbach@chromium.orgCC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174, v8:8769
Change-Id: Iee299569caa7abdc0307ecf606136669034a28a2
Reviewed-on: https://chromium-review.googlesource.com/c/1445881
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59361}
This reverts commit 6e03d7ee42.
Reason for revert: This breaks the Android bot:
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Android%20Arm64%20-%20N5X/2933
It's quite hidden due to https://crbug.com/v8/8731 but all shards
time out. It looks like to to this change, testing takes
dramatically longer, maybe the pushing takes now much longer than
before. If we want decide for this, the builder needs to get
many more shards.
Original change's description:
> Reland "[tools] Push files using high-level device.PushChangedFiles method"
>
> This is a reland of d045f66682
>
> Original change's description:
> > [tools] Push files using high-level device.PushChangedFiles method
> >
> > R=machenbach@chromium.org
> >
> > No-Try: true
> > Bug: chromium:893593
> > Change-Id: I11cce7694eb7755ccee42c9a342fc1aa22663d85
> > Reviewed-on: https://chromium-review.googlesource.com/c/1382468
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#58407}
>
> Bug: chromium:893593
> Change-Id: I88a7143b3f31d87d266b89221f81efe831ea3823
> Reviewed-on: https://chromium-review.googlesource.com/c/1443055
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59221}
TBR=machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,bpastene@chromium.org,jbudorick@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:893593
Change-Id: Ifea307b5de8f39b660966fc6bef54601df91d841
Reviewed-on: https://chromium-review.googlesource.com/c/1450119
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59305}
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.
This CL eliminates that overhead by utilizing generators and threading.
LoadingProc now loads test after receiving the results of the loaded tests.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174,v8:8731
Change-Id: Ifee79c3e213da568f092de0f1623016174e9410c
Reviewed-on: https://chromium-review.googlesource.com/c/1439240
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59223}
Procs return the result by increasing recursion through result_for.
This CL eliminates that mechanism from the Processor interface and uses boolen
return values for sending tests to signal success or the failure to load the
test into the execution queue.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174,v8:8731
Change-Id: I073a86ca84bcf88da11132b90013d4c8455bc61e
Reviewed-on: https://chromium-review.googlesource.com/c/1439239
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59201}
This adds the new runtime flag as a variant and runs it on a subset of
builders corresponding to the "extra" testing set.
Currently failing tests are skipped in the new variant.
After https://crrev.com/c/1433777 this costs only little additional
resources.
Bug: v8:8678
Change-Id: Ibd0e38872814d11252e55a7c6a58d313aa84ebe3
Reviewed-on: https://chromium-review.googlesource.com/c/1433774
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59080}
This reverts commit 25457c60a7.
Reason for revert: https://crbug.com/v8/8731
Original change's description:
> [testrunner] load tests concurrently into test execution processor
>
> loading every test up-front into the processing queue costs about 224MB for a
> x64 testsuite run.
>
> This CL eliminates that overhead by utilizing generators and threading.
>
> LoadingProc now loads test after receiving the results of the loaded tests.
>
> R=machenbach@chromium.org
> CC=yangguo@chromium.org,sergiyb@chromium.org
>
> Bug: v8:8174
> Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
> Reviewed-on: https://chromium-review.googlesource.com/c/1420678
> Commit-Queue: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59056}
TBR=machenbach@chromium.org,tmrts@chromium.org
Change-Id: I1e074a031dced367a32a93827b9e863b0331340f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8174
Reviewed-on: https://chromium-review.googlesource.com/c/1433792
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59065}
Previously, trusted (or no-mitigations) has been tested on a subset of builders
from all platforms. This reduces it to arm-sim and native Android devices.
Change-Id: I90066686e6a92db4a944025538e01a117f324421
Reviewed-on: https://chromium-review.googlesource.com/c/1433777
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59062}
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.
This CL eliminates that overhead by utilizing generators and threading.
LoadingProc now loads test after receiving the results of the loaded tests.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I8f4e6de38430c54fe126e4504b52851866769efb
Reviewed-on: https://chromium-review.googlesource.com/c/1420678
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59056}
Testrunner has ancient support for JUnit compatible XML output.
This CL removes this old feature.
R=mstarzinger@chromium.org,jgruber@chromium.org,jkummerow@chromium.org
CC=machenbach@chromium.org
Bug: v8:8728
Change-Id: I7e1beb011dbaec3aa1a27398a5c52abdd778eaf0
Reviewed-on: https://chromium-review.googlesource.com/c/1430065
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59045}
Moving to a lazy test loading approach makes counting the total number of tests
non-trivial.
For CI runs, we output the total number of tests after the run.
For users, progress indicator signals the status of the run.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I0731964515aac60a3629acee6c7243433a2b4e04
Reviewed-on: https://chromium-review.googlesource.com/c/1420677
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58944}
V8 testrunner is loading every test it has to run into memory greedily in order
to sort by slowness of the test case. The memory and CPU overhead for loading
the test-suites are non-trivial.
This CL restructures it by changing the sorting method.
R=machenbach@chromium.org
CC=sergiyb@chromium.org,yangguo@chromium.org
Bug: v8:8174
Change-Id: I08331182147b92cf4ac54823eea0e2b472f51e84
Reviewed-on: https://chromium-review.googlesource.com/c/1406684
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58821}
TestSuite has a static method LoadTestSuite that should properly configure the
TestSuite instance (i.e. loaded status files and tests), however the method
leaves some configuration logic to the caller.
The leaky abstraction causes the caller to do a bunch of loading operations (see
the removed methods in base_runner.py).
This CL isolates the TestSuite loading logic to the static method only.
This is a refactoring only change without any intended logical changes.
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: I105059c9c9e050f03bb584174e2bd7ceeae2b228
Reviewed-on: https://chromium-review.googlesource.com/c/1396417
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58744}
- Removed the old test.
- Created a fake test suite and added a test for loading it with a TestConfig
R=machenbach@chromium.org
CC=yangguo@chromium.org,sergiyb@chromium.org
Bug: v8:8174
Change-Id: Ib7587ceec9e31ecd4cb8f45c3158e73c79a9bc5b
Reviewed-on: https://chromium-review.googlesource.com/c/1396082
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58593}
Also disables --stress-flush-bytecode on some mjsunit tests which fail
when bytecode flushing is stressed due to test invariants.
Bug=v8:8395
Change-Id: If627910214b3c266e7776340ba182829148e8289
Reviewed-on: https://chromium-review.googlesource.com/c/1372071
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58230}
The additional variant does not use wrapper disabling phase tests and negated
outcome processor. This allows to ensure that tests marked FAIL_PHASE_ONLY, do
actually fail without it.
R=machenbach@chromium.org
Bug: v8:8467
Change-Id: I66e07bd7107520872cc013bf0f33fdc6664baf56
Reviewed-on: https://chromium-review.googlesource.com/c/1361164
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58211}
This commit adds a --run-skipped flag to the test runner that will
bypass the 'SKIP' status.
Bug: v8:8485
Change-Id: Iac012bdaf2de6b0f8e44ed3a65bc9330709527bb
Reviewed-on: https://chromium-review.googlesource.com/c/1346490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Sigurd Schneider <sigurds@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57710}
This reverts commit f401cd4b2c.
Reason for revert: did not help
Original change's description:
> [tools] Re-land: Add retries when trying to discover the device
>
> This is an experimental change that may help mitigate the issue.
>
> TBR=machenbach@chromium.org
>
> No-Try: true
> No-Tree-Checks: true
> Bug: chromium:893593
> Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
> Reviewed-on: https://chromium-review.googlesource.com/c/1339401
> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#57558}
TBR=machenbach@chromium.org,sergiyb@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:893593
Change-Id: I30c529a627d1e6fa52099939c5c209110e9d0eb9
Reviewed-on: https://chromium-review.googlesource.com/c/1342931
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57650}
This is an experimental change that may help mitigate the issue.
TBR=machenbach@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Idf15a63006c2c7ba2c31482e5103b2a0b1d64510
Reviewed-on: https://chromium-review.googlesource.com/c/1339401
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57558}
This is an experimental change that may help mitigate the issue.
TBR=machenbach@chromium.org
No-Try: true
No-Tree-Checks: true
Bug: chromium:893593
Change-Id: Ideb74a83b9937dbe917e8c7c93305d9824b48a93
Reviewed-on: https://chromium-review.googlesource.com/c/1339419
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57556}
Also add the ability to skip tests only when --optimize-for-size is passed.
TBR=sergiyb@chromium.org
Bug: v8:7819
Change-Id: I1dcc26ea8664d014b6c50f0d636c13bf21e26ff2
Reviewed-on: https://chromium-review.googlesource.com/c/1328945
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57396}
'simulator_run' is now used exclusively in test/*/*.status.
Change-Id: I501b7ffd19e2476f4c803ed3d25cba69d67039fa
Reviewed-on: https://chromium-review.googlesource.com/c/1318610
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57300}
This is a reland of a12203c64b
Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
>
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
>
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}
Change-Id: I25fcd95bfc1aee1d21da390359423e5dfed112a4
Reviewed-on: https://chromium-review.googlesource.com/c/1286952
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Ben Smith <binji@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56791}
This reverts commit a12203c64b.
Reason for revert: Breaks isolate_tests
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux%20-%20builder/36777
Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
>
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
>
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}
TBR=binji@chromium.org,machenbach@chromium.org,yangguo@chromium.org,ahaas@chromium.org,clemensh@chromium.org,mathias@chromium.org
Change-Id: I2edd0ca94cb5990322571879c81671fa835f3ecd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/1286526
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56746}
These changes were necessary to run with the new style of jsapi tests
introduced in https://github.com/WebAssembly/spec/pull/883.
Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
Reviewed-on: https://chromium-review.googlesource.com/c/1277724
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56745}
The regexp for finding resources to be pushed to Android devices was
too lax. On empty strings it tried to check for more resources on a
directory and hung.
The last test262 roll contains tests with empty imports that started
hanging in this way.
TBR=neis@chromium.org
NOTRY=true
Bug: v8:7834
Change-Id: Ie58f1b18bdd99b7b40c1fb39b25e2f481932e0f3
Reviewed-on: https://chromium-review.googlesource.com/c/1270579
Reviewed-by: Georg Neis <neis@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56465}
The cctest and unittests suites call the test executable to list the
tests, which requires pushing the executables to the device on Android.
NOTRY=true
TBR=sergiyb@chromium.org
Bug: chromium:866862
Change-Id: I318dff7af2b2de3b8642ec4b6ba30b602da808bb
Reviewed-on: https://chromium-review.googlesource.com/1213202
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55723}
Path names in import/export statements are relative to the file they are in.
This fixes the logic and unblocks using the messages test suite on Android,
which has cases importing files from mjsunit, which import more files from
there.
Bug: chromium:866862
Change-Id: I8d2ff645f69b67fbdaf4a622d06308e55298b0ce
Reviewed-on: https://chromium-review.googlesource.com/1206570
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55643}
This shares logic for finding additional resources in JS source code.
Previously the logic was implemented for mjsunit, now it will be used
across all d8-based test cases.
This'll enable adding those test suites for Android testing.
Bug: chromium:866862
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I7c89ba141483aaf692a03c0e168edb61bbb7b010
Reviewed-on: https://chromium-review.googlesource.com/1193873
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55503}
This is a reland of 4c0943424c
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
Bug: chromium:866862
Change-Id: Icf7e04c75d4abeab7254d10ba21240e46b0022ae
Reviewed-on: https://chromium-review.googlesource.com/1170643
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55059}
This reverts commit 4c0943424c.
Reason for revert: Unfortunately this broke all perf builders.
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org
Change-Id: If80129810586b709dab762c9b5724888e15daec2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:866862
Reviewed-on: https://chromium-review.googlesource.com/1170962
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55058}
This adds a new command abstraction for running commands on Android
using dockered devices on swarming.
The new abstraction handles pushing all required files to the device.
The logic used for pushing and running is reused from the perf runner.
This adds only the mjsunit test suite. Others will be handled in
follow up CLs. The suite logic is enhanced with auto-detection of files
to be pushed to devices, for e.g. load or import statements.
Some test cases need an extra resource section for specifying required
files.
Remaining failing tests are marked in the status files for later
triage.
Bug: chromium:866862
Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
Reviewed-on: https://chromium-review.googlesource.com/1150153
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55041}
The test driver compares command-line arguments against test names
it finds on disk. Using Python's "fnmatch" for this nicely handles
wildcards, but is relatively slow. For given test names that don't
contain any '*', we can use string equality testing, which is much
faster.
Example: the time to evaluate
tools/run-tests.py --arch x64 --mode release \
$(grep 'object-spread' -l -r test/test262/data/test/ | \
sed -E 's|\.js$||' | \
sed -E 's|^test/test262/data/test/|test262/|')
goes from "I gave up and killed the process after several minutes"
to a couple of seconds with this patch.
Change-Id: I9ec404b7516afd801fe6126347f6dff533d1977c
Reviewed-on: https://chromium-review.googlesource.com/1149196
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54704}
Before flipping the flag, some tests need to be adapted. This CL
prepares these tests, such that the flag flip CL really just flips a
flag.
R=titzer@chromium.org, hablich@chromium.org
Bug: v8:6600, chromium:787421
Change-Id: I8030df69cda5f3fb81354350a37f65c0d1c669bd
Reviewed-on: https://chromium-review.googlesource.com/1110363
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53959}
This is to speed up slow nosnap runs. Allocation verification is covered by
running mksnapshot on other builders.
Change-Id: I9c286f7dd4abac9cf1be45be4a483a7b36d09f8a
Reviewed-on: https://chromium-review.googlesource.com/1059113
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53234}
D8 enables the Wasm trap handler by default now, but we need to make sure the
older bounds check case still gets test coverage too, as bounds checks will
continue to be a supported configuration.
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I5b0bdded6929a9b3a8480e87d038398b8d2a0fd8
Reviewed-on: https://chromium-review.googlesource.com/1048835
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53078}
This makes testing with random gc stress use the status file entrees
for gc fuzzing as well, since many test cases not suitable for one, are
not suitable for the other.
This also skips two more tests that rely on assert(Un)Optimized, which
is unreliable with gc fuzzing.
TBR=sigurds@chromium.org
Change-Id: I33a617b251d5cf65cf6e486d07ec55cde050b8ab
Reviewed-on: https://chromium-review.googlesource.com/1021082
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52701}
Embedded builtins are now based off the v8_enable_embedded_builtins
gn flag instead, which conditionally defines V8_EMBEDDED_BUILTINS.
Bug: v8:6666
Change-Id: I44d40d30fce3a3ed9bbf973d46c4990ba3fade40
Reviewed-on: https://chromium-review.googlesource.com/964361
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52092}
The feature in question has been enabled by default for a while and we
no longer need to maintain a configuration without it enabled. Note that
this change only removes the mechanical pieces. Further cleanup enabled
by this will be done as follow-ups.
R=clemensh@chromium.org
BUG=v8:7549
Change-Id: I90e5bcddabe74a18a4d2a88132e8dc93317bcff4
Reviewed-on: https://chromium-review.googlesource.com/958424
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51883}
We now unconditionally both parse and compile StreamedSource on the background
thread.
BUG=v8:5203
Change-Id: I42d6fe9059bc1745da3a415d270f46cf1c08b306
Reviewed-on: https://chromium-review.googlesource.com/948854
Reviewed-by: Mythri Alle <mythria@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51758}
This enables the v8_enable_embedded_builtins gn flag on non-ia32 builds
and adds a new --stress-off-heap-code test mode to fyi bots.
v8_enable_embedded_builtins=true changes accesses to constants and
external references to go through the root list in builtins code.
--stress-off-heap-code copies builtins code off-heap on isolate
creation.
A few drive-by-fixes:
- ensure that we actually inspect the correct builtin during
isolate-independence testing.
- relax tests to decrease maintenance (now we only fail if a builtin
should be isolate-independent but isn't).
- switch to a different off-heap-trampoline register on arm due to
conflicts with custom stub linkages.
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Bug: v8:6666
Change-Id: I09ad3c75cb4342f4c548ea780f275993730896c8
Reviewed-on: https://chromium-review.googlesource.com/934281
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51717}
Introducing a new status-file variable: mips_arch_variant. With this
variable, in status files will be possible to define selections which
are based on MIPS architecture revisions/variants.
TEST=
BUG=
Change-Id: Ifd682552db2f26be4e56dc94ad50bed063ff14c5
Reviewed-on: https://chromium-review.googlesource.com/941212
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#51647}
This is a partial revert of:
https://crrev.com/c/890938 and https://crrev.com/c/893982
Before this CL, the test runner blocked on ongoing tests in order to
process their results after an internal timeout. However, the logic
required for this feature was overly complicated and prevented an
acceptable implementation for fast aborts. Furthermore, also the fuzzers
suffered from timeouts on swarming due to hanging tests.
Instead, we now abort immediately on internal timeout (used on
fuzzers), SIGINT (Ctrl-C) and SIGTERM. Ongoing tests are immediately
terminated and their results are disregarded. On SIGTERM and SIGINT,
we return with non-zero exit codes, and zero on internal timeout.
This will also properly return json output, when the external hard
timeout is reached on swarming (causes SIGTERM).
TBR=sergiyb@chromium.org
Bug: v8:7423, chromium:813065
Change-Id: Ib20f835f58a0970693bdd3b21dc5d766d8e115d8
Reviewed-on: https://chromium-review.googlesource.com/924852
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51399}
This reverts commit 9808093ef6.
Reason for revert:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/19149
Original change's description:
> Update test configurations for code serializer
>
> Update test configurations on the bots to test for serializing
> after execute (--cache=after-execute) and producing full code
> cache (--cache=full-code-cache) options. We no longer need
> to test serializing before execute (--cache=code) on the bots.
>
> Bug: v8:7302
> Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
> Reviewed-on: https://chromium-review.googlesource.com/869931
> Commit-Queue: Mythri Alle <mythria@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#51235}
TBR=machenbach@chromium.org,yangguo@chromium.org,mythria@chromium.org
Change-Id: I47e03101804194c21b0edf418b744b0ccb66cbf6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7302
Reviewed-on: https://chromium-review.googlesource.com/913134
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51236}
Update test configurations on the bots to test for serializing
after execute (--cache=after-execute) and producing full code
cache (--cache=full-code-cache) options. We no longer need
to test serializing before execute (--cache=code) on the bots.
Bug: v8:7302
Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
Reviewed-on: https://chromium-review.googlesource.com/869931
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51235}
The suppression flag must be set before loading the tests to have an effect.
This got turned around in https://crrev.com/c/899366 and is fixed again by
this CL.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6917
Change-Id: Ie58465288a0d6eec9a99a23d610710de9e1cdddd
Reviewed-on: https://chromium-review.googlesource.com/911114
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51209}
This new variant is executed on a small number of bots as part of the
"extra" suite. It checks that the wasm-jit-to-native flag can still be
disabled if unexpected failures pop up on a release branch.
R=machenbach@chromium.org
Bug: v8:7417
Change-Id: I1658cb2f04302fa80915b59bfedd85d980742db5
Reviewed-on: https://chromium-review.googlesource.com/909213
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51204}
Timeouts on num-fuzzer only lead to false positives, as tests might
take unpredictably longer.
This CL forgives timeouts through a global override mechanism of the
expected outcomes. This allows to remove already existing scattered
code that allowed timeouts in some test suites only.
Bug: v8:6917
Change-Id: Ib131765d360e335789c1952bc6793ed051e016ea
Reviewed-on: https://chromium-review.googlesource.com/908454
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51188}
Load mjsunit.js inside the realm as otherwise the functions are not
available in the realm's scope.
This also prints timestamps after each test to easier track down slow
tests.
We also pass --omit-quit to not stop too early.
This also adds the ability to skip certain tests for endurance
fuzzing and skips some tests with known problems.
TBR=ulan@chromium.org,hpayer@chromium.org
Bug: v8:6972, v8:7400
Change-Id: I44464c28bfb10c84f2e59972e7b86945a47ca3b3
Reviewed-on: https://chromium-review.googlesource.com/899008
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51066}
Only test processors code left. It enabled to move more stuff to
the base runner, like progress indicators creation.
Bug: v8:6917
Change-Id: Ie6dd211cec561a07d92bcc4431ea88eb1842c8fa
Reviewed-on: https://chromium-review.googlesource.com/897624
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51030}
System tests updated to pass. I will cleanup the code in a
separate CL removing the old code.
Bug: v8:6917
Change-Id: Ie49efe06c43faade846e322b35c4a2068137a88b
Reviewed-on: https://chromium-review.googlesource.com/897565
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51025}
For mips, if 'mips_arch_variant=="r6"' and if 'mips_use_msa' flag is set
to 'true', then test-run-wasm-simd tests won't be skipped for mips. It
will also force 'MIPS_SIMD' bit in CpuFeatures to be set.
ARM processors are assumed to support SIMD.
Change-Id: Iea668b97ef995ca4949ddbf2ffc734aad89d3aa3
Reviewed-on: https://chromium-review.googlesource.com/868430
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#50981}
This is a reland of 0db74d4974.
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
This reverts commit 0db74d4974.
Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
This adds back an option for interrupt budget available in no-snap
builds. This also adds a fuzzer configuration for numfuzz that enables
fuzzing the interrupt budget option. A new flag --disable-analysis
allows to generally skip the fuzzer's analysis phase, which can be
chosen for interrupt budget, which doesn't support an analysis phase.
Bug: v8:6917
Change-Id: I546dd9ee41c3e0fb027108ef4606a34514f230d4
Reviewed-on: https://chromium-review.googlesource.com/885805
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50929}
Without realm-eval, the toplevel references in loaded tests are not
garbage-collected and changes in the tests can affect the global
object.
TBR=ulan@chromium.org
NOTRY=true
Bug: v8:7379, v8:7376
Change-Id: Id4d451e38fe49c6531a07d541ad3c67298b6acf2
Reviewed-on: https://chromium-review.googlesource.com/890519
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50915}
They have been replaced with num_fuzzer.
Bug: v8:6917
Change-Id: I2f78df308cec0a58f0d665bce82503dee68fcebc
Reviewed-on: https://chromium-review.googlesource.com/888641
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50903}
This is a reland of 8f4407fbd6.
Original change's description:
> [test] Keep the order of variants
>
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}
Bug: v8:6917
Change-Id: I9d4c617ccec6db8929af3095f2f27d8964c5d6b8
Reviewed-on: https://chromium-review.googlesource.com/888582
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50901}
This reverts commit 8f4407fbd6.
Reason for revert: Failing in old test runner. See comment.
Original change's description:
> [test] Keep the order of variants
>
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: Ia8608053e080c70d2ac09d7921a6abb6cd0e338f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/888749
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50895}
A timeout result of randomly combined tests is not a useful indicator.
Therefore we ignore those now. This also prevents timeouts from being
re-run.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6917
Change-Id: I8e5eb93e2f13067ca5270fb152c3dc1eda1bb3c2
Reviewed-on: https://chromium-review.googlesource.com/888559
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50879}
This will affect all manual test runs with x64. Most bots on x64 already
migrated.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:7343
Change-Id: I87f46f1848a813c0b320b3e9901481b9232025a5
Reviewed-on: https://chromium-review.googlesource.com/883101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50825}
This adds a tri-state flag --infra-staging and --no-infra-staging, which
can be used to explicitly opt in or out of the staging test runner.
If not specified, a new architecture whitelist will enable roll-out per
arch for manual (none CI) runs.
We'll start whitelisting archs in follow ups.
Bug: v8:7343
Change-Id: I1228e48969fd379f5c231a2b8fad4afc01da94c0
Reviewed-on: https://chromium-review.googlesource.com/881169
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50812}
Introduce option to run fuzzer processor without analysis phase.
It will be used in fuzzing combined tests.
Bug: v8:6917
Change-Id: Ic96d6b8c5a35c81da80340555bdd75c0d518cb5a
Reviewed-on: https://chromium-review.googlesource.com/880948
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50810}
The extra output on mac was to investigate a bug that's now fixed.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6927
Change-Id: Iac8074c7e89a987e164f676442da1ed4f93987cc
Reviewed-on: https://chromium-review.googlesource.com/880623
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50787}
Since we're not winning anything by changing the result between
processors on the main process, reduce is noop there and result is
immutable.
Bug: v8:6917
Change-Id: Ieb282e7abd4ab31162aee6b52493a6e1b6a25109
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/878239
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50784}
This is added only to uniformly call all tools by the infra side.
NOTRY=true
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: I85a14ea51abfe1bfc775fd73d2fee02bfb2fb31e
Reviewed-on: https://chromium-review.googlesource.com/878361
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50754}
This adds flags to numfuzz for rerunning flaky tests and for storing
json test results. With those flags added, the infra-side can call
numfuzz with the same API as the standard test-runner.
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: I02d1cd02d90677c83f10e072383f3650c041cab1
Reviewed-on: https://chromium-review.googlesource.com/877890
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50753}
1. Fix infinite loop caused by time based fuzzing
2. Shallow copy of the result to avoid dropping output
by different processor.
Bug: v8:6917
Change-Id: Icf823e853be9d3cc8dfd46ed2fb954979bf02d2f
Reviewed-on: https://chromium-review.googlesource.com/877761
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50737}
The tests currently time out due to too many deopt points and hence too many
tests generated.
TBR=sergiyb@chromium.org
Bug: v8:6900
Change-Id: I0998097024a4ed9c087728bb5ef288ab17d3371e
Reviewed-on: https://chromium-review.googlesource.com/876322
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50727}
This adds the benchmarks suite to the deopt fuzzer and removes the
obsolete deopt configs, which since a while are part of the num-fuzz
configs.
TBR=sergiyb@chromium.org
Bug: v8:6900
Change-Id: I9202e2a66a132eca5940b4444fda97816b034e54
Reviewed-on: https://chromium-review.googlesource.com/876085
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50720}
This adds gc stress and scavenge stress to legacy gc fuzzer. Each
flag can be used in separation or combined.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6972
Change-Id: Id03ef96bd42b77415dd201e7f78b3b42786dd3bc
Reviewed-on: https://chromium-review.googlesource.com/874452
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50708}
This prepares to add the stress-marking flag on the infra side.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6972
Change-Id: Ibee30beadb167d06fd7965dfd3cc05fb523158cb
Reviewed-on: https://chromium-review.googlesource.com/874350
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50697}
Since test processors create tests dynamically we cannot simply
count how many tests will be run. Instead we count only base tests
that we've loaded, before creating variants.
Bug: v8:6917
Change-Id: Ibc5b9a73f6afad423572afa575f477ca661a99d5
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/868290
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50642}
Also sort all variants in configuration mapping.
Bug: v8:7285
Change-Id: I2d3032b911e48c34557e47e50345fb29da7bbe85
Reviewed-on: https://chromium-review.googlesource.com/866502
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50584}
This reverts commit 2de796fc57.
Reason for revert: Fails tests on Linux dbg:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/18708
Original change's description:
> [test] Add more d8-based test suites to d8_default
>
> This will run those suites with gc stress, code serializer, and verify-
> predictable mode.
>
> TBR=sergiyb@chromium.org
>
> Bug: v8:7285
> Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
> Reviewed-on: https://chromium-review.googlesource.com/864922
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50555}
TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I8d769829bdb96820e57b1cefcf92fb6976a1031d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7285
Reviewed-on: https://chromium-review.googlesource.com/865334
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50556}
This will run those suites with gc stress, code serializer, and verify-
predictable mode.
TBR=sergiyb@chromium.org
Bug: v8:7285
Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
Reviewed-on: https://chromium-review.googlesource.com/864922
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50555}
This adds a reduced version of bot_default with only d8-based test suites.
Those can be used for gc stress testing, fuzzing and predictable mode.
This also adds a missing explicit build rule for the debugger test suite.
It's working as it's part of the isolate file, but incremental
re-isolation might break on code changes.
Running the tests is still commented out to allow the infra-side to
land. Afterwards, we'll activate the extra testing as a V8 CL
commenting the lines back in.
Bug: v8:7285
Change-Id: I7b758c212f0c4ae3d2f79beea5ec597cee479565
Reviewed-on: https://chromium-review.googlesource.com/864045
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50545}
This removes the explicit --write-protect-code-memory flag from the list
of flags in the "stress_incremental_marking". The feature is enabled by
default by now and no longer needs explicit testing.
R=ulan@chromium.org
BUG=v8:6792
Change-Id: I5d6ba21dff261488bbe1b0148ce204bf78d57334
Reviewed-on: https://chromium-review.googlesource.com/860661
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50500}
Temporary workaround to avoid the same variant name for multiple
variants of the same test.
Bug: v8:6917
Change-Id: I9a25dcaf81d35da0dc2617c089cb4811c2a958cb
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/852833
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50428}
Test processors can be enabled with --infra-staging flag.
Rerunning tests, execution and verbose progress indicator already
work as test processors.
Bug: v8:6917
Change-Id: I40fc42db94dbc8629e8000a3d363030045532fe3
Reviewed-on: https://chromium-review.googlesource.com/850398
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50399}
I added additional exception logging in the execution.py since
errors in processing results were really difficult to debug.
There is a problem on Windows with class serialization when
it comes from dynamically loaded module. To fix it I moved all
output processors to the tools/testrunner/outproc/ and import
them in test/*/testcfg.py.
Bug: v8:6917
Change-Id: Ida604641d659b006e91faf1d56a37769ec47f5f3
Reviewed-on: https://chromium-review.googlesource.com/842784
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50361}
This reverts commit 1685b5d27a.
Reason for revert: Was probably caused by infra change:
https://crrev.com/c/845781
Original change's description:
> Revert "[test] Move has unexpected output to outproc."
>
> This reverts commit 71605b3ea4.
>
> Reason for revert: Seems to break static-initializers step:
> https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
>
> Original change's description:
> > [test] Move has unexpected output to outproc.
> >
> > Expected outcomes optimized to serialize [PASS] as None.
> >
> > Keeping expected outcomes inside output processors should be
> > optimized in the future. Few possible optimizations:
> > - separate classes for tests that are expected to PASS - done as
> > an example in mozilla test suite.
> > - cache output processors inside testcase.
> > - share output processors between copies of the same test - needs
> > some updates to the create_variant to update outproc only if
> > expected outcomes changed.
> >
> > Bug: v8:6917
> > Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> > Reviewed-on: https://chromium-review.googlesource.com/843025
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#50347}
>
> TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
>
> Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6917
> Reviewed-on: https://chromium-review.googlesource.com/849572
> Reviewed-by: Bill Budge <bbudge@chromium.org>
> Commit-Queue: Bill Budge <bbudge@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50348}
TBR=bbudge@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I7a522b6487de6e96985d223524533493eb9171f0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/848975
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50350}
This reverts commit 71605b3ea4.
Reason for revert: Seems to break static-initializers step:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
Original change's description:
> [test] Move has unexpected output to outproc.
>
> Expected outcomes optimized to serialize [PASS] as None.
>
> Keeping expected outcomes inside output processors should be
> optimized in the future. Few possible optimizations:
> - separate classes for tests that are expected to PASS - done as
> an example in mozilla test suite.
> - cache output processors inside testcase.
> - share output processors between copies of the same test - needs
> some updates to the create_variant to update outproc only if
> expected outcomes changed.
>
> Bug: v8:6917
> Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> Reviewed-on: https://chromium-review.googlesource.com/843025
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50347}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/849572
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50348}
Expected outcomes optimized to serialize [PASS] as None.
Keeping expected outcomes inside output processors should be
optimized in the future. Few possible optimizations:
- separate classes for tests that are expected to PASS - done as
an example in mozilla test suite.
- cache output processors inside testcase.
- share output processors between copies of the same test - needs
some updates to the create_variant to update outproc only if
expected outcomes changed.
Bug: v8:6917
Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
Reviewed-on: https://chromium-review.googlesource.com/843025
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50347}
