Also add the ability to skip tests only when --optimize-for-size is passed.
TBR=sergiyb@chromium.org
Bug: v8:7819
Change-Id: I1dcc26ea8664d014b6c50f0d636c13bf21e26ff2
Reviewed-on: https://chromium-review.googlesource.com/c/1328945
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57396}
'simulator_run' is now used exclusively in test/*/*.status.
Change-Id: I501b7ffd19e2476f4c803ed3d25cba69d67039fa
Reviewed-on: https://chromium-review.googlesource.com/c/1318610
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57300}
This is a reland of a12203c64b
Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
>
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
>
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}
Change-Id: I25fcd95bfc1aee1d21da390359423e5dfed112a4
Reviewed-on: https://chromium-review.googlesource.com/c/1286952
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Ben Smith <binji@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56791}
This reverts commit a12203c64b.
Reason for revert: Breaks isolate_tests
https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux%20-%20builder/36777
Original change's description:
> [wasm] Add a new wasm-js testsuite to run js-api tests
>
> These changes were necessary to run with the new style of jsapi tests
> introduced in https://github.com/WebAssembly/spec/pull/883.
>
> Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
> Reviewed-on: https://chromium-review.googlesource.com/c/1277724
> Commit-Queue: Ben Smith <binji@chromium.org>
> Reviewed-by: Andreas Haas <ahaas@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Mathias Bynens <mathias@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56745}
TBR=binji@chromium.org,machenbach@chromium.org,yangguo@chromium.org,ahaas@chromium.org,clemensh@chromium.org,mathias@chromium.org
Change-Id: I2edd0ca94cb5990322571879c81671fa835f3ecd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/1286526
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56746}
These changes were necessary to run with the new style of jsapi tests
introduced in https://github.com/WebAssembly/spec/pull/883.
Change-Id: I4629dd48d595ed97ed0607dec9e7d9808c706a7e
Reviewed-on: https://chromium-review.googlesource.com/c/1277724
Commit-Queue: Ben Smith <binji@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56745}
The regexp for finding resources to be pushed to Android devices was
too lax. On empty strings it tried to check for more resources on a
directory and hung.
The last test262 roll contains tests with empty imports that started
hanging in this way.
TBR=neis@chromium.org
NOTRY=true
Bug: v8:7834
Change-Id: Ie58f1b18bdd99b7b40c1fb39b25e2f481932e0f3
Reviewed-on: https://chromium-review.googlesource.com/c/1270579
Reviewed-by: Georg Neis <neis@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56465}
The cctest and unittests suites call the test executable to list the
tests, which requires pushing the executables to the device on Android.
NOTRY=true
TBR=sergiyb@chromium.org
Bug: chromium:866862
Change-Id: I318dff7af2b2de3b8642ec4b6ba30b602da808bb
Reviewed-on: https://chromium-review.googlesource.com/1213202
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55723}
Path names in import/export statements are relative to the file they are in.
This fixes the logic and unblocks using the messages test suite on Android,
which has cases importing files from mjsunit, which import more files from
there.
Bug: chromium:866862
Change-Id: I8d2ff645f69b67fbdaf4a622d06308e55298b0ce
Reviewed-on: https://chromium-review.googlesource.com/1206570
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55643}
This shares logic for finding additional resources in JS source code.
Previously the logic was implemented for mjsunit, now it will be used
across all d8-based test cases.
This'll enable adding those test suites for Android testing.
Bug: chromium:866862
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I7c89ba141483aaf692a03c0e168edb61bbb7b010
Reviewed-on: https://chromium-review.googlesource.com/1193873
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55503}
This is a reland of 4c0943424c
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
Bug: chromium:866862
Change-Id: Icf7e04c75d4abeab7254d10ba21240e46b0022ae
Reviewed-on: https://chromium-review.googlesource.com/1170643
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55059}
This reverts commit 4c0943424c.
Reason for revert: Unfortunately this broke all perf builders.
Original change's description:
> [test] Add logic to run tests on Android
>
> This adds a new command abstraction for running commands on Android
> using dockered devices on swarming.
>
> The new abstraction handles pushing all required files to the device.
> The logic used for pushing and running is reused from the perf runner.
>
> This adds only the mjsunit test suite. Others will be handled in
> follow up CLs. The suite logic is enhanced with auto-detection of files
> to be pushed to devices, for e.g. load or import statements.
>
> Some test cases need an extra resource section for specifying required
> files.
>
> Remaining failing tests are marked in the status files for later
> triage.
>
> Bug: chromium:866862
> Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
> Reviewed-on: https://chromium-review.googlesource.com/1150153
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#55041}
TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org
Change-Id: If80129810586b709dab762c9b5724888e15daec2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:866862
Reviewed-on: https://chromium-review.googlesource.com/1170962
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55058}
This adds a new command abstraction for running commands on Android
using dockered devices on swarming.
The new abstraction handles pushing all required files to the device.
The logic used for pushing and running is reused from the perf runner.
This adds only the mjsunit test suite. Others will be handled in
follow up CLs. The suite logic is enhanced with auto-detection of files
to be pushed to devices, for e.g. load or import statements.
Some test cases need an extra resource section for specifying required
files.
Remaining failing tests are marked in the status files for later
triage.
Bug: chromium:866862
Change-Id: I2b957559f07fdcd8c1bd2f7034f5ba7754a31fb7
Reviewed-on: https://chromium-review.googlesource.com/1150153
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55041}
The test driver compares command-line arguments against test names
it finds on disk. Using Python's "fnmatch" for this nicely handles
wildcards, but is relatively slow. For given test names that don't
contain any '*', we can use string equality testing, which is much
faster.
Example: the time to evaluate
tools/run-tests.py --arch x64 --mode release \
$(grep 'object-spread' -l -r test/test262/data/test/ | \
sed -E 's|\.js$||' | \
sed -E 's|^test/test262/data/test/|test262/|')
goes from "I gave up and killed the process after several minutes"
to a couple of seconds with this patch.
Change-Id: I9ec404b7516afd801fe6126347f6dff533d1977c
Reviewed-on: https://chromium-review.googlesource.com/1149196
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54704}
Before flipping the flag, some tests need to be adapted. This CL
prepares these tests, such that the flag flip CL really just flips a
flag.
R=titzer@chromium.org, hablich@chromium.org
Bug: v8:6600, chromium:787421
Change-Id: I8030df69cda5f3fb81354350a37f65c0d1c669bd
Reviewed-on: https://chromium-review.googlesource.com/1110363
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53959}
This is to speed up slow nosnap runs. Allocation verification is covered by
running mksnapshot on other builders.
Change-Id: I9c286f7dd4abac9cf1be45be4a483a7b36d09f8a
Reviewed-on: https://chromium-review.googlesource.com/1059113
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53234}
D8 enables the Wasm trap handler by default now, but we need to make sure the
older bounds check case still gets test coverage too, as bounds checks will
continue to be a supported configuration.
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I5b0bdded6929a9b3a8480e87d038398b8d2a0fd8
Reviewed-on: https://chromium-review.googlesource.com/1048835
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53078}
This makes testing with random gc stress use the status file entrees
for gc fuzzing as well, since many test cases not suitable for one, are
not suitable for the other.
This also skips two more tests that rely on assert(Un)Optimized, which
is unreliable with gc fuzzing.
TBR=sigurds@chromium.org
Change-Id: I33a617b251d5cf65cf6e486d07ec55cde050b8ab
Reviewed-on: https://chromium-review.googlesource.com/1021082
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52701}
Embedded builtins are now based off the v8_enable_embedded_builtins
gn flag instead, which conditionally defines V8_EMBEDDED_BUILTINS.
Bug: v8:6666
Change-Id: I44d40d30fce3a3ed9bbf973d46c4990ba3fade40
Reviewed-on: https://chromium-review.googlesource.com/964361
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52092}
The feature in question has been enabled by default for a while and we
no longer need to maintain a configuration without it enabled. Note that
this change only removes the mechanical pieces. Further cleanup enabled
by this will be done as follow-ups.
R=clemensh@chromium.org
BUG=v8:7549
Change-Id: I90e5bcddabe74a18a4d2a88132e8dc93317bcff4
Reviewed-on: https://chromium-review.googlesource.com/958424
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51883}
We now unconditionally both parse and compile StreamedSource on the background
thread.
BUG=v8:5203
Change-Id: I42d6fe9059bc1745da3a415d270f46cf1c08b306
Reviewed-on: https://chromium-review.googlesource.com/948854
Reviewed-by: Mythri Alle <mythria@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51758}
This enables the v8_enable_embedded_builtins gn flag on non-ia32 builds
and adds a new --stress-off-heap-code test mode to fyi bots.
v8_enable_embedded_builtins=true changes accesses to constants and
external references to go through the root list in builtins code.
--stress-off-heap-code copies builtins code off-heap on isolate
creation.
A few drive-by-fixes:
- ensure that we actually inspect the correct builtin during
isolate-independence testing.
- relax tests to decrease maintenance (now we only fail if a builtin
should be isolate-independent but isn't).
- switch to a different off-heap-trampoline register on arm due to
conflicts with custom stub linkages.
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Bug: v8:6666
Change-Id: I09ad3c75cb4342f4c548ea780f275993730896c8
Reviewed-on: https://chromium-review.googlesource.com/934281
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51717}
Introducing a new status-file variable: mips_arch_variant. With this
variable, in status files will be possible to define selections which
are based on MIPS architecture revisions/variants.
TEST=
BUG=
Change-Id: Ifd682552db2f26be4e56dc94ad50bed063ff14c5
Reviewed-on: https://chromium-review.googlesource.com/941212
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#51647}
This is a partial revert of:
https://crrev.com/c/890938 and https://crrev.com/c/893982
Before this CL, the test runner blocked on ongoing tests in order to
process their results after an internal timeout. However, the logic
required for this feature was overly complicated and prevented an
acceptable implementation for fast aborts. Furthermore, also the fuzzers
suffered from timeouts on swarming due to hanging tests.
Instead, we now abort immediately on internal timeout (used on
fuzzers), SIGINT (Ctrl-C) and SIGTERM. Ongoing tests are immediately
terminated and their results are disregarded. On SIGTERM and SIGINT,
we return with non-zero exit codes, and zero on internal timeout.
This will also properly return json output, when the external hard
timeout is reached on swarming (causes SIGTERM).
TBR=sergiyb@chromium.org
Bug: v8:7423, chromium:813065
Change-Id: Ib20f835f58a0970693bdd3b21dc5d766d8e115d8
Reviewed-on: https://chromium-review.googlesource.com/924852
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51399}
This reverts commit 9808093ef6.
Reason for revert:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/19149
Original change's description:
> Update test configurations for code serializer
>
> Update test configurations on the bots to test for serializing
> after execute (--cache=after-execute) and producing full code
> cache (--cache=full-code-cache) options. We no longer need
> to test serializing before execute (--cache=code) on the bots.
>
> Bug: v8:7302
> Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
> Reviewed-on: https://chromium-review.googlesource.com/869931
> Commit-Queue: Mythri Alle <mythria@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#51235}
TBR=machenbach@chromium.org,yangguo@chromium.org,mythria@chromium.org
Change-Id: I47e03101804194c21b0edf418b744b0ccb66cbf6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7302
Reviewed-on: https://chromium-review.googlesource.com/913134
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51236}
Update test configurations on the bots to test for serializing
after execute (--cache=after-execute) and producing full code
cache (--cache=full-code-cache) options. We no longer need
to test serializing before execute (--cache=code) on the bots.
Bug: v8:7302
Change-Id: I123b07028d9231f6da6145b72b62b9ee31352388
Reviewed-on: https://chromium-review.googlesource.com/869931
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51235}
The suppression flag must be set before loading the tests to have an effect.
This got turned around in https://crrev.com/c/899366 and is fixed again by
this CL.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6917
Change-Id: Ie58465288a0d6eec9a99a23d610710de9e1cdddd
Reviewed-on: https://chromium-review.googlesource.com/911114
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51209}
This new variant is executed on a small number of bots as part of the
"extra" suite. It checks that the wasm-jit-to-native flag can still be
disabled if unexpected failures pop up on a release branch.
R=machenbach@chromium.org
Bug: v8:7417
Change-Id: I1658cb2f04302fa80915b59bfedd85d980742db5
Reviewed-on: https://chromium-review.googlesource.com/909213
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51204}
Timeouts on num-fuzzer only lead to false positives, as tests might
take unpredictably longer.
This CL forgives timeouts through a global override mechanism of the
expected outcomes. This allows to remove already existing scattered
code that allowed timeouts in some test suites only.
Bug: v8:6917
Change-Id: Ib131765d360e335789c1952bc6793ed051e016ea
Reviewed-on: https://chromium-review.googlesource.com/908454
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51188}
Load mjsunit.js inside the realm as otherwise the functions are not
available in the realm's scope.
This also prints timestamps after each test to easier track down slow
tests.
We also pass --omit-quit to not stop too early.
This also adds the ability to skip certain tests for endurance
fuzzing and skips some tests with known problems.
TBR=ulan@chromium.org,hpayer@chromium.org
Bug: v8:6972, v8:7400
Change-Id: I44464c28bfb10c84f2e59972e7b86945a47ca3b3
Reviewed-on: https://chromium-review.googlesource.com/899008
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51066}
Only test processors code left. It enabled to move more stuff to
the base runner, like progress indicators creation.
Bug: v8:6917
Change-Id: Ie6dd211cec561a07d92bcc4431ea88eb1842c8fa
Reviewed-on: https://chromium-review.googlesource.com/897624
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51030}
System tests updated to pass. I will cleanup the code in a
separate CL removing the old code.
Bug: v8:6917
Change-Id: Ie49efe06c43faade846e322b35c4a2068137a88b
Reviewed-on: https://chromium-review.googlesource.com/897565
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#51025}
For mips, if 'mips_arch_variant=="r6"' and if 'mips_use_msa' flag is set
to 'true', then test-run-wasm-simd tests won't be skipped for mips. It
will also force 'MIPS_SIMD' bit in CpuFeatures to be set.
ARM processors are assumed to support SIMD.
Change-Id: Iea668b97ef995ca4949ddbf2ffc734aad89d3aa3
Reviewed-on: https://chromium-review.googlesource.com/868430
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Commit-Queue: Ivica Bogosavljevic <ivica.bogosavljevic@mips.com>
Cr-Commit-Position: refs/heads/master@{#50981}
This is a reland of 0db74d4974.
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
This reverts commit 0db74d4974.
Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
This adds back an option for interrupt budget available in no-snap
builds. This also adds a fuzzer configuration for numfuzz that enables
fuzzing the interrupt budget option. A new flag --disable-analysis
allows to generally skip the fuzzer's analysis phase, which can be
chosen for interrupt budget, which doesn't support an analysis phase.
Bug: v8:6917
Change-Id: I546dd9ee41c3e0fb027108ef4606a34514f230d4
Reviewed-on: https://chromium-review.googlesource.com/885805
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50929}
Without realm-eval, the toplevel references in loaded tests are not
garbage-collected and changes in the tests can affect the global
object.
TBR=ulan@chromium.org
NOTRY=true
Bug: v8:7379, v8:7376
Change-Id: Id4d451e38fe49c6531a07d541ad3c67298b6acf2
Reviewed-on: https://chromium-review.googlesource.com/890519
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50915}
They have been replaced with num_fuzzer.
Bug: v8:6917
Change-Id: I2f78df308cec0a58f0d665bce82503dee68fcebc
Reviewed-on: https://chromium-review.googlesource.com/888641
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50903}
This is a reland of 8f4407fbd6.
Original change's description:
> [test] Keep the order of variants
>
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}
Bug: v8:6917
Change-Id: I9d4c617ccec6db8929af3095f2f27d8964c5d6b8
Reviewed-on: https://chromium-review.googlesource.com/888582
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50901}
This reverts commit 8f4407fbd6.
Reason for revert: Failing in old test runner. See comment.
Original change's description:
> [test] Keep the order of variants
>
> Bug: v8:6917
> Change-Id: I7e04255649f62a28f0ed07ccb4fbaf4825498bf6
> Reviewed-on: https://chromium-review.googlesource.com/886841
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50874}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: Ia8608053e080c70d2ac09d7921a6abb6cd0e338f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/888749
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50895}
A timeout result of randomly combined tests is not a useful indicator.
Therefore we ignore those now. This also prevents timeouts from being
re-run.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6917
Change-Id: I8e5eb93e2f13067ca5270fb152c3dc1eda1bb3c2
Reviewed-on: https://chromium-review.googlesource.com/888559
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50879}
This will affect all manual test runs with x64. Most bots on x64 already
migrated.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:7343
Change-Id: I87f46f1848a813c0b320b3e9901481b9232025a5
Reviewed-on: https://chromium-review.googlesource.com/883101
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50825}
This adds a tri-state flag --infra-staging and --no-infra-staging, which
can be used to explicitly opt in or out of the staging test runner.
If not specified, a new architecture whitelist will enable roll-out per
arch for manual (none CI) runs.
We'll start whitelisting archs in follow ups.
Bug: v8:7343
Change-Id: I1228e48969fd379f5c231a2b8fad4afc01da94c0
Reviewed-on: https://chromium-review.googlesource.com/881169
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50812}
Introduce option to run fuzzer processor without analysis phase.
It will be used in fuzzing combined tests.
Bug: v8:6917
Change-Id: Ic96d6b8c5a35c81da80340555bdd75c0d518cb5a
Reviewed-on: https://chromium-review.googlesource.com/880948
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50810}
The extra output on mac was to investigate a bug that's now fixed.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6927
Change-Id: Iac8074c7e89a987e164f676442da1ed4f93987cc
Reviewed-on: https://chromium-review.googlesource.com/880623
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50787}
Since we're not winning anything by changing the result between
processors on the main process, reduce is noop there and result is
immutable.
Bug: v8:6917
Change-Id: Ieb282e7abd4ab31162aee6b52493a6e1b6a25109
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/878239
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50784}
This is added only to uniformly call all tools by the infra side.
NOTRY=true
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: I85a14ea51abfe1bfc775fd73d2fee02bfb2fb31e
Reviewed-on: https://chromium-review.googlesource.com/878361
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50754}
This adds flags to numfuzz for rerunning flaky tests and for storing
json test results. With those flags added, the infra-side can call
numfuzz with the same API as the standard test-runner.
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: I02d1cd02d90677c83f10e072383f3650c041cab1
Reviewed-on: https://chromium-review.googlesource.com/877890
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50753}
1. Fix infinite loop caused by time based fuzzing
2. Shallow copy of the result to avoid dropping output
by different processor.
Bug: v8:6917
Change-Id: Icf823e853be9d3cc8dfd46ed2fb954979bf02d2f
Reviewed-on: https://chromium-review.googlesource.com/877761
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50737}
The tests currently time out due to too many deopt points and hence too many
tests generated.
TBR=sergiyb@chromium.org
Bug: v8:6900
Change-Id: I0998097024a4ed9c087728bb5ef288ab17d3371e
Reviewed-on: https://chromium-review.googlesource.com/876322
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50727}
This adds the benchmarks suite to the deopt fuzzer and removes the
obsolete deopt configs, which since a while are part of the num-fuzz
configs.
TBR=sergiyb@chromium.org
Bug: v8:6900
Change-Id: I9202e2a66a132eca5940b4444fda97816b034e54
Reviewed-on: https://chromium-review.googlesource.com/876085
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50720}
This adds gc stress and scavenge stress to legacy gc fuzzer. Each
flag can be used in separation or combined.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6972
Change-Id: Id03ef96bd42b77415dd201e7f78b3b42786dd3bc
Reviewed-on: https://chromium-review.googlesource.com/874452
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50708}
This prepares to add the stress-marking flag on the infra side.
TBR=sergiyb@chromium.org
NOTRY=true
Bug: v8:6972
Change-Id: Ibee30beadb167d06fd7965dfd3cc05fb523158cb
Reviewed-on: https://chromium-review.googlesource.com/874350
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50697}
Since test processors create tests dynamically we cannot simply
count how many tests will be run. Instead we count only base tests
that we've loaded, before creating variants.
Bug: v8:6917
Change-Id: Ibc5b9a73f6afad423572afa575f477ca661a99d5
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/868290
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50642}
Also sort all variants in configuration mapping.
Bug: v8:7285
Change-Id: I2d3032b911e48c34557e47e50345fb29da7bbe85
Reviewed-on: https://chromium-review.googlesource.com/866502
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50584}
This reverts commit 2de796fc57.
Reason for revert: Fails tests on Linux dbg:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20debug/builds/18708
Original change's description:
> [test] Add more d8-based test suites to d8_default
>
> This will run those suites with gc stress, code serializer, and verify-
> predictable mode.
>
> TBR=sergiyb@chromium.org
>
> Bug: v8:7285
> Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
> Reviewed-on: https://chromium-review.googlesource.com/864922
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50555}
TBR=machenbach@chromium.org,yangguo@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I8d769829bdb96820e57b1cefcf92fb6976a1031d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:7285
Reviewed-on: https://chromium-review.googlesource.com/865334
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50556}
This will run those suites with gc stress, code serializer, and verify-
predictable mode.
TBR=sergiyb@chromium.org
Bug: v8:7285
Change-Id: I4184376cf3fc9dd161b1e85ec562ed4266f5b8cf
Reviewed-on: https://chromium-review.googlesource.com/864922
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50555}
This adds a reduced version of bot_default with only d8-based test suites.
Those can be used for gc stress testing, fuzzing and predictable mode.
This also adds a missing explicit build rule for the debugger test suite.
It's working as it's part of the isolate file, but incremental
re-isolation might break on code changes.
Running the tests is still commented out to allow the infra-side to
land. Afterwards, we'll activate the extra testing as a V8 CL
commenting the lines back in.
Bug: v8:7285
Change-Id: I7b758c212f0c4ae3d2f79beea5ec597cee479565
Reviewed-on: https://chromium-review.googlesource.com/864045
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50545}
This removes the explicit --write-protect-code-memory flag from the list
of flags in the "stress_incremental_marking". The feature is enabled by
default by now and no longer needs explicit testing.
R=ulan@chromium.org
BUG=v8:6792
Change-Id: I5d6ba21dff261488bbe1b0148ce204bf78d57334
Reviewed-on: https://chromium-review.googlesource.com/860661
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50500}
Temporary workaround to avoid the same variant name for multiple
variants of the same test.
Bug: v8:6917
Change-Id: I9a25dcaf81d35da0dc2617c089cb4811c2a958cb
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/852833
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50428}
Test processors can be enabled with --infra-staging flag.
Rerunning tests, execution and verbose progress indicator already
work as test processors.
Bug: v8:6917
Change-Id: I40fc42db94dbc8629e8000a3d363030045532fe3
Reviewed-on: https://chromium-review.googlesource.com/850398
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50399}
I added additional exception logging in the execution.py since
errors in processing results were really difficult to debug.
There is a problem on Windows with class serialization when
it comes from dynamically loaded module. To fix it I moved all
output processors to the tools/testrunner/outproc/ and import
them in test/*/testcfg.py.
Bug: v8:6917
Change-Id: Ida604641d659b006e91faf1d56a37769ec47f5f3
Reviewed-on: https://chromium-review.googlesource.com/842784
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50361}
This reverts commit 1685b5d27a.
Reason for revert: Was probably caused by infra change:
https://crrev.com/c/845781
Original change's description:
> Revert "[test] Move has unexpected output to outproc."
>
> This reverts commit 71605b3ea4.
>
> Reason for revert: Seems to break static-initializers step:
> https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
>
> Original change's description:
> > [test] Move has unexpected output to outproc.
> >
> > Expected outcomes optimized to serialize [PASS] as None.
> >
> > Keeping expected outcomes inside output processors should be
> > optimized in the future. Few possible optimizations:
> > - separate classes for tests that are expected to PASS - done as
> > an example in mozilla test suite.
> > - cache output processors inside testcase.
> > - share output processors between copies of the same test - needs
> > some updates to the create_variant to update outproc only if
> > expected outcomes changed.
> >
> > Bug: v8:6917
> > Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> > Reviewed-on: https://chromium-review.googlesource.com/843025
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#50347}
>
> TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
>
> Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6917
> Reviewed-on: https://chromium-review.googlesource.com/849572
> Reviewed-by: Bill Budge <bbudge@chromium.org>
> Commit-Queue: Bill Budge <bbudge@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50348}
TBR=bbudge@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I7a522b6487de6e96985d223524533493eb9171f0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/848975
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50350}
This reverts commit 71605b3ea4.
Reason for revert: Seems to break static-initializers step:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
Original change's description:
> [test] Move has unexpected output to outproc.
>
> Expected outcomes optimized to serialize [PASS] as None.
>
> Keeping expected outcomes inside output processors should be
> optimized in the future. Few possible optimizations:
> - separate classes for tests that are expected to PASS - done as
> an example in mozilla test suite.
> - cache output processors inside testcase.
> - share output processors between copies of the same test - needs
> some updates to the create_variant to update outproc only if
> expected outcomes changed.
>
> Bug: v8:6917
> Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> Reviewed-on: https://chromium-review.googlesource.com/843025
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50347}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/849572
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50348}
Expected outcomes optimized to serialize [PASS] as None.
Keeping expected outcomes inside output processors should be
optimized in the future. Few possible optimizations:
- separate classes for tests that are expected to PASS - done as
an example in mozilla test suite.
- cache output processors inside testcase.
- share output processors between copies of the same test - needs
some updates to the create_variant to update outproc only if
expected outcomes changed.
Bug: v8:6917
Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
Reviewed-on: https://chromium-review.googlesource.com/843025
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50347}
This seperates OS-specific code of the Command class into a Posix and a
WindowsCommand to simplify scattered OS checks.
This also removes some temporary mac debug output that's obsolete after
https://crbug.com/v8/6927 got resolved.
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: Iaab3b527ce556dfba797a164ae58e8dd358eb56f
Reviewed-on: https://chromium-review.googlesource.com/847000
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50346}
This replaces multiprocessing with threading in tests to make python
coverage work.
TBR=sergiyb@chromium.org
Bug: v8:6917
Change-Id: Idff763dfefa4a7fc782133d94089b3a5b00a194d
Reviewed-on: https://chromium-review.googlesource.com/844735
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50344}
This tests some high-level scenarios of the V8 test runner, with
fake executable, test-suite extension and build configs.
The runners are slightly modified to be easier testable. Args
are passed from the tests now and the V8 root folder can be faked
by the tests.
We support coverage if python coverage 4.0 is installed. Otherwise
we run without it.
Bug: v8:6917
Change-Id: Ib149fd88027cbdc3382bcaea2d82020582f79d2d
Reviewed-on: https://chromium-review.googlesource.com/831506
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50310}
All get outcome logic moved out from testsuite which makes output
processors serializable. Output processors that are not global
are stored inside testcase to share them with all test copies.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I625466b2b00565d885556f96cab03d8d16fa7ff4
Reviewed-on: https://chromium-review.googlesource.com/837069
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50275}
First CL moving parsing output to a separate class.
Following CLs will move (and deduplicate) code from
testcases by sharing common features like comparing
stdout to the expected output.
Bug: v8:6917
Change-Id: I1174a985949a0b946cefbf7e7c91e8815cbb4f15
Reviewed-on: https://chromium-review.googlesource.com/834130
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50228}
Fix report mode and make it use testcase properties so statusfile
outcomes can be private.
Bug: v8:6917
Change-Id: Id38e89e0ba427c3bbb7ad12ba93e02beb7e46219
Reviewed-on: https://chromium-review.googlesource.com/833909
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50209}
Pass shell name instead of an absolute path.
Bug: v8:796166
Change-Id: Ia9472e893fd2cb3fde2a94997f3e9daf30da06ea
Reviewed-on: https://chromium-review.googlesource.com/833917
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50207}
First step in moving all statusfile logic into statusfile.py.
Introduce StatusFile object that will be used for storing and managing
outcomes.
Bug: v8:6917
Change-Id: I024f9b1d029830345149422a08a8905e92545252
Reviewed-on: https://chromium-review.googlesource.com/832433
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50202}
Fix unittests since they were incompatible with the
new testcase and testsuite API.
Bug: v8:6917
Change-Id: I917bf58e21402e0b90bc91c0483ade0e7c90bdd6
Reviewed-on: https://chromium-review.googlesource.com/832392
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50201}
If a command prefix should be used in predictable mode, it should
wrap around d8 and not around the predictable wrapper.
NOTRY=true
TBR=majeski@google.com
Bug: v8:7166
Change-Id: I16f33090c647ea4f6f040c6194908df6500b82a3
Reviewed-on: https://chromium-review.googlesource.com/832206
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50160}
This moves the error-code check for verify-predictable mode to the API
method checking for status file outcomes, overwriting the default
behavior.
This is resembling the behavior prior to:
https://chromium-review.googlesource.com/c/808971/
Otherwise, the status file outcomes will expect some negative tests
to fail in the mozilla test suite, which pass in predictable mode.
Now, negative tests are simply not supported.
Bug: v8:7166
Change-Id: I1d4bcaf66cb54c5fbb217dd9091b88ecc5b0e456
Reviewed-on: https://chromium-review.googlesource.com/817741
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50110}
- All testcase/testsuite/variant generator subclasses renamed to
just TestCase/TestSuite/VariantGenerator since they're private
implementation.
- All `testcase` variables renamed to `test` to not conflict with
a module name.
- No more two statements in the same line.
- Removed some unused testsuite methods.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I7710f3419f738a5f9ddca73765dd2cad2e35b952
Reviewed-on: https://chromium-review.googlesource.com/823964
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50076}
- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
Some refactor moved from
https://chromium-review.googlesource.com/c/v8/v8/+/798331.
Bug: v8:6917
Change-Id: I8cae6cfca7a0d7d8e234052c0ab0bfe252355e60
Reviewed-on: https://chromium-review.googlesource.com/819550
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50020}
Since we have only unittests that are under GoogleTestSuite there
is no need to keep it as a default suite and we can make it
specific for unittests.
Bug: v8:6917
Change-Id: Ie2d57342773f228dea72184ab0f2abfc9d2daa70
Reviewed-on: https://chromium-review.googlesource.com/819253
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50011}
This moves the verify-predictable logic from the test runner into
a python wrapper script.
This revealed two more tests that don't print allocations, which are
now skipped.
Bug: v8:7166, v8:7177
Change-Id: Ie4a541cb2a20900414ffe1caf4b3fccc4a5edb52
Reviewed-on: https://chromium-review.googlesource.com/808971
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49964}
- in certain cases, we need both modification scopes because we may
mutate JS functions, even in the jit-to-native case - e.g. JS-to-wasm
wrappers
- added handling for wasm-to-wasm wrappers in the context of lazy
compilation.
Bug: v8:7105
Change-Id: I085c14e03ef0b08d040998f2207abf7bc3fff01c
Reviewed-on: https://chromium-review.googlesource.com/811285
Commit-Queue: Mircea Trofin <mtrofin@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49947}
This is a reland of 3b06511052
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
Bug: v8:6917
Change-Id: Ia39010a0a0f63537ad12490dfab17897d70d4930
Reviewed-on: https://chromium-review.googlesource.com/806034
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49830}
This is a reland of 3b06511052
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
Bug: v8:6917
Change-Id: I981994224e493bee4c9435cb80772b6e2ad8fbb1
Reviewed-on: https://chromium-review.googlesource.com/805336
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49827}
This reverts commit 3b06511052.
Reason for revert: Broke fuzzers:
https://build.chromium.org/p/client.v8.clusterfuzz/builders/V8%20Deopt%20Fuzzer/builds/384
Original change's description:
> Reland "[test] Creating command before execution phase."
>
> This is a reland of 98cc9e862f
> Original change's description:
> > [test] Creating command before execution phase.
> >
> > Immutable command class with shell, flags and
> > environment.
> >
> > Command creation moved from worker to the main
> > process. Because of that there is no need to send
> > test cases beyond process boundaries and load test
> > suites in worker processes.
> >
> > Bug: v8:6917
> > Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> > Reviewed-on: https://chromium-review.googlesource.com/791710
> > Commit-Queue: Michał Majewski <majeski@google.com>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#49746}
>
> Bug: v8:6917
> Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
> Reviewed-on: https://chromium-review.googlesource.com/800370
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michał Majewski <majeski@google.com>
> Cr-Commit-Position: refs/heads/master@{#49756}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: v8:6917
Change-Id: I4938642c4396366be1e13daf6998c4b8538b688b
Reviewed-on: https://chromium-review.googlesource.com/804254
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49805}
This is a reland of 98cc9e862f
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
Bug: v8:6917
Change-Id: I49c29a8db813c47909f2cc45070ac7721a447c7a
Reviewed-on: https://chromium-review.googlesource.com/800370
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49756}
This reverts commit 98cc9e862f.
Reason for revert: Breaks test isolation:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20builder/builds/29746
Original change's description:
> [test] Creating command before execution phase.
>
> Immutable command class with shell, flags and
> environment.
>
> Command creation moved from worker to the main
> process. Because of that there is no need to send
> test cases beyond process boundaries and load test
> suites in worker processes.
>
> Bug: v8:6917
> Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
> Reviewed-on: https://chromium-review.googlesource.com/791710
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#49746}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I44b99468d18fd093833f4185dad067a9eeaf2bc1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/800292
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49747}
Immutable command class with shell, flags and
environment.
Command creation moved from worker to the main
process. Because of that there is no need to send
test cases beyond process boundaries and load test
suites in worker processes.
Bug: v8:6917
Change-Id: Ib6a44278095b4f7141eb9b96802fe3e8117678a6
Reviewed-on: https://chromium-review.googlesource.com/791710
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49746}
This derives two global variables to ease adding new variants.
Bug: chromium:788104
Change-Id: Ibe933dc0999f6dfa0ae8bf2afa0a5de8e44aefca
Reviewed-on: https://chromium-review.googlesource.com/799790
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49741}
