Commit Graph

144 Commits

Author SHA1 Message Date
Liviu Rau
b5a7ca848c [test] Refactor testrunner (2)
- moved test data closer to tests
- removed the coverage related code
- refactored to remove boilerplate from test code

Bug: v8:12785
Change-Id: I1013d29d8ff2c3ecb786c294ae3b3ab6decdca20
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3683610
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80921}
2022-06-02 16:49:28 +00:00
Liviu Rau
0281df0421 [test] Refactor testrunner (1)
Bringing test closer together

Bug: v8:12785
Change-Id: I2eed79fd3b1bda210b1f58b341422fc0c6435bfd
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3678210
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80887}
2022-06-01 10:37:21 +00:00
Camillo Bruni
46ca99766c [tools] Fix run_perf.py --filter=<regexp>
In the previous refactoring --filter was accidentally filtering out
paths that matched <regexp>. This CL restores the original behavior
where only the matching paths are kept.

Bug: v8:12821, v8:11113
Change-Id: I7e7d7b793107fbf9b4944b1674874150803f4bb4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3623539
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80426}
2022-05-09 13:34:50 +00:00
Camillo Bruni
a42a2f4183 [tools] Add variants support for run_perf.py
We usually run benchmarks in multiple variants: default, future, noopt
This is currently only achieved by copying the run-perf json file and
changing the flags at the top-level (or copy whole subsections).

Using "variants" we can duplicate the tests at the current level with
different values and easily create benchmarks that differ only in v8
flags.

Drive-by-fix:
- Add Node.__iter__ and log the whole config graph in debug mode
- Add GraphConfig.__str__ method for better debugging
- Rename TraceConfig to LeafTraceConfig
- Rename RunnableTraceConfig to RunnableLeafTraceConfig
- Make --filter accept a regexp to better filter out variants

Bug: v8:12821, v8:11113
Change-Id: I56a2ba2dd24da15c7757406e9961746219cd8061
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3596128
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80307}
2022-05-02 14:28:15 +00:00
Alexander Schulze
9dd0d7147b [py3] Bump tools/unittests/*
Bug: v8:12581
Change-Id: Iea05e0171ad6edbda569c443c0db97e5c0bfc9ad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3569222
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Alexander Schulze <alexschulze@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79769}
2022-04-05 08:12:20 +00:00
Alexander Schulze
ea2bbcbe87 [py3] Bump tools/unittests/compare_torque_output_test.py
Bug: v8:12581
Change-Id: I2e869a81f6a8c785954a73cf2e97dae2a9ab3ade
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3568450
Auto-Submit: Alexander Schulze <alexschulze@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79738}
2022-04-04 12:16:16 +00:00
Michael Achenbach
b086206161 Reland "[tools] Clean up py2 code"
This is a reland of commit 1289704aae

Mac-arm64 problem fixed by:
https://crrev.com/c/3550199

Original change's description:
> [tools] Clean up py2 code
>
> Bug: chromium:1292013
> Change-Id: Ic2c3a197005a2136bb0eda4cbb36d8eb57f42a7c
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3523047
> Reviewed-by: Liviu Rau <liviurau@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#79507}

Bug: chromium:1292013
Change-Id: Iadf0ccf94c82012088b76a866296c8e008dff02f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3550274
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79624}
2022-03-25 12:14:26 +00:00
Michael Achenbach
f7aac0cf9f Revert "[tools] Clean up py2 code"
This reverts commit 1289704aae.

Reason for revert:
https://ci.chromium.org/ui/p/v8/builders/ci/V8%20Mac%20-%20arm64%20-%20release/8735/overview

Original change's description:
> [tools] Clean up py2 code
>
> Bug: chromium:1292013
> Change-Id: Ic2c3a197005a2136bb0eda4cbb36d8eb57f42a7c
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3523047
> Reviewed-by: Liviu Rau <liviurau@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#79507}

Bug: chromium:1292013
Change-Id: I48c38209220b775d0caa45f487ed163f78333e0c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3532228
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#79508}
2022-03-17 10:07:38 +00:00
Michael Achenbach
1289704aae [tools] Clean up py2 code
Bug: chromium:1292013
Change-Id: Ic2c3a197005a2136bb0eda4cbb36d8eb57f42a7c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3523047
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79507}
2022-03-17 09:16:57 +00:00
Alexander Schulze
7601854ddd [v8] py3 migration of tools/predictable_wrapper.py
Migrate predictable_wrapper to py3. Run test in v8_presubmit.

R=liviurau@chromium.org, machenbach@chromium.org

Bug: chromium:1245634
Change-Id: I941e248ffcf12ce26a55a5f5889dab06ee74e66e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3448379
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Alexander Schulze <alexschulze@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79200}
2022-02-22 09:48:35 +00:00
Alexander Schulze
604f661495 [v8] Migrate compare_torque_output to py3
R=liviurau@chromium.org, machenbach@chromium.org

Bug: chromium:1292010
Change-Id: Ice96bda42e1597b7d266cafb61565a54ea96d9a8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3448374
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Alexander Schulze <alexschulze@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78997}
2022-02-08 15:46:11 +00:00
Michael Achenbach
efd28c14c2 [infra] Make various scripts compatible with Python3
This fixes all Python3 problems in scripts and tests running via
v8_presubmit.py. It includes:
- Test runner
- Release tools
- Perf runner
- Torque formatter
- V8's main presubmit

On bots, v8_presubmit is run with vpython, hence we also add
the required dependencies. After the Python3 migration, most
of the transitional code in this CL can be removed again.

Bug: chromium:1293709,chromium:1292016
Change-Id: Ic25e5965948b212c047e9d5194d2a4b6db1fa91b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3432213
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78936}
2022-02-04 07:17:24 +00:00
Shu-yu Guo
eb722a0408 Skip JS shared memory tests on build configs that don't have it
Drive-by fix to align what builds the test runner considers to be able
to have shared memory features and what builds can create a shared
Isolate.

Bug: v8:12007
Change-Id: I151513ccbfbee31e5b35c5ce8e9192732eabfee2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3421507
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Shu-yu Guo <syg@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78824}
2022-01-27 20:57:54 +00:00
Samuel Groß
277fdd1de7 V8 Sandbox rebranding
This CL renames a number of things related to the V8 sandbox.
Mainly, what used to be under V8_HEAP_SANDBOX is now under
V8_SANDBOXED_EXTERNAL_POINTERS, while the previous V8 VirtualMemoryCage
is now simply the V8 Sandbox:

V8_VIRTUAL_MEMORY_CAGE => V8_SANDBOX
V8_HEAP_SANDBOX => V8_SANDBOXED_EXTERNAL_POINTERS
V8_CAGED_POINTERS => V8_SANDBOXED_POINTERS
V8VirtualMemoryCage => Sandbox
CagedPointer => SandboxedPointer
fake cage => partially reserved sandbox
src/security => src/sandbox

This naming scheme should simplify things: the sandbox is now the large
region of virtual address space inside which V8 mainly operates and
which should be considered untrusted. Mechanisms like sandboxed pointers
are then used to attempt to prevent escapes from the sandbox (i.e.
corruption of memory outside of it). Furthermore, the new naming scheme
avoids the confusion with the various other "cages" in V8, in
particular, the VirtualMemoryCage class, by dropping that name entirely.

Future sandbox features are developed under their own V8_SANDBOX_X flag,
and will, once final, be merged into V8_SANDBOX. Current future features
are sandboxed external pointers (using the external pointer table), and
sandboxed pointers (pointers guaranteed to point into the sandbox, e.g.
because they are encoded as offsets). This CL then also introduces a new
build flag, v8_enable_sandbox_future, which enables all future features.

Bug: v8:10391
Change-Id: I5174ea8f5ab40fb96a04af10853da735ad775c96
Cq-Include-Trybots: luci.v8.try:v8_linux64_heap_sandbox_dbg_ng,v8_linux_arm64_sim_heap_sandbox_dbg_ng
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3322981
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Samuel Groß <saelo@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78384}
2021-12-15 17:09:36 +00:00
Leszek Swirski
b7acd5100b [dict-tracking] Disable --concurrent-inlining under dict-tracking
Disable --concurrent-inlining when v8_dict_property_const_tracking is
enabled, since the latter doesn't support the former. Add statusfile
variables to mark this as an incompatible flag, and otherwise disable it
via flags.

Change-Id: Ie2e7dac6f4a1ddc97e7e7f4d4d2c77e638e216a0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3229565
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77455}
2021-10-19 12:08:37 +00:00
Samuel Groß
e84ac8bc3b [sandbox] Disallow ArrayBuffers outside the VM Cage
In a follow-up CL, the backing stores will, when the sandbox is enabled,
be referenced from V8 objects through offsets rather than raw pointers.
For that to work, all backing stores must be located inside the virtual
memory cage. This CL prepares for that.

Bug: chromium:1218005
Change-Id: Ibb989626ed7094bd4f02ca15464539f4e2bda90f
Cq-Include-Trybots: luci.v8.try:v8_linux64_heap_sandbox_dbg_ng
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3114136
Commit-Queue: Samuel Groß <saelo@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Cr-Commit-Position: refs/heads/main@{#76486}
2021-08-25 09:52:38 +00:00
Michael Achenbach
fb0a2ea25e Revert "Reland "[build] Add V8-specific dcheck_always_on""
This reverts commit 67960ba110.

Reason for revert:
This has been properly fixed by https://crrev.com/c/3053740.
Now dcheck_always_on already defaults to false for subprojects
like V8 and no other switch is required. The switch didn't fully
work anyways due to https://crbug.com/1231890.

Original change's description:
> Reland "[build] Add V8-specific dcheck_always_on"
>
> This is a reland of cecc666f4d
>
> Depends on:
> https://crrev.com/c/3043611
>
> Original change's description:
> > [build] Add V8-specific dcheck_always_on
> >
> > This makes the V8 dcheck control independent of Chromium's and
> > prepares switching Chromium's default behavior without affecting V8
> > developers or builders.
> >
> > Preparation for: https://crrev.com/c/2893204
> >
> > Bug: chromium:1225701
> > Change-Id: I520b96019b04196f4420716ff3500ebd6c21666f
> > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3038528
> > Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#75827}
>
> Bug: chromium:1225701
> Change-Id: I56568b78592addba01793d2d14f768c9ee10103d
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3041670
> Reviewed-by: Liviu Rau <liviurau@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#75839}

Bug: chromium:1225701, chromium:1231890
Change-Id: I7e27f5774d8e162977f30f685da4b15dadcc1084
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3055294
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#75935}
2021-07-27 11:14:57 +00:00
Michael Achenbach
67960ba110 Reland "[build] Add V8-specific dcheck_always_on"
This is a reland of cecc666f4d

Depends on:
https://crrev.com/c/3043611

Original change's description:
> [build] Add V8-specific dcheck_always_on
>
> This makes the V8 dcheck control independent of Chromium's and
> prepares switching Chromium's default behavior without affecting V8
> developers or builders.
>
> Preparation for: https://crrev.com/c/2893204
>
> Bug: chromium:1225701
> Change-Id: I520b96019b04196f4420716ff3500ebd6c21666f
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3038528
> Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#75827}

Bug: chromium:1225701
Change-Id: I56568b78592addba01793d2d14f768c9ee10103d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3041670
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#75839}
2021-07-21 14:25:24 +00:00
Zhi An Ng
a1472dc1cc Revert "[build] Add V8-specific dcheck_always_on"
This reverts commit cecc666f4d.

Reason for revert: Many crashes on clusterfuzz bots https://ci.chromium.org/ui/p/v8/builders/ci/V8%20Clusterfuzz%20Win64%20ASAN%20-%20release%20builder/23404/overview

Original change's description:
> [build] Add V8-specific dcheck_always_on
>
> This makes the V8 dcheck control independent of Chromium's and
> prepares switching Chromium's default behavior without affecting V8
> developers or builders.
>
> Preparation for: https://crrev.com/c/2893204
>
> Bug: chromium:1225701
> Change-Id: I520b96019b04196f4420716ff3500ebd6c21666f
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3038528
> Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#75827}

Bug: chromium:1225701
Change-Id: I20329f77707caf8fba2405919aa13c67811a1469
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3042841
Auto-Submit: Zhi An Ng <zhin@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#75830}
2021-07-20 22:06:20 +00:00
Michael Achenbach
cecc666f4d [build] Add V8-specific dcheck_always_on
This makes the V8 dcheck control independent of Chromium's and
prepares switching Chromium's default behavior without affecting V8
developers or builders.

Preparation for: https://crrev.com/c/2893204

Bug: chromium:1225701
Change-Id: I520b96019b04196f4420716ff3500ebd6c21666f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3038528
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#75827}
2021-07-20 17:56:20 +00:00
Michael Achenbach
f370d60711 [test] Add verbose output of flaky tests
The verbose output shown on bots didn't print the first failing result
of a flaky test before. Now the result line shows all results and
the details in the end show the output of the first failure.

Previously it was confusing as it seemed that the json results and
the test runner output differed.

We now print PASS in all caps like the other statuses. A test for
this case already existed and the output is now updated.

Bug: v8:8434
Change-Id: I473ec392e0028bf64b3da53d4b37446ffcd17277
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2919670
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74792}
2021-05-26 16:22:59 +00:00
Michael Achenbach
ee56a9863e [test] Run heavy tests sequentially
This adds a new status file indicator "HEAVY" to mark tests with high
resource demands. There will be other tests running in parallel,
but only a limited number of other heavy tests. The limit is
controlled with a new parameter --max-heavy-tests and defaults to 1.

The change also marks a variety of tests as heavy that recently had
flaky timeouts. Heavy also implies slow, hence heavy tests are
executed at the beginning with a higher timeout like other slow tests.

The implementation is encapsulated in the test-processor chain. A
new processor buffers heavy tests in a queue and adds buffered tests
only if other heavy tests have ended their computation.

Bug: v8:5861
Change-Id: I89648ad0030271a3a5af588ecc9c43285b728d6d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2905767
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74712}
2021-05-21 12:59:49 +00:00
Wenyu Zhao
2cd77745d9 [heap] Fix failed tests when enabling single generation
* Filtered some tests that rely on incremental_marking and shape tracking

Bug: v8:11644
Change-Id: Ic9833bf1e49e6413422484858cd1054dd2500092
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2822284
Commit-Queue: Wenyu Zhao <wenyu.zhao@anu.edu.au>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74078}
2021-04-21 07:33:51 +00:00
Ulan Degenbaev
e43e1dcd17 [test] Introduce a third_party_heap test flag
The flag is useful for disabling tests that are not supported in
the third_party_heap build configuration.

Example usage in the status files:
['third_party_heap', {
  'testname': [SKIP],
}],  # third_party_heap

Bug: v8:11155
Change-Id: I991532bf7cdf89d8c505e4d6cbd7cf9e4d70dd63
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2821960
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73957}
2021-04-14 14:55:35 +00:00
Shu-yu Guo
3ada6f2740 [ptr-cage] Introduce PtrComprCage
The pointer compression cage is the virtual memory reservation
that all compressed pointers fall within. This CL splits pointer
compression into two modes: a per-Isolate cage and a shared cage
among multiple Isolates.

When multiple Isolates are sharing a cage, they can decompress
each others' pointers and share the same virtual memory range.

Bug: v8:11460
Change-Id: I7b89b7413b8e7ca6b8b6faafd083dc387542a8b4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2783674
Reviewed-by: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Shu-yu Guo <syg@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73916}
2021-04-12 17:49:43 +00:00
Clemens Backes
1813665413 [no-wasm] Skip wasm tests if wasm is disabled
This CL introduces a test runner flag to detect if webassembly has been
disabled. Since all tests that require wasm are alrady skipped in
lite mode, we introduce a has_webassembly flag for the test runner which
checks for v8_enable_webassembly=true and v8_enable_lite_mode=false.
As a drive-by, we also do not set the V8_ENABLE_WEBASSEMBLY
preprocessor flag if lite mode is enabled.

The status files are updated by splitting wasm tests from the
"lite_mode" section and checking for "not has_webassembly" instead.

Note that the v8_enable_webassembly=false configuration is not tested
on any bot currently, but I will make sure that all tests keep passing
on further changes in this configuration.

R=machenbach@chromium.org

Bug: v8:11238
Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
Change-Id: I1841eb1f1633cb47e0c079f4a4a4d769ca3a9cbb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2710425
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#72898}
2021-02-22 11:06:40 +00:00
Leszek Swirski
def5e9c364 [sparkplug] Disable baseline tests under CFI
Add a .status file variable for the "v8_control_flow_integrity" gn arg,
and disable baseline tests for now in that configuration.

No-Tree-Checks: true
No-Try: true
Bug: v8:11439
Change-Id: I7274a168893cfd6619ce98fdd14a692217fd56c9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2692206
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#72698}
2021-02-12 13:57:39 +00:00
Michael Achenbach
373a9a8cfc [test] Switch to flattened json output
This flattens the json output to one result record as a dict. In
the past several records with different arch/mode combinations
could be run, but this is deprecated since several releases.

We also drop storing the arch/mode information in the record as it
isn't used on the infra side for anything.

This was prepared on the infra side by:
https://crrev.com/c/2453562

Bug: chromium:1132088
Change-Id: I944514dc00a671e7671bcdbcaa3a72407476d7ad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2456987
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70402}
2020-10-08 13:05:11 +00:00
Michael Achenbach
608b732d14 [test] Overhaul mode processing in test runner
This simplifies mode processing as follows:
- Passing the --mode parameter is deprecated.
- The build output is now only searched in the --outdir parameter
that was passed (previously some combinations of mode and outdir
were possible).
- The mode is deduced from the build artifacts based on the gn
arguments "is_debug" and "dcheck_always_on".
- Timeouts and status file entries in release mode with dchecks are
treated like in debug mode.

This change was prepared on the infrastructure side by deprecating
the --mode flag and passing --outdir=out/build:
https://crrev.com/c/2426643

Bug: chromium:1132088, v8:10893
Change-Id: I0f34ebc003b220f07df4ecdbf69ea6c06ac1f66a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2450016
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70363}
2020-10-07 07:09:30 +00:00
Michael Achenbach
ef2e6dc5ee [test] Run more variants on no-CM builder
This additionally combines --future with all other standard testing
variants.

This also enables using concurrent_marking in status files to skip
tests in this variant.

This also marks a slow test that times out in the new config.

Bug: v8:10875
Change-Id: Id904f6a2c51b814eecfccb523a897de2f5d96f56
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2423719
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70180}
2020-09-28 19:04:19 +00:00
Michael Achenbach
4f015e85fa [test] Make finding build directory more flexible
This prepares moving the build directory on bots to out/build. For a
smooth transition, the performance runner will dynamically check
for the build in several locations.

This prepares:
https://crrev.com/c/2426643

NOTREECHECKS=true

Bug: chromium:1132088
Change-Id: Ia12fcdedec0f4ac2bfe087e8154c0acb8771a43f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2431364
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70158}
2020-09-28 12:38:16 +00:00
Michael Achenbach
d8d6110bbe [test] Print less in verbose mode
I/O is quite expensive on the bots. This cuts down a bit of it by
printing slightly fewer characters per test in verbose mode.

This leads to an overall speed improvement of ~20% for large test
suites, e.g. Test262 output-collection time goes from ~2m30 to ~2m.

The averages to a 5-10% overall speed improvement for slow tryjobs.

Bug: v8:10916
Change-Id: I56dcb072af8eb32a1e09e17a05db5782c6d79315
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2414038
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69950}
2020-09-16 15:52:00 +00:00
Liviu Rau
5f7bc600a6 [testing] Collect all relevant data about slow tests
Slow tests are now collected in a heap with a fixed size.
When the maximum size is reached we evict the fastest test
after adding a new test to the heap.

Bug: v8:10168
Change-Id: If3298df85d6e924451f55fe9350e293169cc849d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2106205
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67572}
2020-05-05 17:35:09 +00:00
Michael Achenbach
c12f395da7 [test] Don't use --testing-d8-test-runner on NumFuzz
The extra checks done with --testing-d8-test-runner might not hold
when using NumFuzz. This refactors the test runner and allows passing
implementation specific flags. The --testing-d8-test-runner flag is
now only passed in the standard-runner not in the numfuzz
implementation.

Bug: v8:10220
Change-Id: I83cac57a948c98c34f2d84f41d719e0434e25ee7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2064217
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mythri Alle <mythria@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66341}
2020-02-19 14:40:19 +00:00
Liviu Rau
734169f9b0 [torque] Add support for torque output comparison
Bug: v8:9641
Change-Id: Ia85d14677e8eb7d2d14ff7eb150a1b0b279642f4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1998078
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65892}
2020-01-21 14:39:04 +00:00
Michael Achenbach
d64a48dbdf [testrunner] Prevent erroneous overriding of signal handlers
When an overall timeout is reached, swarming sends a SIGTERM to
terminate the test runner. The test runner has a signal handler on the
main process to terminate all workers gracefully.

Additionally, every worker process installs a signal handler for
terminating ongoing tests wrapped by command.Command.

Also, command.Command is used on the main process to list tests for
cctest and gtest executables, which led to overriding the test runner's
main signal handler.

This CL disables using signal handlers in commands by default and only
explicitly enables it in safe source locations.

Bug: v8:8292
Change-Id: Ifceadaff75bdd2b77e761498bccbe00b6a3e265c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2002528
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65815}
2020-01-16 14:44:14 +00:00
Simon Zünd
2d4fe83a43 Remove 'embedded_builtins' variant
Embedded builtins are now unconditionally enabled, which removes the
need to differentiate between enabled/disabled embedded builtins.

This Cl removes the 'embedded_builtins' variant and related
*.status entries.

R=machenbach@chromium.org

Bug: v8:8519
Change-Id: I55d0dd54735b7cc437832af6fa2836fd6c14a317
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1864936
Commit-Queue: Simon Zünd <szuend@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64464}
2019-10-22 12:58:02 +00:00
Simon Zünd
b1b4ea3ebe Unconditionally enable embedded builtins
This CL removes the 'v8_enable_embedded_builtins' GN argument and sets
the 'V8_EMBEDDED_BUILTIN' define unconditionally.

Removing the now obsolete 'define' and corresponding C++ code will be
done in a separate CL.

R=machenbach@chromium.org

Bug: v8:8519
Change-Id: Ibaa3e958121931c0ce14baf48ad401fab87c5337
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1864929
Commit-Queue: Simon Zünd <szuend@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64456}
2019-10-22 10:48:46 +00:00
Simon Zünd
69efc4c34c Reland "Unconditionally enable snapshot builds and remove 'v8_use_snapshot'"
This is a reland of 1c56974f2a

This is a plain reland of the original CL. The original CL was speculatively
reverted, but ended up not being the cause for bot failures.

Original change's description:
> Unconditionally enable snapshot builds and remove 'v8_use_snapshot'
>
> This CL removes 'v8_use_snapshot' and the usages of the implied
> V8_USE_SNAPSHOT define. One test runner unittest was updated to use the
> "asan" variant instead of the now obsolete "no_snap" variant.
>
> Related chromium CL: https://crrev.com/c/1796325.
>
> Bug: v8:8531
> Change-Id: I5da7c9f8e9110fe7bc0f4e4f821bcb7f7d98f927
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1784282
> Commit-Queue: Simon Zünd <szuend@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Reviewed-by: Jakob Gruber <jgruber@chromium.org>
> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64290}

TBR=thakis@chromium.org,machenbach@chromium.org,mstarzinger@chromium.org,jgruber@chromium.org,tmrts@chromium.org,szuend@chromium.org

Bug: v8:8531
Change-Id: Id75a802279238138f7aefec62e0b6425a5acc08d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1864649
Reviewed-by: Simon Zünd <szuend@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Simon Zünd <szuend@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64305}
2019-10-16 06:01:05 +00:00
Clemens Backes
609f566b34 Revert "Unconditionally enable snapshot builds and remove 'v8_use_snapshot'"
This reverts commit 1c56974f2a.

Reason for revert: Causes several bots to timeout, e.g. https://ci.chromium.org/p/v8/builders/ci/V8%20Linux%20-%20debug/27945

Original change's description:
> Unconditionally enable snapshot builds and remove 'v8_use_snapshot'
> 
> This CL removes 'v8_use_snapshot' and the usages of the implied
> V8_USE_SNAPSHOT define. One test runner unittest was updated to use the
> "asan" variant instead of the now obsolete "no_snap" variant.
> 
> Related chromium CL: https://crrev.com/c/1796325.
> 
> Bug: v8:8531
> Change-Id: I5da7c9f8e9110fe7bc0f4e4f821bcb7f7d98f927
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1784282
> Commit-Queue: Simon Zünd <szuend@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Reviewed-by: Jakob Gruber <jgruber@chromium.org>
> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64290}

TBR=thakis@chromium.org,machenbach@chromium.org,mstarzinger@chromium.org,jgruber@chromium.org,tmrts@chromium.org,szuend@chromium.org

Change-Id: I4024d818877e534b9f7908a2d14f33dca35b5924
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8531
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1862572
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64293}
2019-10-15 14:20:52 +00:00
Simon Zünd
1c56974f2a Unconditionally enable snapshot builds and remove 'v8_use_snapshot'
This CL removes 'v8_use_snapshot' and the usages of the implied
V8_USE_SNAPSHOT define. One test runner unittest was updated to use the
"asan" variant instead of the now obsolete "no_snap" variant.

Related chromium CL: https://crrev.com/c/1796325.

Bug: v8:8531
Change-Id: I5da7c9f8e9110fe7bc0f4e4f821bcb7f7d98f927
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1784282
Commit-Queue: Simon Zünd <szuend@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64290}
2019-10-15 12:44:24 +00:00
Mythri A
b086cb7b9a Check that function was prepared before optimizing using manually
With lazy feedback allocation and bytecode flushing we need to call
%PrepareFunctionForOptimize before we call %OptimizeFunctionOnNextCall/
%OptimizeOsr. This cl:
1. Adds an additional state in pending optimized table to check if the
optimization was triggered manually.
2. Changes the compilation pipeline to delete the entry from pending
optimized table only if the optimization was triggered through
%OptimizeFunctionOnNextCall / %OptimizeOsr.
3. Adds a check to enforce %PrepareFunctionForOptimize was called.
4. Adds a new run-time flag to only check in the d8 test runner. We
don't want this check enabled in other cases like clusterfuzz that doesn't
ensure %PrepareFunctionForOptimize is called.

Bug: v8:8394, v8:8801, v8:9183
Change-Id: I9ae2b2da812e313c746b6df0b2da864c2ed5de51
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1664810
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62653}
2019-07-11 14:57:49 +00:00
Sergiy Belozorov
80451b07bc [tools] Implement confidence-based number of runs
R=machenbach@chromium.org, tmrts@chromium.org

Bug: chromium:880724
Change-Id: I2b8ede244fa09868eef384b967223a3788ddd2a1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1581180
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61370}
2019-05-09 09:42:28 +00:00
Michael Achenbach
c1e08aa52d [test] Increase timeout for full debug and predictable
Bug: v8:9145
Change-Id: Idb1a838666ea21cb260a141929fd80b400913836
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1596731
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61262}
2019-05-07 08:36:41 +00:00
Sergiy Belozorov
b3e2bcf95a [tools] Default to 4 retries for all perf tests
Overall, total test runtime that was wasted due to timeouts is 3420 seconds in
the last 2 weeks. Even with 4 retries, assuming all of them time out, needed
additional capacity is under 2 hours per week. Based on this analysis, I think
it's safe to land this CL.

Note that this is not intended as a long-term solution of the timeout problem,
but rather a temporary solution to prevent ongoing errors. Proper investigation
and correct long-term solution are still needed and tracked in the bug.

R=machenbach@chromium.org, tmrts@chromium.org

Bug: chromium:841700
Change-Id: Id16e6b784fa85bb9e28ed8c6b267b583636e2dc1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1593342
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61224}
2019-05-04 18:25:27 +00:00
Sergiy Belozorov
ceaeca9d2c [tools] Record durations also for unsuccesful runs
This is needed to allow deriving timeout and near_timeout runnables on the
recipe side and drop corresponding fields in the output.

We also remove some unused code that was removed in previous refactoring CLs.

R=tmrts@chromium.org

No-Try: true
No-Tree-Checks: true
Bug: chromium:841700
Change-Id: Ib7cec9baceff994fb9b32be09ba4a4079ebff2c8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1588417
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61095}
2019-04-29 16:00:41 +00:00
Sergiy Belozorov
7a70c55dc0 [tools] Replace yield-based output processing with explicit method calls
This also makes processing immediate, i.e. outputs are parsed into results as
soon as test run is finished, which will allow us to implement logic that checks
whether we have enough runs based on already-accumulated results.

Since we process each output immediately, we do not need Measurement class any
longer and its ConsumeOutput is now integrated directly into TraceConfig.
Similarly AccumulateResults is replaced with RunnableConfig.ProcessOutput as we
do not accumulate results any longer.

R=machenbach@chromium.org

No-Try: true
No-Tree-Checks: true
Bug: chromium:880724
Change-Id: I0fc4846024c43258c10ba8d568312aa4746d746f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1584325
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61048}
2019-04-26 13:33:51 +00:00
Sergiy Belozorov
631c4d4f6b [tools] Refactor Results class into ResultsTracker class based on dicts
This is part of the refactoring needed to implement a feature that allows
re-running benchmarks until needed confidence is reached.

R=machenbach@chromium.org

No-Try: true
No-Tree-Checks: true
Bug: chromium:880724
Change-Id: I45f584a3503ecf567f4c2661a302a74fc5e516af
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1581605
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61037}
2019-04-26 09:57:50 +00:00
Sergiy Belozorov
f8374ee4ae [tools] Export runtime duration and timeout for each runnable
R=machenbach@chromium.org, tmrts@chromium.org

Bug: chromium:841700
Change-Id: I9852ccc573eda6b1bab7a7db295f7fd6e7410581
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1570010
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60967}
2019-04-24 09:25:37 +00:00
Sergiy Belozorov
b2942ae973 [tools] Remove RunnableGenericConfig and its corresponding tests
This is dead code as no benchmarks seem to be using it and removing this
additional logic will reduce maintenance cost for the perf infrastructure.

R=machenbach@chromium.org, tmrts@chromium.org

No-Try: true
No-Tree-Checks: true
Change-Id: I952af069d4d18d63624510b4c461fa1891703db9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1570008
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60947}
2019-04-23 13:00:01 +00:00