Since --shared-space is now enabled by default, we don't need this
flag for testing anymore.
Bug: v8:13267
Change-Id: Ib4c1111a75dbf93d05ccf3bac752c0ef089e3c15
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4194715
Auto-Submit: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#85488}
This will allow on the infra side to easier link to the respective
shard on a test failure. Without that it's a hassle to find out on
which shard the failing test ran.
This also simplifies how the global test_config stores information.
Some information was duplicated, but is now rather shared through
properties if the owning object is already present.
Bug: v8:13681
Change-Id: I52f01a4fac74627575d80f25923faba99eb6a1fb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4181030
Reviewed-by: Liviu Rau <liviurau@google.com>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#85429}
Add custom analysis that is only done if the ClusterFuzz analyzer fails
analyze the stacktrace given without overriding any of its functionality.
Bug: v8:12313
Change-Id: I9e4bc40404387160eb122b8f58999717bd4e7672
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4165085
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#85390}
This replaces the dummy values for the crash analysis. We use the
stack parser from ClusterFuzz to extract the crash type and state from
the stderr of the output and we store it in the test results. We also
added some tests to check the functionality with some example stderrs.
Bug: v8:12313
Change-Id: I67994a752a7aa01cacfb7ebfe7e9d3d61820a4dc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4067428
Reviewed-by: Liviu Rau <liviurau@google.com>
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84789}
Test ids would get an uniform format between different kinds of tests:
prefix//test_id//suffix
- prefix:
- empty for regular tests
- or 'special test' token, as in 'numfuzz'
- test_id is the full name of the test as generated by test runner:
- suite_name/path/to/actual/test_name
- suffix is anything a test runner processor might want to add to the name:
- numfuzz processor will add 'analysis' of a numeric value
- variant processor will add the variant name
Bug: v8:13316
Change-Id: Ied8f958173f82d8e26c62e39ccc21167ca2928ab
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4084763
Commit-Queue: Liviu Rau <liviurau@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84736}
Data added:
- subtest_id as it is generated by Processors
- processor_name to trace back the processors that generate subtests
- path of the test
- test_id suffixes introduced by processors:
- numfuzz will have
- an `analysis` suffix for analysis phase
- a number suffix for the variants generated after analysis
- variant processor adds a suffix for every variant
- subtests will inherit suffixes from the parent tests (origin)
V8-Recipe-Flags: resultdb
Cq-Include-Trybots: luci.v8.try:v8_numfuzz_dbg,v8_numfuzz_rel,v8_numfuzz_tsan_compile_rel,v8_numfuzz_tsan_rel
Bug: v8:13316
Change-Id: I67d8b92b575c31b201238cfbcfc76cd076a2f7af
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4081127
Commit-Queue: Liviu Rau <liviurau@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84720}
Drive by: refactor framework_name propagation. The property was already injected in the TestSuite objects. Since it finally got attached to the result record it was natural to have it attached on the TestCase object at creation time. This eliminates the need to inject it through progress objects.
Change-Id: Ic4028d24589a241fb6225dc53ccef2215728d569
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4079228
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Liviu Rau <liviurau@google.com>
Cr-Commit-Position: refs/heads/main@{#84670}
Adding the crash status and type to the test result with dummy values,
with the intention of populating these values with data from
ClusterFuzz analyzer or our own logic.
Bug: v8:12313
Change-Id: I8a537734a9c48393b77d582516ae45638986aa55
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4047486
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84511}
--use-map-space was already disabled by default. This CL removes
the possibility to enable map space again by removing that flag and
all its usages.
Bug: v8:12578
Change-Id: I8af18f39e9bf645316aa8718b49589eb4b852374
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3959658
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83758}
After removing flag in https://crrev.com/c/3952935 it was still used
on fuzzers.
Bug: v8:12578
Change-Id: I35790b11224c9487923b322f575530159a09b129
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3956734
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Auto-Submit: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83718}
This is a reland of commit 237de893e1
We now guard against requests Python module not being available when running the testrunner. If preconditions (modules & luci context) are not met we no longer add ResultDBIndicator to the chain.
Original change's description:
> [resultdb] Add ResultDB indicator
>
> Adds a new indicator that will send every result to ResultDB (and ultimately in a bq table; to be configured later).
>
> If we are not running in a ResultDB context we introduce only a minimal overhead by exiting early from indicator.
>
> To test these changes in a luci context with ResultDB we activated resultdb feature flag via V8-Recipe-Flags. This feature got implemented in https://crrev.com/c/3925576 .
>
>
> V8-Recipe-Flags: resultdb
> Bug: v8:13316
> Change-Id: I5d98e8f27531b536686a8d63b993313b9d6f62c5
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3905385
> Commit-Queue: Liviu Rau <liviurau@google.com>
> Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#83672}
V8-Recipe-Flags: resultdb
Bug: v8:13316
Change-Id: I0bdfae13cc7f250c41a18f2d3a513a3bfc580f6d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3955263
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Liviu Rau <liviurau@google.com>
Cr-Commit-Position: refs/heads/main@{#83711}
This reverts commit 237de893e1.
Reason for revert: breaks flake bisect:
https://ci.chromium.org/ui/p/v8/builders/try.triggered/v8_flako/b8800423657665797553/overview
The change added the implicit requirement to run testing with
vpython3, which is not given everywhere.
Original change's description:
> [resultdb] Add ResultDB indicator
>
> Adds a new indicator that will send every result to ResultDB (and ultimately in a bq table; to be configured later).
>
> If we are not running in a ResultDB context we introduce only a minimal overhead by exiting early from indicator.
>
> To test these changes in a luci context with ResultDB we activated resultdb feature flag via V8-Recipe-Flags. This feature got implemented in https://crrev.com/c/3925576 .
>
>
> V8-Recipe-Flags: resultdb
> Bug: v8:13316
> Change-Id: I5d98e8f27531b536686a8d63b993313b9d6f62c5
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3905385
> Commit-Queue: Liviu Rau <liviurau@google.com>
> Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#83672}
Bug: v8:13316
Change-Id: I7e55668e365475298ed46d2fc8ee0fe1282c3e8e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3952131
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#83677}
Adds a new indicator that will send every result to ResultDB (and ultimately in a bq table; to be configured later).
If we are not running in a ResultDB context we introduce only a minimal overhead by exiting early from indicator.
To test these changes in a luci context with ResultDB we activated resultdb feature flag via V8-Recipe-Flags. This feature got implemented in https://crrev.com/c/3925576 .
V8-Recipe-Flags: resultdb
Bug: v8:13316
Change-Id: I5d98e8f27531b536686a8d63b993313b9d6f62c5
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3905385
Commit-Queue: Liviu Rau <liviurau@google.com>
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83672}
All --stress-* flags are now automatically tested. This also removes
a superfluous option that was never changed. The default value is
now inlined.
No-Try: true
Bug: v8:13113
Change-Id: If7428b383ed01ff36a93f618badababfc448db26
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3899259
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83258}
A recent refactoring changed the behavior of dropping/keeping
results after test execution. The numfuzz loop has previously
treated all results as analysis results, as it expected that others
are dropped. After keeping all results, the second round invalidated
the analysis results and the test loop stopped early.
We now add an additional safeguard that ensures the received result
is indeed associated with an analysis run and do not depend anymore
on result presence/absence.
This also adds all analysis-based instances to the test cases.
No-Try: true
Bug: v8:13295
Change-Id: Ic1ede904d279a0c2b318ec997e7c77542dbc75bb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3901812
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83244}
Based on bots and local testing, MinorMC has reached a stable state in
terms of correctness.
Enable fuzzing with MinorMC to flush out additional issues.
Bug: v8:12612
Change-Id: I9cf8c5791d7256ff63c777b295863506436ee165
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3872265
Reviewed-by: Almothana Athamneh <almuthanna@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83190}
The numfuzz fuzzer.py has a loop to send a new test after receiving
a result. When all test processors go into stopped state, attempts
of sending new tests return False. That case wasn't handled here
and we kept looping forever.
Bug: v8:13113
Change-Id: Ief2686614d9703fb590400ac3e73b6ac9008c8f6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3891373
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83188}
.. which sets the --interrupt-budget-for-maglev to a very low value
s.t. that tiering to Maglev happens very early. Note this affects
both normal tierup and OSR.
Also add flag handling to fuzzer.py, both as added globally with
probability 0.1, and added to InterruptBudgetFuzzer (which I also
updated with other tiering-related flags).
Bug: v8:7700
Change-Id: I844cf53a6a2da459565d0ad0ccae02b04853cd26
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3878165
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Auto-Submit: Jakob Linke <jgruber@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83033}
This adds Python logging to several code locations that previously
used to just print. The locations aren't yet complete. The changed
code locations should help for investigating hanging test runs.
The default level is WARNING for running tests locally, and INFO
when called from bots that pass the --swarming parameter.
Bug: v8:13113
Change-Id: If3a336703e7c346a5c718f2359b1a80e37e1ca6d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3876183
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83006}
- Unify old Pool interface with the new context related interface
- Add single threaded execution pool
- Defer task killing back to OS context
- Defer process listing in indicators back to OS context
Bug: v8:12785
Cq-Include-Trybots: luci.v8.try:v8_numfuzz_dbg_ng,v8_numfuzz_ng,v8_numfuzz_tsan_ng,v8_android_arm64_n5x_rel_ng
Change-Id: I8ffe01c5d567411203f69ecc451c718ff35d81c9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3781347
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Commit-Queue: Liviu Rau <liviurau@google.com>
Cr-Commit-Position: refs/heads/main@{#82371}
This extends the ExpectedOutProc runner to print a diff between the
expected and actual output. The behavior of other runners is unchanged.
Change-Id: If2b89d39cf98b8d257b1a209b5471a79ec3868ff
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3771641
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Auto-Submit: Matthias Liedtke <mliedtke@google.com>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#81966}
- Removed duplication and unnecessary indirection from all suites testcfgs.
- Introduced a more comprehensive context to cover both command context and other platform specific concerns.
- Propagated above context to TestLoader to allow for test counting command execution on all platforms.
- Wrapped original pool with another class to give it a new interface and allow injecting different implementations in the future.
- Consolidated progress indicators under a single processor in the pipeline.
- Consolidated result retention requirements calculation outside of pipeline chain.
- Refactored LoaderProc and got it under tests.
- Added some more tests for the standard runner.
- Extracted BuildConfig class.
Bug: v8:12785
Change-Id: I87be040e91f792a983662bb5a10d55b36a14ea7f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3701595
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/main@{#81770}
- removed some 'feature envy' instances:
- created a AugmentedOptions class to take care of non trivial option look-ups
- moved some calculation closer the the actual data
- promoted parameter that was passed around a lot (options) to filed in the offending class
- similar object creation looks a bit more similar
- CommandContext provides a wrapper that ensures resource de-allocation
- separate tests from standard and num_fuzz runners
- added a couple of more tests to improve coverage
This is still in flux. But further changes risk creating a disconnect between the original implementation and further refactorings.
Bug: v8:12785
Change-Id: I0ec2e9c6a81604a5cd1d4a80982dd3329c1b48db
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3686411
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#81148}
NumFuzz passes various flags to V8 testing randomly, which can lead to
various flag contradictions with existing flags. Up to now the system
ignored the check for contradictions and kept running the test cases,
leading to false positives.
This change adds a new v8 flag --exit-on-contradictory-flags that
exists gracefully when a contradiction is detected. On the numfuzz
side we now filter simple contradictions beforehand.
Measurements showed that ~2% of all numfuzz tests ran into
contradictions. Around half of them are simple contradictions
(repetitions and inversions), which are now filtered beforehand.
The remaining ones (redundant or contradictory implications) are
now ignored.
Bug: v8:11826
Change-Id: I9942e203ba9668a097fabe1343dd1365c9da94c1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3650746
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Almothana Athamneh <almuthanna@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80589}
Turn on fuzzing for Maglev to get additional test coverage.
Bug: v8:7700
Change-Id: I5b72d851639e31dff1bd91361cd81ad448c2d69e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3629334
Auto-Submit: Jakob Linke <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80416}
To be consistent with the all the other tiers and avoid confusion, we
rename --opt to ---turbofan, and --always-opt to --always-turbofan.
Change-Id: Ie23dc8282b3fb4cf2fbf73b6c3d5264de5d09718
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3610431
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Jakob Linke <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80336}
Turn on fuzzing for disabled map space to get additional test coverage.
Bug: v8:12578
Change-Id: I82c577c8c24b51df627c873fde95fb239e16d36f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3592892
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80012}
This is a follow up CL for https://crrev.com/c/3538284.
Tests credited to https://crrev.com/c/3468352.
- Add unicode filename path test.
- Convert ansi encoded filename argv to utf8 encoded on windows. Because
the ansi encoded filename argv may lose some information for unicode
filename, and we need to use GetCommandLineW to get the actual unicode
filename argument. And we convert it to utf8 encoded to be consistent
with subsequent processing.
- Use REPLACEMENT CHARACTER to replace the characters which cannot be
encoded with sys.stdout.encoding in progress.py.
This CL should be **reverted** if new unicode filenames cause problems.
Bug: v8:12541
Change-Id: Ic5c5ae342b3a5b11c3119452af03c9165d429ed7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3568926
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: 王澳 <wangao.james@bytedance.com>
Cr-Commit-Position: refs/heads/main@{#79850}
Search for all files with testing naming convention and run that on v8_presubmit.
Also modify all PRESUBMIT files in the tools directory to include any test file
with the appropriate naming convention.
Bug: chromium:1306474
Change-Id: I61c1b7c71badbbc3b99705289588aa8280824d66
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3532266
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79651}
Now that we are able to compact map space, we can also get rid of the
map space and allocate maps in the old space instead. This CL introduces
a FLAG_map_space for enabling/disabling the map space but the map space
remains enabled by default for now.
Without a separate space for maps, the GC can't prevent relocation of
maps anymore. Therefore this CL always allows compaction of maps when
running without a map space. Rename flag to --compact-maps to better fit
this scenario.
mkgrokdump and debug_helper also need to be updated to look for maps
also in the old space. The map space is now optional.
Bug: v8:12578
Change-Id: Ic4e4abd0b58bee26e64329b1c92dbccb07d8105a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3424483
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Seth Brenith <seth.brenith@microsoft.com>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79165}
Turn on fuzzing to get additional test coverage.
Bug: v8:12578
Change-Id: Ia614fb9e4ba519163024f3b7b40f22ad6ebb3297
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3442677
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78977}
