Turn on fuzzing for disabled map space to get additional test coverage.
Bug: v8:12578
Change-Id: I82c577c8c24b51df627c873fde95fb239e16d36f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3592892
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80012}
This is a follow up CL for https://crrev.com/c/3538284.
Tests credited to https://crrev.com/c/3468352.
- Add unicode filename path test.
- Convert ansi encoded filename argv to utf8 encoded on windows. Because
the ansi encoded filename argv may lose some information for unicode
filename, and we need to use GetCommandLineW to get the actual unicode
filename argument. And we convert it to utf8 encoded to be consistent
with subsequent processing.
- Use REPLACEMENT CHARACTER to replace the characters which cannot be
encoded with sys.stdout.encoding in progress.py.
This CL should be **reverted** if new unicode filenames cause problems.
Bug: v8:12541
Change-Id: Ic5c5ae342b3a5b11c3119452af03c9165d429ed7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3568926
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: 王澳 <wangao.james@bytedance.com>
Cr-Commit-Position: refs/heads/main@{#79850}
Search for all files with testing naming convention and run that on v8_presubmit.
Also modify all PRESUBMIT files in the tools directory to include any test file
with the appropriate naming convention.
Bug: chromium:1306474
Change-Id: I61c1b7c71badbbc3b99705289588aa8280824d66
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3532266
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79651}
Now that we are able to compact map space, we can also get rid of the
map space and allocate maps in the old space instead. This CL introduces
a FLAG_map_space for enabling/disabling the map space but the map space
remains enabled by default for now.
Without a separate space for maps, the GC can't prevent relocation of
maps anymore. Therefore this CL always allows compaction of maps when
running without a map space. Rename flag to --compact-maps to better fit
this scenario.
mkgrokdump and debug_helper also need to be updated to look for maps
also in the old space. The map space is now optional.
Bug: v8:12578
Change-Id: Ic4e4abd0b58bee26e64329b1c92dbccb07d8105a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3424483
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Seth Brenith <seth.brenith@microsoft.com>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79165}
Turn on fuzzing to get additional test coverage.
Bug: v8:12578
Change-Id: Ia614fb9e4ba519163024f3b7b40f22ad6ebb3297
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3442677
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78977}
This also fuzzes values of --budget-for-feedback-vector-allocation.
Boundaries for the intervals are the default values in the code.
No-Try: true
Bug: v8:12434
Change-Id: I0a9d7421408a51c717c2edfe0e67c459f0a2834c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3303792
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Reviewed-by: Almothana Athamneh <almuthanna@chromium.org>
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78111}
This fuzzes more values of interrupt-budget on numfuzz. For now
as a single instance. In a follow up we add it also to combined
flags.
No-Try: true
Bug: v8:12434
Change-Id: I836c5e829ffeabfa4a4686d4d3d2fd43fce1ee88
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3302797
Reviewed-by: Almothana Athamneh <almuthanna@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#78107}
This flag will force the usage of the mid-tier register allocator, which
is otherwise only used for huge functions, hence does not get much
fuzzer coverage.
R=machenbach@chromium.org
Bug: v8:12330
Change-Id: I30b602b54adf858a2a3aaf73d8828875c1f6df7a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3234492
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77500}
This makes flako usable on Windows using standard test names.
A workaround to bisecting to failures before this CL is to replace the
test_name by something like "mjsunit/regress\regress-1138075", i.e.
using this on the command line to trigger flako:
-p 'test_name="mjsunit/regress\\regress-1138075"'
R=liviurau@chromium.org
Bug: v8:9218
Change-Id: I37596efcaeca780eeacb27c2841fe8302ddb1e49
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3081610
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#76222}
The verbose output shown on bots didn't print the first failing result
of a flaky test before. Now the result line shows all results and
the details in the end show the output of the first failure.
Previously it was confusing as it seemed that the json results and
the test runner output differed.
We now print PASS in all caps like the other statuses. A test for
this case already existed and the output is now updated.
Bug: v8:8434
Change-Id: I473ec392e0028bf64b3da53d4b37446ffcd17277
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2919670
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74792}
This adds a new status file indicator "HEAVY" to mark tests with high
resource demands. There will be other tests running in parallel,
but only a limited number of other heavy tests. The limit is
controlled with a new parameter --max-heavy-tests and defaults to 1.
The change also marks a variety of tests as heavy that recently had
flaky timeouts. Heavy also implies slow, hence heavy tests are
executed at the beginning with a higher timeout like other slow tests.
The implementation is encapsulated in the test-processor chain. A
new processor buffers heavy tests in a queue and adds buffered tests
only if other heavy tests have ended their computation.
Bug: v8:5861
Change-Id: I89648ad0030271a3a5af588ecc9c43285b728d6d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2905767
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74712}
This will give extra coverage of the mode on the numfuzz tsan bot.
No-Try: true
Bug: chromium:11655
Change-Id: Ifcc4ad304116fa1b0331cc48e41c07405193d514
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2862809
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74316}
This flattens the json output to one result record as a dict. In
the past several records with different arch/mode combinations
could be run, but this is deprecated since several releases.
We also drop storing the arch/mode information in the record as it
isn't used on the infra side for anything.
This was prepared on the infra side by:
https://crrev.com/c/2453562
Bug: chromium:1132088
Change-Id: I944514dc00a671e7671bcdbcaa3a72407476d7ad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2456987
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70402}
This is a reland of 0ba115e6a9
Changes compared to last reland:
- Fix Python code trying to write to expected_outcomes, which is now a
computed property.
- Fix remaining place in d8.cc that ignored the --fuzzing flag.
- Expect flag contradictions for --cache in code_serializer variant.
Original change's description:
> Reland^3 "[flags] warn about contradictory flags"
>
> Changes:
> - Also allow second parameter influenced by --cache to be reassigned.
> - Fix --stress-opt to only --always-opt in the last iteration as before.
>
> Original change's description:
> > Reland^2 "[flags] warn about contradictory flags"
> >
> > This is a reland of d8f8a7e210
> > Change compared to last reland:
> > - Do not check for d8 flag contradictions in the presence of --fuzzing
> > - Allow identical re-declaration of --cache=*
> >
> > Original change's description:
> > > Reland "[flags] warn about contradictory flags"
> > >
> > > This is a reland of b8f9166664
> > > Difference to previous CL: Additional functionality to specify
> > > incompatible flags based on GN variables and extra-flags, used
> > > to fix the issues that came up on the waterfall.
> > >
> > > This also changes the rules regarding repeated flags: While
> > > explicitly repeated flags are allowed for boolean values as long
> > > as they are identical, repeated flags or explicit flags in the
> > > presence of an active implication are disallowed for non-boolean
> > > flags. The latter simplifies specifying conflict rules in
> > > variants.py. Otherwise a rule like
> > >
> > > INCOMPATIBLE_FLAGS_PER_EXTRA_FLAG = {
> > > "--gc-interval=*": ["--gc-interval=*"],
> > > }
> > >
> > > wouldn't work because specifying the same GC interval twice
> > > wouldn't actually count as a conflict. This was an issue with
> > > test/mjsunit/wasm/gc-buffer.js, which specifies
> > > --gc-interval=500 exactly like the extra flag by the stress bot.
> > >
> > > Also, this now expands contradictory flags checking to d8 flags
> > > for consistency.
> > >
> > > Original change's description:
> > > > [flags] warn about contradictory flags
> > > >
> > > > Design Doc: https://docs.google.com/document/d/1lkvu8crkK7Ei39qjkPCFijpNyxWXsOktG9GB-7K34jM/
> > > >
> > > > Bug: v8:10577
> > > > Change-Id: Ib9cfdffa401c48c895bf31caed5ee03545beddab
> > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2154792
> > > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > > Cr-Commit-Position: refs/heads/master@{#68168}
> > >
> > > Bug: v8:10577
> > > Change-Id: I268e590ee18a535b13dee14eeb15ddd0a9ee8341
> > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2235115
> > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > > Reviewed-by: Tamer Tas <tmrts@chromium.org>
> > > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > > Reviewed-by: Georg Neis <neis@chromium.org>
> > > Cr-Commit-Position: refs/heads/master@{#68989}
> >
> > Bug: v8:10577
> > Change-Id: I31d2794d4f9ff630f3444210100c64d67d881276
> > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2339464
> > Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> > Reviewed-by: Clemens Backes <clemensb@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#69339}
>
> Bug: v8:10577
> Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng
> Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_isolates_rel_ng
> Change-Id: I4a69dc57a102782cb453144323e3752ac8278624
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2352770
> Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#69433}
Change-Id: Ib6d2aeb495210f581ac671221c265df58e8e5e70
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2398640
Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69954}
I/O is quite expensive on the bots. This cuts down a bit of it by
printing slightly fewer characters per test in verbose mode.
This leads to an overall speed improvement of ~20% for large test
suites, e.g. Test262 output-collection time goes from ~2m30 to ~2m.
The averages to a 5-10% overall speed improvement for slow tryjobs.
Bug: v8:10916
Change-Id: I56dcb072af8eb32a1e09e17a05db5782c6d79315
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2414038
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69950}
This adds a simple test runner that prints a line for every test with
the appropriate status prefix: PASS, FAIL, CRASH or TIMEOUT
Change-Id: Ic1ba78667c38cd4392af027bb6cb671b274680b7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2264098
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68570}
This modernizes python code without breaking Py2 compat.
Ran with command:
futurize --stage1 -w tools/testrunner/testpro
Manual fixup to util_unittest to modify import paths and change to
absolute imports.
Bug: v8:9871
Change-Id: I2ac29622aff5daebc9dc42145c1d96dff8258546
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2252549
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Zhi An Ng <zhin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68492}
Passing tests that are marked as fail in a status file are not
immediately visible as such.
- Always show "--- FAILED ---" for failing tests
- Show "--- UNEXPECTED PASS ---" for unexpectedly passing tests
Drive-by-fixes:
- Color failures in red with --progress=color
- Color repro command in yellow with --progress=color
Change-Id: Id43ecec348dbfd4ff627ea6aa4ba458a2e5a8445
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2213434
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68131}
Slow tests are now collected in a heap with a fixed size.
When the maximum size is reached we evict the fastest test
after adding a new test to the heap.
Bug: v8:10168
Change-Id: If3298df85d6e924451f55fe9350e293169cc849d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2106205
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67572}
If we do not want _all_ tests durations, it makes sense to reuse the existing slow test collection and aggregation
Bug: v8:10168
Change-Id: I500acdb799f41ee6f0fb2c57afb95e1e1830b2dd
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2064221
Auto-Submit: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66342}
This adds 3 flags to the numfuzz fuzzer depending on a probability:
--budget-for-feedback-vector-allocation=0
--interrupt-budget=100
--no-lazy-feedback-allocation
No-Try: true
Bug: v8:10204
Change-Id: I83dabcd0e3ca80bebe596d65d03b3e99d8ecbf03
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2060490
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66291}
We will be able to collect test duration and later upload them in BQ.
Change-Id: Ie5610d4e872259857bf3f26ba698fa65d23058be
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2020952
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66040}
This is a reland of ab1b511c16
The offending flags are removed.
Original change's description:
> [test] Add more flags to numfuzz flags fuzzer
>
> This adds a selection of flags to numfuzz that are already used
> for different testing variants or on clusterfuzz for
> correctness testing.
>
> No-Try: true
> Change-Id: I79745b281b001f57d2b24977f3a8e9ce3bbab2a4
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906573
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64884}
No-Try: true
Change-Id: Ie01f244147be0b0fda8cec83f48ac3f73c5a81ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910113
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64905}
This reverts commit ab1b511c16.
Reason for revert: too many spurious errors
Original change's description:
> [test] Add more flags to numfuzz flags fuzzer
>
> This adds a selection of flags to numfuzz that are already used
> for different testing variants or on clusterfuzz for
> correctness testing.
>
> No-Try: true
> Change-Id: I79745b281b001f57d2b24977f3a8e9ce3bbab2a4
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906573
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64884}
TBR=machenbach@chromium.org,clemensb@chromium.org,almuthanna@google.com,liviurau@chromium.org
Change-Id: Iba9cfa8e6e8e2cb3b9fe0f803b07376ae55d783c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910112
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64895}
Produces too many spurious errors with tests that normally get
skipped in jitless variant.
TBR=clemensb@chromium.org
No-Try: true
Change-Id: Iddf0e39e4c454a3b17568ba17a014e8d38922052
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1910107
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64889}
This adds a selection of flags to numfuzz that are already used
for different testing variants or on clusterfuzz for
correctness testing.
No-Try: true
Change-Id: I79745b281b001f57d2b24977f3a8e9ce3bbab2a4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906573
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64884}
This combines flags switching off CPU-features randomly with all
other flags on the numfuzz fuzzers.
In a follow up we can add also other interesting flags for testing
more features.
Bug: chromium:1021463
No-Try: true
Change-Id: Ia27d1b8b82dff2bf497242b30ec0b6ef2dfd722d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1905846
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64858}
Using test runner with option --progress=ci can generate
output timeouts in an actual CI environment. To avoid
that we gonna write a timestamp in the standard output
at every minute.
Bug: v8:9146
Change-Id: Id2f05530956b01d9b07809e509cd0cefc0be54b2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1863196
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64311}
Also remove or adapt all mentioning of buildbot. The flag was removed
on the infra side here:
https://crrev.com/c/1827557
Change-Id: I8a6be80faa65f99cd42a084c804fc5bfae7f02c2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1826722
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64017}
The new indicator behaves like verbose without
printing the names of tests that passed. Also
a new option (--ci-test-completion=/path) was
added to represent a file where we can collect
test completion messages.
Bug: v8:9146
Change-Id: I0f1bbef4036a3019b60b094687b734d3d33a5915
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1806916
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63837}
