A new realm doesn't contain the mocks and suppressions on the global
object for correctness fuzzing. We replace Realm.eval with eval to
keep exercising the code.
Bug: chromium:1071133
Change-Id: Iffe82d37bf08829fc5937c17c2089277403e71dc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2153206
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67187}
Yet another corner case how non-deterministic timestamps slipped into
the tests.
Bug: chromium:1064900
Change-Id: I33e8b4c8141b3854b7eca5d7ad9b45b6f5130d9e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2120584
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66881}
This adds tests for the mock logic used in differential fuzzing. The
tests uncovered a couple of issues in the mock files that are also
fixed.
This also does some minor code clean up in the mock code.
Bug: chromium:1044942
Change-Id: I5b67f70f8b104bb681548f742ab863395a88360f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2043843
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66304}
This reduces the pointer-compression experiment by 3% as it is in
production since a while.
This also uses turbofan when comparing across architectures as tests
run faster and likely find the same or more relevant bugs.
No-Try: true
Change-Id: I35ec97188a101a7cb43454903f7bf9765e57893c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2059995
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66290}
Pass --no-lazy-feedback-allocation in all second runs depending
on a probability. Also combine with --interrupt-budget=100.
This also allows adding several extra flags behind one probability.
The tests are improved to ensure valid flags and configs.
No-Try: true
Bug: v8:10215
Change-Id: I2766ef5044cd8c7096f6b76f39b60b568f550bde
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2059991
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Mythri Alle <mythria@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66289}
This will allow sharing the configs between Python and JavaScript
based fuzzers.
No-Try: true
Bug: chromium:1044942
Change-Id: Idd47ad31430cfed0629dbb5b9dd5fd2ef8c764f6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2051951
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66243}
We ensure now that fuzzers don't produce the f.arguments pattern and
instead replace it with a random variable.
No-Try: true
Bug: chromium:1044942, chromium:1020573
Change-Id: I899985f1a238f36d10fb22f76d93b7d5f444eab1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2051944
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66238}
Un-ignore some files that only failed in obsolete comparisons.
We don't compare eager anymore and console functions are mocked out
in d8 by now. The exponentiation operator bug doesn't repro anymore.
No-Try: true
Bug: chromium:1044942
Change-Id: I0a572836bc9a4a6aa4736447f638a522ff8e8168
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2050400
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66233}
The assumtion that V8 has no output differences within a single line
before a stack overflow, didn't hold. The prefix of e.g. console.info
can lead to a difference in a recursive call.
This change makes foozzie's output capping before a crash work on the
level of characters instead of lines to fix this.
No-Try: true
Bug: chromium:1050942
Change-Id: I13f747caf4f5848d40c31bd4232811285bab3c17
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2049844
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66217}
The bug appears to be fixed.
No-Try: true
Bug: chromium:693426
Change-Id: Ibd029c906dca87a3fd0c3e00be0b795e87390bd4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2037439
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66134}
This will allow uploading repro test cases to clusterfuzz for
already suppressed known issues. This will allow tracking if those
issues still reproduce and that suppressions don't become stale.
No-Try: true
Bug: chromium:1044942
Change-Id: I997f11293c51836b97d143b0fea992055b39955e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2036083
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66114}
Respective differences were fixed and the suppressions didn't fire
anymore according to the statistics.
No-Try: true
Bug: chromium:664068, chromium:667678
Change-Id: I5e1aa802e0b0288a73d0f236c154289ab45314a0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2035878
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66100}
Crashes in the presence of RangeError happen often during differential
fuzzing. Until now we have ignored such cases completely.
After this change we compare as much output as possible when one or
both runs have crashed, dramatically increasing the coverage.
No-Try: true
Bug: chromium:1048099
Change-Id: I923c10e9064b5dc6cae1e39a254e221d2867e0e7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2030914
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66085}
This adds two flags for whitelisting natives on fuzzers:
--allow-natives-for-fuzzing:
Enables a small subset of runtime functions and replaces others with
undefined.
--allow-natives-for-differential-fuzzing
Restricts the allowed runtime functions even further, excluding the
ones that break differential fuzzing.
Bug: chromium:1044942
Change-Id: I890bd4a6ff7c2e483dd74155290a7ace06f85239
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2020773
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66068}
This is a cleanup to remove unneeded flags after these changes (in
https://crrev.com/c/1988548):
* --future does not imply --wasm-tier-up any more, and
* --wasm-tier-up does not imply --liftoff any more.
Instead, now
* --wasm-tier-up is enabled by default,
* --wasm-tier-up has no effect if --liftoff is not set, and
* --future implies --liftoff.
R=ahaas@chromium.org
Bug: chromium:1040061
Change-Id: I5d04ee1f1d84ddcd0654df0e0a4c6298f80aee9e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1993280
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65666}
Some test cases stress all global functions and hang when calling
readline.
No-Try: true
Bug: chromium:1034285
Change-Id: I0eaf64437c0806cf3df0a5306aecea4367763cad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1967381
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65460}
Some fuzz test cases set Math.min to a different function. Let the fake
typed array constructors remember the original Math.min.
No-Try: true
Bug: chromium:1024815
Change-Id: Ia389439fe6757f2497e22187042c5cdf1d1c649f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1958050
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65383}
This adds a regresson test case for the revert reason of:
https://crrev.com/c/1906378
The test data is tidied up by keeping the different fake d8s in
separate build directories like it would be in production.
A new test simulates an architecture difference and ensures we
pass the architecture mocks in all runs.
No-Try: true
Bug: chromium:1023091
Change-Id: Ic33c426ba8eb9c4b6b0fbb66d43c0859dc2edfcd
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1918248
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65140}
This adds an abstraction for command-line arguments for each of the
two comparison runs done in correctness fuzzing. No functional
changes intended.
No-Try: true
Bug: chromium:1023091
Change-Id: I9421715c4904416b9aaf53848954a5248c79ffd9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906372
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64979}
This is a reland of 1d493d31ce
Original change's description:
> [foozzie] Refactor command abstraction
>
> This moves code for running d8 into its own class. No functional
> changes intended.
>
> No-Try: true
> Bug: chromium:1023091
> Change-Id: I7cbfeebd2911dc758322f89cf93666550f2956d9
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906378
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64928}
Bug: chromium:1023091
Change-Id: I7df6e12084e20510a400ce209827c2bba8325f86
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1914209
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64966}
This moves code for running d8 into its own class. No functional
changes intended.
No-Try: true
Bug: chromium:1023091
Change-Id: I7cbfeebd2911dc758322f89cf93666550f2956d9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906378
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64928}
We used the same random seed for all test cases of a fuzz session
for transitioning from choosing the flags on V8 side.
Since the grace period for stable bisection is over, we now use
the same random number generator throughout the fuzz session which
leads to a wider range of differently chosen flags.
TBR=tmrts@chromium.org
No-Try: true
Bug: chromium:813833
Change-Id: I07b9fe5de378c01344afd486bfd85fcbf0fcd8d2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906377
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64910}
This adds a fake toolchain for pointer compression, used for
correctness fuzzing. The toolchain enables us to have an extra build
with inverse pointer-compression defaults side-by-side.
The extra build is used similarly to existing x64/x86 comparisons,
except that we now compare builds with different compile-time flags.
Change-Id: I75491371262204b86eaa006ca8d04848f49121ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1829275
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64123}
The sanity checks never really worked since the file was missing in the
output dir. Runs bailed out with "Error loading file".
As both configurations bailed out in the same way, the correctness fuzzer
treated it as pass.
NOTRY=true
Bug: chromium:933076
Change-Id: Id3a2f7e8e3fa8c6654e3e08654d2e9e6b7010484
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1778289
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63495}
This also ignores the line number for type errors.
NOTRY=true
Bug: chromium:995075
Change-Id: I9d9a91842383eba60db858c2c2e02df7dda1a9e1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1778287
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63494}
This makes sure the language mode of the module is correctly propagated
through the WebAssembly module, so that exported functions are allocated
with the correct language mode. It extends the existing {ModuleOrigin}
enum to consist of three values now.
R=clemensh@chromium.org
TEST=mjsunit/regress/wasm/regress-985154
BUG=chromium:985154
Change-Id: Id7b566738b1e710cc5001b894022bcd0f2c01bc3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1708484
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62826}
In order to migrate the extra flags into the fuzzer and keep bisection stable,
we need to use the same RNG state for each call to generating fuzz flags.
Throughout one fuzzing session the same random-seed is used
(https://crbug.com/983128) and we'll pass it to the fuzz config in a follow up.
TBR=tmrts@chromium.org
NOTRY=true
Bug: chromium:813833
Change-Id: I3203c86028a5d283238e6ef739f82eccee1302b1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1697254
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62646}
Currently, probabilities for extra flags are calculated in the correctness
fuzzer harness, which makes the RNG fragile when bisecting backwards, when
the script's config changes during bisection.
This adds the possibility to pass extra flags on command line to the
script. After a grace period, we will migrate the flag calculation to
clusterfuzz.
NOTRY=true
Bug: chromium:813833
Change-Id: I515181847474515089b847f8aaffc7c6560d9390
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1675945
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62359}
To correctly fuzz the behaviour of optimized code and ICs we need to
allocate feedback vectors. So for the configurations testing these we
should turn off lazy feedback allocation.
It is also good to fuzz without lazy feedback allocation on other
configurations to flush out any other issues. So we also fuzz this with
0.35 chance. We also fuzz aggressive lazy feedback allocation (allocate
feedback vectors on first branch / return) with 0.05 chance to test
corner cases related to lazy feedback allocation.
Bug: v8:9342
Change-Id: Id0761d1396bfc0866988abb8fb20168b86a5da20
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1672939
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62347}
We have too many dupes in the no-ic comparisons. We'll increase the
experiment size again once bugs are fixed.
TBR=jarin@chromium.org
NOTRY=true
Bug: chromium:961709
Change-Id: Ic946100b45fd73e1bee59f188a766384836bcdcf
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1660624
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62212}
