This is a JavaScript fuzzer originally authored by Oliver Chang. It
is a mutation based fuzzer using Babel code transformations. For more
information see the included README.md.
The original code was altered:
- Add new V8 copyright headers.
- Make the test expectation generator aware of the headers.
- Fix file endings for presubmit checks.
- Fix `npm test` on fresh checkout with a new fake DB.
- Make test skipping work with new v8/tools location.
- OWNERS file.
- New title section in README.md.
No-Try: true
Bug: chromium:1109770
Change-Id: Ie71752c0a37491a50500c49060a3c526716ef933
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2320330
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69164}
This is another case of the known .caller difference that's now added
to the mapping of known issues.
No-Try: true
Bug: chromium:1101870
Change-Id: I6cfca6887362564f625648ba34820cb92a77efb6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2280087
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68674}
Mostly gc-related flags didn't flush out any bugs yet, but often reduce
test performance and lead to timeouts.
No-Try: true
Bug: chromium:1044942
Change-Id: I2a7b55f78bfa3d597de1a5674658829e0812d01a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2273861
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68608}
Previously we ran baseline (e.g. ignition) and one random secondary
comparison configuration (e.g. turbofan) from the list of experiments.
But Clusterfuzz imposes limitations on the total amount of fuzz tests.
Therefore this change enables more throughput by always running the
default configuration (ignition_turbofan like V8 is shipped)
additionally to the baseline and the secondary configuration.
This, hence, doubles the number of comparisons we run, with less than
50% additional runtime, since the slow baseline configuration is only
run once.
The experiments table is updated accordingly. Explicit entries running
ignition_turbofan are removed (as it always runs now), instead some
of the other configurations are increased in their relative
percentage. We also get a few new configurations that didn't run
before (e.g. forcing the slow path on x86).
No-Try: true
Bug: chromium:1100114
Change-Id: I69b2a41d78c06e556b309743a2aace1053c22f91
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270307
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68607}
This shares the code of running the comparisons on the fuzz test
and on the smoke tests. While this change retains behavior for running
the fuzz test, the smoke test has slight changes in favor of code
readability:
1. Smoke tests can now bail out on output suppressions (There are
none now. And if we'd ever add any we'd need to write them in a way,
such that they ignore the output of the smoke tests, which don't
have much output anyways).
2. Crashes in smoke tests are now a hard failure. This is a desired
feature anyways. It's unlikely that the smoke test crashes as then
nothing would work.
No-Try: true
Bug: chromium:1100114
Change-Id: Ice3c6d18b549727c486a70756a72038c8b2029ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2273125
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68586}
This adds a class representing all values needed for an execution.
It simplifies the parameters passed to formatting functions and
lifts the restriction of a first/second config pair in the options.
This will allow to run more than two configs in a follow up.
No-Try: true
Bug: chromium:1100114
Change-Id: I568253b8ca3220870b8f5af615108140587680a8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270550
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68583}
This makes output and test-case suppressions independent of the used
comparison configs and architecture. Such fine-grained suppressions
were only needed during the inception of differential fuzzing, but
by now, most remaining suppressions are implemented in d8 behind
a flag.
This prepares for running with more than two comparison configs in a
follow up.
No-Try: true
Bug: chromium:1100114
Change-Id: I072769adb3ef7c6e6c43459aa23ac906f461b307
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270095
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68579}
It is obsolete to filter out error-message differences since the
time we pass --correctness-fuzzer-suppressions to d8, which already
stubs all messages:
https://cs.chromium.org/chromium/src/v8/src/execution/messages.cc?l=1031
No-Try: true
Bug: chromium:1100114
Change-Id: Iac42a8e2a32f9bae4034f79eaff429bf3ee41724
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270024
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68577}
This simplifies the lengthy main method by extracting some code and
by replacing the scattered returns with exceptions.
We introduce two exceptions for early bail-out. This enables helper
methods on multiple layers. The early bail-out on time-out is
moved to the point where it is detected.
Previously on timeout and crash we also printed out the step number.
Clusterfuzz doesn't parse this, it was only for statistical purposes,
and the latest version of the experimental workbench only parses
crashes and timeouts, not the step in which they happened. Hence,
this CL removes those step numbers.
Except the change described in the last paragraph, this CL doesn't
intend to change behavior.
No-Try: true
Bug: chromium:1100114
Change-Id: Ie8c18f183e4fc538577f3eb49aaf6df1acd1e4e1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270547
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68576}
If we pass flags that make runs very slow, also the validity checks
might time out. Previously this wasn't checked and output was just
cut off.
This also tightens the timeout on validity checks as they are
expected to run very fast.
No-Try: true
Bug: chromium:1098646
Change-Id: Iea9a932be86e84040b72a2311aaa1d44100b3378
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2262915
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68510}
This fixes a bug when an encoded character appears in the difference
string. Python3 doesn't require any encoding.
TBR=tmrts@chromium.org
No-Try: true
Bug: chromium:1095964
Change-Id: I49c66b5b9c105ad64d3a7839d0eb5df97ff5f404
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2249660
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68384}
This prepares using ochang_js_fuzzer with foozzie. The fuzzer uses
tests from CrashTests in the corpus. This leads to a loop when
used with differential fuzzing, as foozzie dedupes failures based
on the original file path. Foozzie finds a new failure for the
existing failure in CrashTests, for which clusterfuzz creates a new
crash test and so on.
This subsumes all failures from CrashTests under the same key.
Once such a failure is reported, a developer can add it to a
mapping in foozzie.py, after which the global key can be used
again by clusterfuzz to report another failure.
No-Try: true
Bug: chromium:1044942
Change-Id: I801a23faeb0c672d6ad64b4100c463f53e36cbc2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2214837
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68053}
This prevents bug flooding based on differences from calling
%GetOptimizationStatus in correctness tests. It is supposed to
be suppressed with --allow-natives-for-differential-fuzzing.
This ensures early bail-out in case the flag is forgotten at
some point. The v8_sanity_checks.js file is executed before
each correctness test case for this purpose.
NOTRY=true
Bug: chromium:1044942
Change-Id: I74a836a82562604b35e94e5e123a2a8bff939423
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2184294
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67618}
A new realm doesn't contain the mocks and suppressions on the global
object for correctness fuzzing. We replace Realm.eval with eval to
keep exercising the code.
Bug: chromium:1071133
Change-Id: Iffe82d37bf08829fc5937c17c2089277403e71dc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2153206
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67187}
Yet another corner case how non-deterministic timestamps slipped into
the tests.
Bug: chromium:1064900
Change-Id: I33e8b4c8141b3854b7eca5d7ad9b45b6f5130d9e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2120584
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66881}
This adds tests for the mock logic used in differential fuzzing. The
tests uncovered a couple of issues in the mock files that are also
fixed.
This also does some minor code clean up in the mock code.
Bug: chromium:1044942
Change-Id: I5b67f70f8b104bb681548f742ab863395a88360f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2043843
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66304}
This reduces the pointer-compression experiment by 3% as it is in
production since a while.
This also uses turbofan when comparing across architectures as tests
run faster and likely find the same or more relevant bugs.
No-Try: true
Change-Id: I35ec97188a101a7cb43454903f7bf9765e57893c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2059995
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66290}
Pass --no-lazy-feedback-allocation in all second runs depending
on a probability. Also combine with --interrupt-budget=100.
This also allows adding several extra flags behind one probability.
The tests are improved to ensure valid flags and configs.
No-Try: true
Bug: v8:10215
Change-Id: I2766ef5044cd8c7096f6b76f39b60b568f550bde
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2059991
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Mythri Alle <mythria@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66289}
This will allow sharing the configs between Python and JavaScript
based fuzzers.
No-Try: true
Bug: chromium:1044942
Change-Id: Idd47ad31430cfed0629dbb5b9dd5fd2ef8c764f6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2051951
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66243}
We ensure now that fuzzers don't produce the f.arguments pattern and
instead replace it with a random variable.
No-Try: true
Bug: chromium:1044942, chromium:1020573
Change-Id: I899985f1a238f36d10fb22f76d93b7d5f444eab1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2051944
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66238}
Un-ignore some files that only failed in obsolete comparisons.
We don't compare eager anymore and console functions are mocked out
in d8 by now. The exponentiation operator bug doesn't repro anymore.
No-Try: true
Bug: chromium:1044942
Change-Id: I0a572836bc9a4a6aa4736447f638a522ff8e8168
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2050400
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66233}
The assumtion that V8 has no output differences within a single line
before a stack overflow, didn't hold. The prefix of e.g. console.info
can lead to a difference in a recursive call.
This change makes foozzie's output capping before a crash work on the
level of characters instead of lines to fix this.
No-Try: true
Bug: chromium:1050942
Change-Id: I13f747caf4f5848d40c31bd4232811285bab3c17
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2049844
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66217}
The bug appears to be fixed.
No-Try: true
Bug: chromium:693426
Change-Id: Ibd029c906dca87a3fd0c3e00be0b795e87390bd4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2037439
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66134}
This will allow uploading repro test cases to clusterfuzz for
already suppressed known issues. This will allow tracking if those
issues still reproduce and that suppressions don't become stale.
No-Try: true
Bug: chromium:1044942
Change-Id: I997f11293c51836b97d143b0fea992055b39955e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2036083
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66114}
Respective differences were fixed and the suppressions didn't fire
anymore according to the statistics.
No-Try: true
Bug: chromium:664068, chromium:667678
Change-Id: I5e1aa802e0b0288a73d0f236c154289ab45314a0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2035878
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66100}
Crashes in the presence of RangeError happen often during differential
fuzzing. Until now we have ignored such cases completely.
After this change we compare as much output as possible when one or
both runs have crashed, dramatically increasing the coverage.
No-Try: true
Bug: chromium:1048099
Change-Id: I923c10e9064b5dc6cae1e39a254e221d2867e0e7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2030914
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66085}
This adds two flags for whitelisting natives on fuzzers:
--allow-natives-for-fuzzing:
Enables a small subset of runtime functions and replaces others with
undefined.
--allow-natives-for-differential-fuzzing
Restricts the allowed runtime functions even further, excluding the
ones that break differential fuzzing.
Bug: chromium:1044942
Change-Id: I890bd4a6ff7c2e483dd74155290a7ace06f85239
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2020773
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66068}
This is a cleanup to remove unneeded flags after these changes (in
https://crrev.com/c/1988548):
* --future does not imply --wasm-tier-up any more, and
* --wasm-tier-up does not imply --liftoff any more.
Instead, now
* --wasm-tier-up is enabled by default,
* --wasm-tier-up has no effect if --liftoff is not set, and
* --future implies --liftoff.
R=ahaas@chromium.org
Bug: chromium:1040061
Change-Id: I5d04ee1f1d84ddcd0654df0e0a4c6298f80aee9e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1993280
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65666}
Some test cases stress all global functions and hang when calling
readline.
No-Try: true
Bug: chromium:1034285
Change-Id: I0eaf64437c0806cf3df0a5306aecea4367763cad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1967381
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65460}
Some fuzz test cases set Math.min to a different function. Let the fake
typed array constructors remember the original Math.min.
No-Try: true
Bug: chromium:1024815
Change-Id: Ia389439fe6757f2497e22187042c5cdf1d1c649f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1958050
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65383}
This adds a regresson test case for the revert reason of:
https://crrev.com/c/1906378
The test data is tidied up by keeping the different fake d8s in
separate build directories like it would be in production.
A new test simulates an architecture difference and ensures we
pass the architecture mocks in all runs.
No-Try: true
Bug: chromium:1023091
Change-Id: Ic33c426ba8eb9c4b6b0fbb66d43c0859dc2edfcd
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1918248
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65140}
This adds an abstraction for command-line arguments for each of the
two comparison runs done in correctness fuzzing. No functional
changes intended.
No-Try: true
Bug: chromium:1023091
Change-Id: I9421715c4904416b9aaf53848954a5248c79ffd9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906372
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64979}
