Robustify %CompileBaseline against fuzzing, and allowlist it in the
fuzzer.
Bug: v8:11420
Change-Id: I44947014c8c9362d80ea98636dbbaa5d07d6a177
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2739643
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73325}
This is a reland of 4ad08c82f7
The reland organizes the different error types in separate functions
for separate call stacks. Error simulation is also guarded by
a minimum file size to prevent Clusterfuzz from getting stuck with
its bad-build check.
Original change's description:
> Enable simulating errors to test fuzzer reliability
>
> This adds a d8 flag --simulate-errors, which on shutdown will cause
> certain errors. This enables testing the reliability of sanitizers.
>
> This will cause a fatal error, a dcheck (if available) or a
> violation that can be detected with one of the following sanitizers:
> ASAN, UBSAN, MSAN, CFI.
>
> The same flag used in differential fuzzing will cause an error
> subsumed with the error state "fake_difference".
>
> Bug: chromium:1152412
> Change-Id: I4b36c6fe716797004d634263617d22ca67b05600
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2554999
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#71430}
Bug: chromium:1152412
Change-Id: I604258b4c1ebd215c26b1de6b2822663f857bf64
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2565125
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71538}
When one comparison run crashes, we cap the outputs to compare to the
shorter one. If one of those, however, contains ignored lines, the
comparison get's skewed.
This makes the main source of ignored lines more robust (the line
printed for unknown flags), by not printing it in the first place in
the context of differential fuzzing.
Bug: chromium:1153871
Change-Id: If2e534959779be14a686be5e43630cbf66e215a0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2567692
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71522}
This reverts commit 4ad08c82f7.
Reason for revert: Need to retry with bugs in separate stack
frames.
Original change's description:
> Enable simulating errors to test fuzzer reliability
>
> This adds a d8 flag --simulate-errors, which on shutdown will cause
> certain errors. This enables testing the reliability of sanitizers.
>
> This will cause a fatal error, a dcheck (if available) or a
> violation that can be detected with one of the following sanitizers:
> ASAN, UBSAN, MSAN, CFI.
>
> The same flag used in differential fuzzing will cause an error
> subsumed with the error state "fake_difference".
>
> Bug: chromium:1152412
> Change-Id: I4b36c6fe716797004d634263617d22ca67b05600
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2554999
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#71430}
TBR=machenbach@chromium.org,clemensb@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:1152412
Change-Id: If20cdef372b0e7e92e7080687f446539a587a815
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2565120
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71475}
The error simulation works and crashes during the smoke test are
detected. But now this masks real crashes (https://crbug.com/1153200),
therefore this change subsumes smoke-test crashes with
--simulate-errors under a separate crash state.
Now Clusterfuzz will detect the case "unexpected crash" as fixed, but
create a new bug for "simulated crash". Which we will keep open
forever.
No-Try: true
Bug: chromium:1152412
Change-Id: I511af167d33430d9b89692151cb8abaf3a90c715
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2563270
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71452}
This passes the --simulate-errors flag for differential fuzzing with
a probability of 0.01%.
No-Try: true
Bug: chromium:1152412
Change-Id: I3146c0ea5706be8b5ec13ee8740e1d185e008075
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2562248
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71432}
This adds a d8 flag --simulate-errors, which on shutdown will cause
certain errors. This enables testing the reliability of sanitizers.
This will cause a fatal error, a dcheck (if available) or a
violation that can be detected with one of the following sanitizers:
ASAN, UBSAN, MSAN, CFI.
The same flag used in differential fuzzing will cause an error
subsumed with the error state "fake_difference".
Bug: chromium:1152412
Change-Id: I4b36c6fe716797004d634263617d22ca67b05600
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2554999
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71430}
Don't pass the correctness-fuzzing suppressions to normal fuzzing as
they turn stack overflows and invalid string length checks into
crashes.
This became first now a problem after the flag was passed in an mjsunit
test case.
No-Try: true
Bug: chromium:1151600,chromium:1151599
Change-Id: I5d29900a4b155762cae447fc102055eab1916309
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2551112
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71333}
For people not too familiar with cloud storage, the direct URL makes it
easier to download the zip archive.
R=machenbach@chromium.org
Bug: chromium:1142437, v8:10891
Change-Id: I6b29ebaac37489a73c4f75473d07e04e7200f2c3
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2543395
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71276}
It took me a while to figure out how to set up everything to run the
fuzzer locally. These improved error messages would have helped to find
errors earlier.
R=machenbach@chromium.org
Bug: chromium:1142437
Change-Id: I414de0a8448cd81fdbf999cc7c3c8cf9394354f6
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2543397
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71239}
This is a JavaScript fuzzer originally authored by Oliver Chang. It
is a mutation based fuzzer using Babel code transformations. For more
information see the included README.md.
The original code was altered:
- Add new V8 copyright headers.
- Make the test expectation generator aware of the headers.
- Fix file endings for presubmit checks.
- Fix `npm test` on fresh checkout with a new fake DB.
- Make test skipping work with new v8/tools location.
- OWNERS file.
- New title section in README.md.
No-Try: true
Bug: chromium:1109770
Change-Id: Ie71752c0a37491a50500c49060a3c526716ef933
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2320330
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69164}
This is another case of the known .caller difference that's now added
to the mapping of known issues.
No-Try: true
Bug: chromium:1101870
Change-Id: I6cfca6887362564f625648ba34820cb92a77efb6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2280087
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68674}
Mostly gc-related flags didn't flush out any bugs yet, but often reduce
test performance and lead to timeouts.
No-Try: true
Bug: chromium:1044942
Change-Id: I2a7b55f78bfa3d597de1a5674658829e0812d01a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2273861
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68608}
Previously we ran baseline (e.g. ignition) and one random secondary
comparison configuration (e.g. turbofan) from the list of experiments.
But Clusterfuzz imposes limitations on the total amount of fuzz tests.
Therefore this change enables more throughput by always running the
default configuration (ignition_turbofan like V8 is shipped)
additionally to the baseline and the secondary configuration.
This, hence, doubles the number of comparisons we run, with less than
50% additional runtime, since the slow baseline configuration is only
run once.
The experiments table is updated accordingly. Explicit entries running
ignition_turbofan are removed (as it always runs now), instead some
of the other configurations are increased in their relative
percentage. We also get a few new configurations that didn't run
before (e.g. forcing the slow path on x86).
No-Try: true
Bug: chromium:1100114
Change-Id: I69b2a41d78c06e556b309743a2aace1053c22f91
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270307
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68607}
This shares the code of running the comparisons on the fuzz test
and on the smoke tests. While this change retains behavior for running
the fuzz test, the smoke test has slight changes in favor of code
readability:
1. Smoke tests can now bail out on output suppressions (There are
none now. And if we'd ever add any we'd need to write them in a way,
such that they ignore the output of the smoke tests, which don't
have much output anyways).
2. Crashes in smoke tests are now a hard failure. This is a desired
feature anyways. It's unlikely that the smoke test crashes as then
nothing would work.
No-Try: true
Bug: chromium:1100114
Change-Id: Ice3c6d18b549727c486a70756a72038c8b2029ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2273125
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68586}
This adds a class representing all values needed for an execution.
It simplifies the parameters passed to formatting functions and
lifts the restriction of a first/second config pair in the options.
This will allow to run more than two configs in a follow up.
No-Try: true
Bug: chromium:1100114
Change-Id: I568253b8ca3220870b8f5af615108140587680a8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270550
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68583}
This makes output and test-case suppressions independent of the used
comparison configs and architecture. Such fine-grained suppressions
were only needed during the inception of differential fuzzing, but
by now, most remaining suppressions are implemented in d8 behind
a flag.
This prepares for running with more than two comparison configs in a
follow up.
No-Try: true
Bug: chromium:1100114
Change-Id: I072769adb3ef7c6e6c43459aa23ac906f461b307
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270095
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68579}
It is obsolete to filter out error-message differences since the
time we pass --correctness-fuzzer-suppressions to d8, which already
stubs all messages:
https://cs.chromium.org/chromium/src/v8/src/execution/messages.cc?l=1031
No-Try: true
Bug: chromium:1100114
Change-Id: Iac42a8e2a32f9bae4034f79eaff429bf3ee41724
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270024
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68577}
This simplifies the lengthy main method by extracting some code and
by replacing the scattered returns with exceptions.
We introduce two exceptions for early bail-out. This enables helper
methods on multiple layers. The early bail-out on time-out is
moved to the point where it is detected.
Previously on timeout and crash we also printed out the step number.
Clusterfuzz doesn't parse this, it was only for statistical purposes,
and the latest version of the experimental workbench only parses
crashes and timeouts, not the step in which they happened. Hence,
this CL removes those step numbers.
Except the change described in the last paragraph, this CL doesn't
intend to change behavior.
No-Try: true
Bug: chromium:1100114
Change-Id: Ie8c18f183e4fc538577f3eb49aaf6df1acd1e4e1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270547
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68576}
If we pass flags that make runs very slow, also the validity checks
might time out. Previously this wasn't checked and output was just
cut off.
This also tightens the timeout on validity checks as they are
expected to run very fast.
No-Try: true
Bug: chromium:1098646
Change-Id: Iea9a932be86e84040b72a2311aaa1d44100b3378
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2262915
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68510}
This fixes a bug when an encoded character appears in the difference
string. Python3 doesn't require any encoding.
TBR=tmrts@chromium.org
No-Try: true
Bug: chromium:1095964
Change-Id: I49c66b5b9c105ad64d3a7839d0eb5df97ff5f404
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2249660
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68384}
This prepares using ochang_js_fuzzer with foozzie. The fuzzer uses
tests from CrashTests in the corpus. This leads to a loop when
used with differential fuzzing, as foozzie dedupes failures based
on the original file path. Foozzie finds a new failure for the
existing failure in CrashTests, for which clusterfuzz creates a new
crash test and so on.
This subsumes all failures from CrashTests under the same key.
Once such a failure is reported, a developer can add it to a
mapping in foozzie.py, after which the global key can be used
again by clusterfuzz to report another failure.
No-Try: true
Bug: chromium:1044942
Change-Id: I801a23faeb0c672d6ad64b4100c463f53e36cbc2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2214837
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68053}
This prevents bug flooding based on differences from calling
%GetOptimizationStatus in correctness tests. It is supposed to
be suppressed with --allow-natives-for-differential-fuzzing.
This ensures early bail-out in case the flag is forgotten at
some point. The v8_sanity_checks.js file is executed before
each correctness test case for this purpose.
NOTRY=true
Bug: chromium:1044942
Change-Id: I74a836a82562604b35e94e5e123a2a8bff939423
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2184294
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67618}
A new realm doesn't contain the mocks and suppressions on the global
object for correctness fuzzing. We replace Realm.eval with eval to
keep exercising the code.
Bug: chromium:1071133
Change-Id: Iffe82d37bf08829fc5937c17c2089277403e71dc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2153206
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67187}
Yet another corner case how non-deterministic timestamps slipped into
the tests.
Bug: chromium:1064900
Change-Id: I33e8b4c8141b3854b7eca5d7ad9b45b6f5130d9e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2120584
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66881}
This adds tests for the mock logic used in differential fuzzing. The
tests uncovered a couple of issues in the mock files that are also
fixed.
This also does some minor code clean up in the mock code.
Bug: chromium:1044942
Change-Id: I5b67f70f8b104bb681548f742ab863395a88360f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2043843
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66304}
This reduces the pointer-compression experiment by 3% as it is in
production since a while.
This also uses turbofan when comparing across architectures as tests
run faster and likely find the same or more relevant bugs.
No-Try: true
Change-Id: I35ec97188a101a7cb43454903f7bf9765e57893c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2059995
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66290}
Pass --no-lazy-feedback-allocation in all second runs depending
on a probability. Also combine with --interrupt-budget=100.
This also allows adding several extra flags behind one probability.
The tests are improved to ensure valid flags and configs.
No-Try: true
Bug: v8:10215
Change-Id: I2766ef5044cd8c7096f6b76f39b60b568f550bde
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2059991
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Mythri Alle <mythria@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66289}
This will allow sharing the configs between Python and JavaScript
based fuzzers.
No-Try: true
Bug: chromium:1044942
Change-Id: Idd47ad31430cfed0629dbb5b9dd5fd2ef8c764f6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2051951
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66243}
We ensure now that fuzzers don't produce the f.arguments pattern and
instead replace it with a random variable.
No-Try: true
Bug: chromium:1044942, chromium:1020573
Change-Id: I899985f1a238f36d10fb22f76d93b7d5f444eab1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2051944
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66238}
Un-ignore some files that only failed in obsolete comparisons.
We don't compare eager anymore and console functions are mocked out
in d8 by now. The exponentiation operator bug doesn't repro anymore.
No-Try: true
Bug: chromium:1044942
Change-Id: I0a572836bc9a4a6aa4736447f638a522ff8e8168
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2050400
Reviewed-by: Georg Neis <neis@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66233}
The assumtion that V8 has no output differences within a single line
before a stack overflow, didn't hold. The prefix of e.g. console.info
can lead to a difference in a recursive call.
This change makes foozzie's output capping before a crash work on the
level of characters instead of lines to fix this.
No-Try: true
Bug: chromium:1050942
Change-Id: I13f747caf4f5848d40c31bd4232811285bab3c17
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2049844
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66217}
The bug appears to be fixed.
No-Try: true
Bug: chromium:693426
Change-Id: Ibd029c906dca87a3fd0c3e00be0b795e87390bd4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2037439
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66134}
This will allow uploading repro test cases to clusterfuzz for
already suppressed known issues. This will allow tracking if those
issues still reproduce and that suppressions don't become stale.
No-Try: true
Bug: chromium:1044942
Change-Id: I997f11293c51836b97d143b0fea992055b39955e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2036083
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66114}
Respective differences were fixed and the suppressions didn't fire
anymore according to the statistics.
No-Try: true
Bug: chromium:664068, chromium:667678
Change-Id: I5e1aa802e0b0288a73d0f236c154289ab45314a0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2035878
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66100}
Crashes in the presence of RangeError happen often during differential
fuzzing. Until now we have ignored such cases completely.
After this change we compare as much output as possible when one or
both runs have crashed, dramatically increasing the coverage.
No-Try: true
Bug: chromium:1048099
Change-Id: I923c10e9064b5dc6cae1e39a254e221d2867e0e7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2030914
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66085}
This adds two flags for whitelisting natives on fuzzers:
--allow-natives-for-fuzzing:
Enables a small subset of runtime functions and replaces others with
undefined.
--allow-natives-for-differential-fuzzing
Restricts the allowed runtime functions even further, excluding the
ones that break differential fuzzing.
Bug: chromium:1044942
Change-Id: I890bd4a6ff7c2e483dd74155290a7ace06f85239
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2020773
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66068}
This is a cleanup to remove unneeded flags after these changes (in
https://crrev.com/c/1988548):
* --future does not imply --wasm-tier-up any more, and
* --wasm-tier-up does not imply --liftoff any more.
Instead, now
* --wasm-tier-up is enabled by default,
* --wasm-tier-up has no effect if --liftoff is not set, and
* --future implies --liftoff.
R=ahaas@chromium.org
Bug: chromium:1040061
Change-Id: I5d04ee1f1d84ddcd0654df0e0a4c6298f80aee9e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1993280
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65666}
Some test cases stress all global functions and hang when calling
readline.
No-Try: true
Bug: chromium:1034285
Change-Id: I0eaf64437c0806cf3df0a5306aecea4367763cad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1967381
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65460}
Some fuzz test cases set Math.min to a different function. Let the fake
typed array constructors remember the original Math.min.
No-Try: true
Bug: chromium:1024815
Change-Id: Ia389439fe6757f2497e22187042c5cdf1d1c649f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1958050
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65383}
This adds a regresson test case for the revert reason of:
https://crrev.com/c/1906378
The test data is tidied up by keeping the different fake d8s in
separate build directories like it would be in production.
A new test simulates an architecture difference and ensures we
pass the architecture mocks in all runs.
No-Try: true
Bug: chromium:1023091
Change-Id: Ic33c426ba8eb9c4b6b0fbb66d43c0859dc2edfcd
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1918248
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65140}
This adds an abstraction for command-line arguments for each of the
two comparison runs done in correctness fuzzing. No functional
changes intended.
No-Try: true
Bug: chromium:1023091
Change-Id: I9421715c4904416b9aaf53848954a5248c79ffd9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906372
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64979}
This is a reland of 1d493d31ce
Original change's description:
> [foozzie] Refactor command abstraction
>
> This moves code for running d8 into its own class. No functional
> changes intended.
>
> No-Try: true
> Bug: chromium:1023091
> Change-Id: I7cbfeebd2911dc758322f89cf93666550f2956d9
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906378
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64928}
Bug: chromium:1023091
Change-Id: I7df6e12084e20510a400ce209827c2bba8325f86
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1914209
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64966}
This moves code for running d8 into its own class. No functional
changes intended.
No-Try: true
Bug: chromium:1023091
Change-Id: I7cbfeebd2911dc758322f89cf93666550f2956d9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906378
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64928}
We used the same random seed for all test cases of a fuzz session
for transitioning from choosing the flags on V8 side.
Since the grace period for stable bisection is over, we now use
the same random number generator throughout the fuzz session which
leads to a wider range of differently chosen flags.
TBR=tmrts@chromium.org
No-Try: true
Bug: chromium:813833
Change-Id: I07b9fe5de378c01344afd486bfd85fcbf0fcd8d2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906377
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64910}
This adds a fake toolchain for pointer compression, used for
correctness fuzzing. The toolchain enables us to have an extra build
with inverse pointer-compression defaults side-by-side.
The extra build is used similarly to existing x64/x86 comparisons,
except that we now compare builds with different compile-time flags.
Change-Id: I75491371262204b86eaa006ca8d04848f49121ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1829275
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64123}
The sanity checks never really worked since the file was missing in the
output dir. Runs bailed out with "Error loading file".
As both configurations bailed out in the same way, the correctness fuzzer
treated it as pass.
NOTRY=true
Bug: chromium:933076
Change-Id: Id3a2f7e8e3fa8c6654e3e08654d2e9e6b7010484
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1778289
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63495}
This also ignores the line number for type errors.
NOTRY=true
Bug: chromium:995075
Change-Id: I9d9a91842383eba60db858c2c2e02df7dda1a9e1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1778287
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63494}
This makes sure the language mode of the module is correctly propagated
through the WebAssembly module, so that exported functions are allocated
with the correct language mode. It extends the existing {ModuleOrigin}
enum to consist of three values now.
R=clemensh@chromium.org
TEST=mjsunit/regress/wasm/regress-985154
BUG=chromium:985154
Change-Id: Id7b566738b1e710cc5001b894022bcd0f2c01bc3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1708484
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62826}
In order to migrate the extra flags into the fuzzer and keep bisection stable,
we need to use the same RNG state for each call to generating fuzz flags.
Throughout one fuzzing session the same random-seed is used
(https://crbug.com/983128) and we'll pass it to the fuzz config in a follow up.
TBR=tmrts@chromium.org
NOTRY=true
Bug: chromium:813833
Change-Id: I3203c86028a5d283238e6ef739f82eccee1302b1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1697254
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62646}
Currently, probabilities for extra flags are calculated in the correctness
fuzzer harness, which makes the RNG fragile when bisecting backwards, when
the script's config changes during bisection.
This adds the possibility to pass extra flags on command line to the
script. After a grace period, we will migrate the flag calculation to
clusterfuzz.
NOTRY=true
Bug: chromium:813833
Change-Id: I515181847474515089b847f8aaffc7c6560d9390
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1675945
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62359}
To correctly fuzz the behaviour of optimized code and ICs we need to
allocate feedback vectors. So for the configurations testing these we
should turn off lazy feedback allocation.
It is also good to fuzz without lazy feedback allocation on other
configurations to flush out any other issues. So we also fuzz this with
0.35 chance. We also fuzz aggressive lazy feedback allocation (allocate
feedback vectors on first branch / return) with 0.05 chance to test
corner cases related to lazy feedback allocation.
Bug: v8:9342
Change-Id: Id0761d1396bfc0866988abb8fb20168b86a5da20
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1672939
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62347}
We have too many dupes in the no-ic comparisons. We'll increase the
experiment size again once bugs are fixed.
TBR=jarin@chromium.org
NOTRY=true
Bug: chromium:961709
Change-Id: Ic946100b45fd73e1bee59f188a766384836bcdcf
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1660624
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62212}
Error messages are unspecified in JavaScript and occasional small
differences in the compared configurations lead to an unjustified
maintenance burden of correctness-fuzzing issues.
This CL replaces most error messages with a fixed suppression
message during correctness fuzzing (behind a flag).
The flag covering all extra behavior for correctness fuzzing is now
renamed to --correctness-fuzzer-suppressions.
Bug: chromium:958668,chromium:946476
Change-Id: Iba1197f765138a962d5bbb176730322e5a411707
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1594730
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61249}
The WebAssembly JavaScript Interface specifies[1] that exported
functions are not constructors, hence do not have the "prototype"
property. This is not true for asm.js exported functions which are
expected to look like normal functions (or constructors).
[1] https://webassembly.github.io/spec/js-api/index.html#exported-function-exotic-objectsR=clemensh@chromium.org
TEST=mjsunit/regress/regress-crbug-935800
BUG=chromium:935800
Change-Id: Idecacfb7f5d4668540589af95fd59872334c21a3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1578499
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60943}
Reading --verify_predictable makes me sad, whereas --verify-predictable
makes me happy. This CL introduces more happiness.
R=machenbach@chromium.org
Bug: v8:8834
Change-Id: Id51a75f32e6d5a2f87aed81e058a8b6dff189758
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1550399
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60599}
This lets foozzie call d8 with sanity output before doing the actual
correctness comparisons. This will make clusterfuzz dedupe cases on
the difference found in the sanity checks.
Also adding missing OWNERS file.
NOTRY=true
Bug: chromium:933076
Change-Id: I4229183726064cc0ad76da8fe432e1dbb601a7ba
Reviewed-on: https://chromium-review.googlesource.com/c/1491221
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59938}
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
This CL was uploaded by git cl split.
Bug: v8:8594
Change-Id: I661c52a70527e8ddde841fee6d4dcba282b4a938
Reviewed-on: https://chromium-review.googlesource.com/c/1470123
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59675}
This replaces the "trusted" with the "jitless" correctness-fuzzing experiment.
The former is a no-op already.
NOTRY=true
Bug: v8:8778
Change-Id: Ie9b490df27071980b2049148844b8f716bbbb1f7
Reviewed-on: https://chromium-review.googlesource.com/c/1450120
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59313}
When using correctness fuzzing, this makes sure all non-object
arguments to typed array constructors are bound by 1MiB when
interpreted as numbers.
NOTRY=true
Bug: chromium:910962
Change-Id: I66e87ece27aae7c5fa88429c5d1f1f478de702ae
Reviewed-on: https://chromium-review.googlesource.com/c/1369959
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58144}
To prevent correctness fuzzers from finding spurious differences
between architectures, we need to mock out the maximum length of
all TypedArrays. This patch adds the two new types BigInt64Array
and BigUint64Array to the existing list.
Bug: chromium:894864
Change-Id: I5cdeeafa597b09aee2d9b4d368c07f10008baf58
Reviewed-on: https://chromium-review.googlesource.com/c/1285399
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56710}
Instead of repeating it in every configuration, just add it to the
common FLAGS.
R=machenbach@chromium.org
Change-Id: I93e7ef0f0ad55bfe0a0e24f50d5a73d4658d7554
Reviewed-on: https://chromium-review.googlesource.com/1141733
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54552}
