Commit Graph

47 Commits

Author SHA1 Message Date
Michael Achenbach
f16da87de2 [foozzie] Drop incompatible flags
This adds the option to list disallowed flags for differential
fuzzing directly in the harness. Flags that can crash in smoke
tests shoule be added there.

No-Try: true
Bug: chromium:1240812
Change-Id: I57c772bedeac0ca6ba023c6b4929515b4b0e6cca
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3103314
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Zhi An Ng <zhin@chromium.org>
Reviewed-by: Zhi An Ng <zhin@chromium.org>
Cr-Commit-Position: refs/heads/main@{#76369}
2021-08-18 16:19:11 +00:00
Ross McIlroy
4ab70f6b21 [Compiler] Remove untrusted code mitigations.
These are no longer enabled, so remove the code mitigation logic from
the codebase.

BUG=chromium:1003890

Change-Id: I536bb1732e8463281c21da446bbba8f47ede8ebe
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3045704
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#76256}
2021-08-12 12:58:24 +00:00
Michael Achenbach
8590942c84 [foozzie] Prioritize reporting differences with x64 if present
tldr: This adds an on-demand comparison with x64 when a difference to
a non-x64 build is detected.

Normally foozzie compares the baseline build (just ignition), with
two secondary builds. One, the default, always uses the shipping
configuration, the second passes additional flags. Both can use a
different architecture than the baseline build as well.

Differences between ignition and turbofan are then often detected
independent of the architectures used, but reported several times
(for each compared architecture).

This makes the reporting more specific, by running another build on
demand that uses the baseline architecture, but otherwise the same
configuration that showed a difference. If it shows the difference as
well, the baseline architecture is used for the report.

As a result only pure architecture differences will be reported with
an architecture other than x64.

This also adds some minor refactorings to reduce the code complexity
when looping over comparisons.

For testing this, the fake-d8s are extended with different behavior
for different flags passed. We add two test cases for testing:
x64 vs. ia32 with difference in x64 and ia32
x64 vs. ia32 with difference only in ia32

Bug: chromium:1196633
No-Try: true
Test: tools/clusterfuzz/v8_foozzie_test.py
Change-Id: Ic470ae8f0b37fb1628b32e4fafc0c39377e16f8c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2897099
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74612}
2021-05-17 17:57:03 +00:00
Michael Achenbach
886d7cfee4 Reland "Enable simulating errors to test fuzzer reliability"
This is a reland of 4ad08c82f7

The reland organizes the different error types in separate functions
for separate call stacks. Error simulation is also guarded by
a minimum file size to prevent Clusterfuzz from getting stuck with
its bad-build check.

Original change's description:
> Enable simulating errors to test fuzzer reliability
>
> This adds a d8 flag --simulate-errors, which on shutdown will cause
> certain errors. This enables testing the reliability of sanitizers.
>
> This will cause a fatal error, a dcheck (if available) or a
> violation that can be detected with one of the following sanitizers:
> ASAN, UBSAN, MSAN, CFI.
>
> The same flag used in differential fuzzing will cause an error
> subsumed with the error state "fake_difference".
>
> Bug: chromium:1152412
> Change-Id: I4b36c6fe716797004d634263617d22ca67b05600
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2554999
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#71430}

Bug: chromium:1152412
Change-Id: I604258b4c1ebd215c26b1de6b2822663f857bf64
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2565125
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71538}
2020-12-01 16:26:43 +00:00
Michael Achenbach
bea9978700 [foozzie] Insensitive terms clean-up
No-Try: true
Bug: v8:10619
Change-Id: I1e227c64fa34caf010271b299d9310d19bdfc53a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2563273
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71454}
2020-11-27 16:57:24 +00:00
Michael Achenbach
4f2f106206 [foozzie] Use separate crash state for simulated crashes
The error simulation works and crashes during the smoke test are
detected. But now this masks real crashes (https://crbug.com/1153200),
therefore this change subsumes smoke-test crashes with
--simulate-errors under a separate crash state.

Now Clusterfuzz will detect the case "unexpected crash" as fixed, but
create a new bug for "simulated crash". Which we will keep open
forever.

No-Try: true
Bug: chromium:1152412
Change-Id: I511af167d33430d9b89692151cb8abaf3a90c715
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2563270
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Auto-Submit: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71452}
2020-11-27 14:58:44 +00:00
Michael Achenbach
67cf72dc25 [foozzie] Cluster more .caller issues
This summarizes more dupes of https://crbug.com/1042556

TBR=mslekova@chromium.org

No-Try: true
Bug: chromium:1108048,chromium:1108322
Change-Id: If572b5eeeac9c2750eb1a630dd76490529e60d4a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2314476
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69011}
2020-07-22 19:20:23 +00:00
Michael Achenbach
42a841f8de [foozzie] Cluster a known failure
This is another case of the known .caller difference that's now added
to the mapping of known issues.

No-Try: true
Bug: chromium:1101870
Change-Id: I6cfca6887362564f625648ba34820cb92a77efb6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2280087
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68674}
2020-07-03 12:54:30 +00:00
Michael Achenbach
cca290945d [foozzie] Compare baseline/default in every run
Previously we ran baseline (e.g. ignition) and one random secondary
comparison configuration (e.g. turbofan) from the list of experiments.
But Clusterfuzz imposes limitations on the total amount of fuzz tests.
Therefore this change enables more throughput by always running the
default configuration (ignition_turbofan like V8 is shipped)
additionally to the baseline and the secondary configuration.

This, hence, doubles the number of comparisons we run, with less than
50% additional runtime, since the slow baseline configuration is only
run once.

The experiments table is updated accordingly. Explicit entries running
ignition_turbofan are removed (as it always runs now), instead some
of the other configurations are increased in their relative
percentage. We also get a few new configurations that didn't run
before (e.g. forcing the slow path on x86).

No-Try: true
Bug: chromium:1100114
Change-Id: I69b2a41d78c06e556b309743a2aace1053c22f91
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270307
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68607}
2020-06-30 11:07:47 +00:00
Michael Achenbach
ca674858e5 [foozzie] Align running main comparisons and smoke tests
This shares the code of running the comparisons on the fuzz test
and on the smoke tests. While this change retains behavior for running
the fuzz test, the smoke test has slight changes in favor of code
readability:

1. Smoke tests can now bail out on output suppressions (There are
none now. And if we'd ever add any we'd need to write them in a way,
such that they ignore the output of the smoke tests, which don't
have much output anyways).
2. Crashes in smoke tests are now a hard failure. This is a desired
feature anyways. It's unlikely that the smoke test crashes as then
nothing would work.

No-Try: true
Bug: chromium:1100114
Change-Id: Ice3c6d18b549727c486a70756a72038c8b2029ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2273125
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68586}
2020-06-29 16:38:36 +00:00
Michael Achenbach
79dafc832d [foozzie] Refactoring - add an abstraction for an execution
This adds a class representing all values needed for an execution.
It simplifies the parameters passed to formatting functions and
lifts the restriction of a first/second config pair in the options.

This will allow to run more than two configs in a follow up.

No-Try: true
Bug: chromium:1100114
Change-Id: I568253b8ca3220870b8f5af615108140587680a8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270550
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68583}
2020-06-29 15:21:02 +00:00
Michael Achenbach
4146efbfe6 [foozzie] Refactoring - simplify suppressions
This makes output and test-case suppressions independent of the used
comparison configs and architecture. Such fine-grained suppressions
were only needed during the inception of differential fuzzing, but
by now, most remaining suppressions are implemented in d8 behind
a flag.

This prepares for running with more than two comparison configs in a
follow up.

No-Try: true
Bug: chromium:1100114
Change-Id: I072769adb3ef7c6e6c43459aa23ac906f461b307
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270095
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68579}
2020-06-29 12:59:20 +00:00
Michael Achenbach
c220a05ca9 [foozzie] Refactoring - several code clean-ups
This simplifies the lengthy main method by extracting some code and
by replacing the scattered returns with exceptions.

We introduce two exceptions for early bail-out. This enables helper
methods on multiple layers. The early bail-out on time-out is
moved to the point where it is detected.

Previously on timeout and crash we also printed out the step number.
Clusterfuzz doesn't parse this, it was only for statistical purposes,
and the latest version of the experimental workbench only parses
crashes and timeouts, not the step in which they happened. Hence,
this CL removes those step numbers.

Except the change described in the last paragraph, this CL doesn't
intend to change behavior.

No-Try: true
Bug: chromium:1100114
Change-Id: Ie8c18f183e4fc538577f3eb49aaf6df1acd1e4e1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270547
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68576}
2020-06-29 12:29:39 +00:00
Michael Achenbach
dd58472158 [foozzie] Bail out on timeouts during validity checks
If we pass flags that make runs very slow, also the validity checks
might time out. Previously this wasn't checked and output was just
cut off.

This also tightens the timeout on validity checks as they are
expected to run very fast.

No-Try: true
Bug: chromium:1098646
Change-Id: Iea9a932be86e84040b72a2311aaa1d44100b3378
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2262915
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68510}
2020-06-24 13:25:53 +00:00
Ng Zhi An
ad913fe4f3 [Respect] Prefer inclusive terms
This changes the use of "sane" to "sensible" or "valid". I tried to be
sensible in my choice of replacement, by trying to read the comments or
code to see which word matches the intention closest.

Referenced
https://fuchsia.dev/fuchsia-src/contribute/best-practices/respectful_code?hl=en#what_are_examples_of_terminology_to_be_avoided.

Bug: v8:10619
Change-Id: Id957b2e6ff11e95270e1372005e1006d8cf1008d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2254483
Commit-Queue: Zhi An Ng <zhin@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68471}
2020-06-22 18:11:23 +00:00
Michael Achenbach
d1a02d23ec [foozzie] Fix Python2 encoding
This fixes a bug when an encoded character appears in the difference
string. Python3 doesn't require any encoding.

TBR=tmrts@chromium.org

No-Try: true
Bug: chromium:1095964
Change-Id: I49c66b5b9c105ad64d3a7839d0eb5df97ff5f404
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2249660
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68384}
2020-06-17 08:58:18 +00:00
Michael Achenbach
9036662f6d [foozzie] Defeat the CrashTests loop
This prepares using ochang_js_fuzzer with foozzie. The fuzzer uses
tests from CrashTests in the corpus. This leads to a loop when
used with differential fuzzing, as foozzie dedupes failures based
on the original file path. Foozzie finds a new failure for the
existing failure in CrashTests, for which clusterfuzz creates a new
crash test and so on.

This subsumes all failures from CrashTests under the same key.
Once such a failure is reported, a developer can add it to a
mapping in foozzie.py, after which the global key can be used
again by clusterfuzz to report another failure.

No-Try: true
Bug: chromium:1044942
Change-Id: I801a23faeb0c672d6ad64b4100c463f53e36cbc2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2214837
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68053}
2020-05-28 17:52:57 +00:00
Michael Achenbach
540484445f [foozzie] Fix more Python3 incompatibilities
NOTRY=true
TBR=tmrts@chromium.org

Bug: chromium:1065624
Change-Id: I6e49c48bb95e10b7fad1ff2c589a2dd459fff562
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2124326
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66900}
2020-03-28 12:53:10 +00:00
Michael Achenbach
df0dd74be0 [foozzie] Overhaul --no-lazy-feedback-allocation comparisons
Pass --no-lazy-feedback-allocation in all second runs depending
on a probability. Also combine with --interrupt-budget=100.

This also allows adding several extra flags behind one probability.
The tests are improved to ensure valid flags and configs.

No-Try: true
Bug: v8:10215
Change-Id: I2766ef5044cd8c7096f6b76f39b60b568f550bde
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2059991
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Mythri Alle <mythria@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66289}
2020-02-17 14:10:37 +00:00
Michael Achenbach
28abde86ca [foozzie] Add option to skip suppressions
This will allow uploading repro test cases to clusterfuzz for
already suppressed known issues. This will allow tracking if those
issues still reproduce and that suppressions don't become stale.

No-Try: true
Bug: chromium:1044942
Change-Id: I997f11293c51836b97d143b0fea992055b39955e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2036083
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66114}
2020-02-04 15:33:37 +00:00
Michael Achenbach
3fd58c664b [foozzie] Compare output before crashes
Crashes in the presence of RangeError happen often during differential
fuzzing. Until now we have ignored such cases completely.

After this change we compare as much output as possible when one or
both runs have crashed, dramatically increasing the coverage.

No-Try: true
Bug: chromium:1048099
Change-Id: I923c10e9064b5dc6cae1e39a254e221d2867e0e7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2030914
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66085}
2020-02-03 16:51:39 +00:00
Clemens Backes
816ea12124 [wasm] Adjust flags after changed implications
This is a cleanup to remove unneeded flags after these changes (in
https://crrev.com/c/1988548):
* --future does not imply --wasm-tier-up any more, and
* --wasm-tier-up does not imply --liftoff any more.

Instead, now
* --wasm-tier-up is enabled by default,
* --wasm-tier-up has no effect if --liftoff is not set, and
* --future implies --liftoff.

R=ahaas@chromium.org

Bug: chromium:1040061
Change-Id: I5d04ee1f1d84ddcd0654df0e0a4c6298f80aee9e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1993280
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#65666}
2020-01-09 16:55:42 +00:00
Michael Achenbach
dce12b6349 [foozzie] Refactor argument abstraction
This adds an abstraction for command-line arguments for each of the
two comparison runs done in correctness fuzzing. No functional
changes intended.

No-Try: true
Bug: chromium:1023091
Change-Id: I9421715c4904416b9aaf53848954a5248c79ffd9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906372
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64979}
2019-11-15 09:48:18 +00:00
Michael Achenbach
2475c91bb3 Reland "[foozzie] Refactor command abstraction"
This is a reland of 1d493d31ce

Original change's description:
> [foozzie] Refactor command abstraction
> 
> This moves code for running d8 into its own class. No functional
> changes intended.
> 
> No-Try: true
> Bug: chromium:1023091
> Change-Id: I7cbfeebd2911dc758322f89cf93666550f2956d9
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906378
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64928}

Bug: chromium:1023091
Change-Id: I7df6e12084e20510a400ce209827c2bba8325f86
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1914209
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64966}
2019-11-14 18:51:57 +00:00
Michael Achenbach
53201194d2 Revert "[foozzie] Refactor command abstraction"
This reverts commit 1d493d31ce.

Reason for revert: Fails on clusterfuzz:
https://crbug.com/1024241

Original change's description:
> [foozzie] Refactor command abstraction
> 
> This moves code for running d8 into its own class. No functional
> changes intended.
> 
> No-Try: true
> Bug: chromium:1023091
> Change-Id: I7cbfeebd2911dc758322f89cf93666550f2956d9
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906378
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#64928}

TBR=machenbach@chromium.org,tmrts@chromium.org,almuthanna@google.com,liviurau@chromium.org

Change-Id: I22aede0d2c8bd3cde48c45f4663b43caa4d60b65
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1023091,chromium:1024241
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1914208
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64933}
2019-11-13 14:31:05 +00:00
Michael Achenbach
1d493d31ce [foozzie] Refactor command abstraction
This moves code for running d8 into its own class. No functional
changes intended.

No-Try: true
Bug: chromium:1023091
Change-Id: I7cbfeebd2911dc758322f89cf93666550f2956d9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1906378
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64928}
2019-11-13 08:29:27 +00:00
Michael Achenbach
e6c1385129 [foozzie] Correctness-compare pointer compression build
This adds a fake toolchain for pointer compression, used for
correctness fuzzing. The toolchain enables us to have an extra build
with inverse pointer-compression defaults side-by-side.

The extra build is used similarly to existing x64/x86 comparisons,
except that we now compare builds with different compile-time flags.

Change-Id: I75491371262204b86eaa006ca8d04848f49121ac
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1829275
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64123}
2019-10-07 07:58:42 +00:00
Michael Achenbach
6039d7baa2 [foozzie] Enable wasm staging on correctness fuzzer
NOTRY=true

Bug: v8:9601
Change-Id: I0406326dab681843ef8accadefa10a9e85593fb1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1781055
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63507}
2019-09-02 13:41:34 +00:00
Michael Achenbach
ccd3747222 [foozzie] Migrate extra-flags generation to clusterfuzz side
NOTRY=true

Bug: chromium:813833,chromium:983128
Change-Id: I449796b761f53bb15a3563604d5a4a9018035cb6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1697255
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62662}
2019-07-12 07:40:58 +00:00
Michael Achenbach
1ca89b8c71 [foozzie] Enable passing extra flags on command line
Currently, probabilities for extra flags are calculated in the correctness
fuzzer harness, which makes the RNG fragile when bisecting backwards, when
the script's config changes during bisection.

This adds the possibility to pass extra flags on command line to the
script. After a grace period, we will migrate the flag calculation to
clusterfuzz.

NOTRY=true

Bug: chromium:813833
Change-Id: I515181847474515089b847f8aaffc7c6560d9390
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1675945
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62359}
2019-06-25 13:27:11 +00:00
Mythri A
18fcd0b4ee [clusterfuzz] Teach clusterfuzz to use --no-lazy-feedback-allocation
To correctly fuzz the behaviour of optimized code and ICs we need to
allocate feedback vectors. So for the configurations testing these we
should turn off lazy feedback allocation.

It is also good to fuzz without lazy feedback allocation on other
configurations to flush out any other issues. So we also fuzz this with
0.35 chance. We also fuzz aggressive lazy feedback allocation (allocate
feedback vectors on first branch / return) with 0.05 chance to test
corner cases related to lazy feedback allocation.

Bug: v8:9342
Change-Id: Id0761d1396bfc0866988abb8fb20168b86a5da20
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1672939
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62347}
2019-06-25 09:20:19 +00:00
Michael Achenbach
45bfa681f9 [foozzie] Add interrupt-budget trial
NOTRY=true

Bug: v8:9276
Change-Id: Ifd0ec8d86d880ce29101260833528e1aa230de01
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1624205
Reviewed-by: Jaroslav Sevcik <jarin@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61728}
2019-05-22 09:49:40 +00:00
Michael Achenbach
6207d75e91 [foozzie] Add no-ic configurations
NOTRY=true

Bug: v8:9277
Change-Id: If385439e2bdd8146fe3ba5734920b2096b6c1789
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1622853
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61724}
2019-05-22 08:28:53 +00:00
Michael Achenbach
b3b6b5c655 [foozzie] Ignore error messages in correctness fuzzing
Error messages are unspecified in JavaScript and occasional small
differences in the compared configurations lead to an unjustified
maintenance burden of correctness-fuzzing issues.

This CL replaces most error messages with a fixed suppression
message during correctness fuzzing (behind a flag).

The flag covering all extra behavior for correctness fuzzing is now
renamed to --correctness-fuzzer-suppressions.

Bug: chromium:958668,chromium:946476
Change-Id: Iba1197f765138a962d5bbb176730322e5a411707
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1594730
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61249}
2019-05-06 16:11:37 +00:00
Clemens Hammacher
ac8c78e01d [cleanup] Replace underscore by minus in d8 flags
Reading --verify_predictable makes me sad, whereas --verify-predictable
makes me happy. This CL introduces more happiness.

R=machenbach@chromium.org

Bug: v8:8834
Change-Id: Id51a75f32e6d5a2f87aed81e058a8b6dff189758
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1550399
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60599}
2019-04-03 12:32:08 +00:00
Clemens Hammacher
0a703c5b93 [foozzie] Disable async compilation of WebAssembly
Async compilation introduces nondeterminism, thus disable it for
foozzie.

R=ahaas@chromium.org, machenbach@chromium.org

Bug: v8:8949, chromium:942941
Change-Id: Ieeb92c530de299cdd34f0d9cc3a1293b2dc5e139
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1526194
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60287}
2019-03-18 12:41:08 +00:00
Michael Achenbach
47608ce626 [foozzie] Add sanity checks to avoid bug flooding
This lets foozzie call d8 with sanity output before doing the actual
correctness comparisons. This will make clusterfuzz dedupe cases on
the difference found in the sanity checks.

Also adding missing OWNERS file.

NOTRY=true

Bug: chromium:933076
Change-Id: I4229183726064cc0ad76da8fe432e1dbb601a7ba
Reviewed-on: https://chromium-review.googlesource.com/c/1491221
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59938}
2019-02-28 11:27:32 +00:00
Vadim Gorbachev (bmsdave)
7315d7b3d7 Preparing v8 to use with python3 /tools
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds

This CL was uploaded by git cl split.

Bug: v8:8594
Change-Id: I661c52a70527e8ddde841fee6d4dcba282b4a938
Reviewed-on: https://chromium-review.googlesource.com/c/1470123
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59675}
2019-02-19 09:12:07 +00:00
Michael Achenbach
8981735e8b [foozzie] Add more stress experiments
NOTRY=true

Bug: v8:8278
Change-Id: I0f6c5ca3766ce6a9ffa2a97082b875b2a3fb357f
Reviewed-on: https://chromium-review.googlesource.com/c/1450122
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59326}
2019-02-04 10:21:13 +00:00
Michael Achenbach
026ef7237d [foozzie] Compare running jitless
This replaces the "trusted" with the "jitless" correctness-fuzzing experiment.
The former is a no-op already.

NOTRY=true

Bug: v8:8778
Change-Id: Ie9b490df27071980b2049148844b8f716bbbb1f7
Reviewed-on: https://chromium-review.googlesource.com/c/1450120
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59313}
2019-02-04 07:54:08 +00:00
Clemens Hammacher
a6a727357e [foozzie] Move --suppress-asm-messages flag to FLAGS
Instead of repeating it in every configuration, just add it to the
common FLAGS.

R=machenbach@chromium.org

Change-Id: I93e7ef0f0ad55bfe0a0e24f50d5a73d4658d7554
Reviewed-on: https://chromium-review.googlesource.com/1141733
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54552}
2018-07-19 10:39:37 +00:00
Clemens Hammacher
605f94b700 [foozzie] Update existing configs for liftoff
Instead of having a separate liftoff config, which is tested against
the default (which currently means tier-up from liftoff to turbofan),
just choose reasonable liftoff configs for the existing configs.
'ignition' now implies pure liftoff execution.
'ignition_turbo_opt' always compiles with turbofan.
Other configs use the default (tier up).

R=machenbach@chromium.org

Bug: chromium:824098, v8:6600
Change-Id: I92c008fc1b1fa54d3161fb5695a095127d6ac263
Reviewed-on: https://chromium-review.googlesource.com/1141731
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54548}
2018-07-19 09:51:06 +00:00
Clemens Hammacher
76ac6372fa [wasm] Prepare enabling liftoff by default on ia32 and x64
Before flipping the flag, some tests need to be adapted. This CL
prepares these tests, such that the flag flip CL really just flips a
flag.

R=titzer@chromium.org, hablich@chromium.org

Bug: v8:6600, chromium:787421
Change-Id: I8030df69cda5f3fb81354350a37f65c0d1c669bd
Reviewed-on: https://chromium-review.googlesource.com/1110363
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Michael Hablich <hablich@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53959}
2018-06-22 11:06:26 +00:00
Michael Achenbach
ba5b4a3235 [foozzie] Make correctness runs deterministic
NOTRY=true

Bug: chromium:829681
Change-Id: Iccf9b190e77b1c27fedec74d67f5948803dd1fa9
Reviewed-on: https://chromium-review.googlesource.com/998355
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52437}
2018-04-06 12:16:36 +00:00
Michael Achenbach
adf7ee1791 [foozzie] Add trial for --noanalyze-environment-liveness
NOTRY=true

Bug: chromium:827067
Change-Id: I05073fd85b8b1b1545d5c2be55676aed237d1c10
Reviewed-on: https://chromium-review.googlesource.com/997292
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52376}
2018-04-05 07:22:34 +00:00
Clemens Hammacher
49f23ce642 [foozzie] Add Liftoff testing
This adds 5% testing of 'ignition' vs 'liftoff', which tests Turbofan vs
Liftoff for wasm code, and tests Ignition vs Turbofan for javascript
code.
It also adds 3% testing of 'liftoff' (x64) vs 'liftoff' (ia32), which
does standard x64 vs ia32 testing for javascript code.

R=machenbach@chromium.org

Bug: chromium:824098, v8:6600
Change-Id: I6a6afae0300efc33f3535541a11695a7bb32dcc5
Reviewed-on: https://chromium-review.googlesource.com/973161
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52111}
2018-03-21 12:56:18 +00:00
Michael Achenbach
22fb961b70 [foozzie] Rename folder to account for new clusterfuzz configs
We'll soon also host other configurations for general fuzzing, not only
correctness fuzzing in the new tools/clusterfuzz folder.

TBR=yangguo@chromium.org

Bug: chromium:813833
Change-Id: Icd966bfec91cc547522bad5d1a842500b554754f
Reviewed-on: https://chromium-review.googlesource.com/930331
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51480}
2018-02-22 17:42:39 +00:00