Testing on ClusterFuzz has revealed that the fuzzer sometimes goes
wrong when a shader is very simple - e.g., there have been bugs where
a fuzzer pass has assumed that at least one basic type exists in the
module. This change adds an almost empty SPIR-V example to the shaders
used for testing, to help catch such cases locally.
spirv-fuzz features transformations that should be applicable by
construction. Assertions are used to detect when such transformations
turn out to be inapplicable. Failures of such assertions indicate bugs
in the fuzzer. However, when using the fuzzer at scale (e.g. in
ClusterFuzz) reports of these assertion failures create noise, and
cause the fuzzer to exit early. This change adds an option whereby
inapplicable transformations can be ignored. This reduces noise and
allows fuzzing to continue even when a transformation that should be
applicable but is not has been erroneously created.
This fixes a problem where TransformationInlineFunction could lead to
distinct instructions having identical unique ids. It adds a validity
check to detect this problem in general.
Fixes#3911.
Adds a virtual method, GetFreshIds(), to Transformation. Every
transformation uses this to indicate which ids in its protobuf message
are fresh ids. This means that when replaying a sequence of
transformations the replayer can obtain a smallest id that is not in
use by the module already and that will not be used by any
transformation by necessity. Ids greater than or equal to this id
can be used as overflow ids.
Fixes#3851.
Before this change, the replayer would return a SPIR-V binary. This
did not allow further transforming the resulting module: it would need
to be re-parsed, and the transformation context arising from the
replayed transformations was not available. This change makes it so
that after replay an IR context and transformation context are
returned instead; the IR context can subsequently be turned into a
binary if desired.
This change paves the way for an upcoming PR to integrate spirv-reduce
with the spirv-fuzz shrinker.
In preparation for some upcoming work on the shrinker, this PR changes
the interfaces of Fuzzer, Replayer and Shrinker so that all data
relevant to each class is provided on construction, meaning that the
"Run" method can become a zero-argument method that returns a status,
transformed binary and sequence of applied transformations via a
struct.
This makes greater use of fields, so that -- especially in Fuzzer --
there is a lot less parameter passing.
This change introduces various strategies for controlling the manner
in which fuzzer passes are applied repeatedly, including infrastructure
to allow fuzzer passes to be recommended based on which passes ran
previously.
This change adds the notion of "overflow ids", which can be used
during shrinking to facilitate applying transformations that would
otherwise have become inapplicable due to earlier transformations
being removed.
The SPIR-V data rules say that all uses of an OpSampledImage
instruction must be in the same block as the instruction, and highly
restrict those instructions that can consume the result id of an
OpSampledImage.
This adapts the transformations that split blocks and create synonyms
to avoid separating an OpSampledImage use from its definition, and to
avoid synonym-creation instructions such as OpCopyObject consuming an
OpSampledImage result id.
This change adds a fuzzer pass that allows code from other SPIR-V
modules to be donated into the module under transformation. It also
changes the command-line options of the tools so that, in fuzzing
mode, a file must be specified that contains the names of available
donor modules.
Adds an option to run the validator on the SPIR-V binary after each
fuzzer pass has been applied, to help identify when the fuzzer has
made the module invalid. Also adds a helper method to allow dumping
of the sequence of transformations that have been applied to a JSON
file.
A new pass that allows the fuzzer to change the 'loop control' operand
(and associated literal operands) of OpLoopMerge instructions.
Fixes#2938.
Fixes#2943.
Adds a fuzzer pass and transformation to create a composite (array,
matrix, struct or vector) from available constituent components, and
inform the fact manager that each component of the new composite is
synonymous with the id that was used to construct it. This allows the
"replace id with synonym" pass to then replace uses of said ids with
uses of elements extracted from the composite.
Fixes#2858.
This change to spirv-fuzz uses ideas from "Swarm Testing" (Groce et al. 2012), so that a random subset of fuzzer passes are enabled. These passes are then applied repeatedly in a randomized fashion, with the aggression with which they are applied being randomly chosen per pass.
There is plenty of scope for refining the probabilities introduce in this change; this is just meant to be a reasonable first effort.
To aid in debugging issues in spirv-fuzz, this change adds an option whereby the SPIR-V module is validated after each transformation is applied during replay. This can assist in finding a transformation that erroneously makes the module invalid, so that said transformation can be debugged.
The recently added fuzzer_replayer and fuzzer_shrinker tests were
rather heavyweight and were leading to CI timeouts. This change
reduces the runtime of those tests by having them do fewer iterations.
Similar to the existing 'add dead breaks' pass, this adds a pass to
add dead continues to blocks in loops where such a transformation is
viable. Various functionality common to this new pass and 'add dead
breaks' has been factored into 'fuzzer_util', and some small
improvements to 'add dead breaks' that were identified while reviewing
that code again have been applied.
Fixes#2719.
Adds to spirv-fuzz the option to shrink a sequence of transformations
that lead to an interesting binary to be generated, to find a smaller
sub-sequence of transformations that still lead to an interesting (but
hopefully simpler) binary being generated. The notion of what counts
as "interesting" comes from a user-provided script, the
"interestingness function", similar to the way the spirv-reduce tool
works. The shrinking process will give up after a maximum number of
steps, which can be configured on the command line.
Tests for the combination of fuzzing and shrinking are included, using
a variety of interestingness functions.
* Represent uniform facts via descriptor set and binding.
Previously uniform facts were expressed with resepect to the id of a
uniform variable. Describing them with respect to a descriptor set
and binding is more convenient from the point of view of expressing
facts about a shader without requiring analysis of its SPIR-V.
* Fix equality testing for uniform buffer element descriptors.
The equality test now checks that the lengths of the index vectors
match. Added a test that exposes the previous omission.
Adds a new transformation that can replace a constant with a uniform known to have the same value, and adds a fuzzer pass that (a) replaces a boolean with a comparison of literals (e.g. replacing "true" with "42 > 24"), and then (b) obfuscates the literals appearing in this comparison by replacing them with identically-valued uniforms, if available.
The fuzzer_replayer test file has also been updated to allow initial facts to be provided, and to do error checking of the status results returned by the fuzzer and replayer components.
The replayer takes an existing sequence of transformations and applies
them to a module. Replaying a sequence of transformations that were
obtained via fuzzing should lead to an identical module to the module
that was fuzzed. Tests have been added to check for this.
