Before this change, the replayer would return a SPIR-V binary. This
did not allow further transforming the resulting module: it would need
to be re-parsed, and the transformation context arising from the
replayed transformations was not available. This change makes it so
that after replay an IR context and transformation context are
returned instead; the IR context can subsequently be turned into a
binary if desired.
This change paves the way for an upcoming PR to integrate spirv-reduce
with the spirv-fuzz shrinker.
TransformationContext now holds a std::unique_ptr to a FactManager,
rather than a plain pointer. This makes it easier for clients of
TransformationContext to work with heap-allocated instances of
TransformationContext, which is needed in some upcoming work.
This transformation, given a constant integer (scalar or vector) C,
constants I and S of compatible type and scalar 32-bit integer constant
N, such that C = I - S*N, adds a loop which subtracts S from I, N
times, creating a synonym for C.
The related fuzzer pass randomly chooses constants to which to add
synonyms using this transformation, and the location where they should
be added.
Fixes#3616.
In preparation for some upcoming work on the shrinker, this PR changes
the interfaces of Fuzzer, Replayer and Shrinker so that all data
relevant to each class is provided on construction, meaning that the
"Run" method can become a zero-argument method that returns a status,
transformed binary and sequence of applied transformations via a
struct.
This makes greater use of fields, so that -- especially in Fuzzer --
there is a lot less parameter passing.
This change introduces various strategies for controlling the manner
in which fuzzer passes are applied repeatedly, including infrastructure
to allow fuzzer passes to be recommended based on which passes ran
previously.
This PR modifies the FactManager methods IdIsIrrelevant and GetIrrelevantIds so
that an id is always considered irrelevant if it comes from a dead block.
Fixes#3733.
Introduces two changes:
- duplicated_exit_region refers to a correct block, regardless of the order
of the blocks in the enclosing function.
- Exclude the case where the continue target is the exit block.
This PR implements part of the add bit instruction synonym transformation.
For now, the implementation covers the OpBitwiseOr, OpBitwiseXor and
OpBitwiseAnd cases.
This PR changes the fact manager so that, when calling some of the
functions in submanagers, passes references to other submanagers if
necessary (e.g. to make consistency checks).
In particular:
- DataSynonymAndIdEquationFacts is passed to the AddFactIdIsIrrelevant
function of IrrelevantValueFacts
- IrrelevantValueFacts is passed to the AddFact functions of
DataSynonymAndIdEquationFacts
The IRContext is also passed when necessary and the calls to the
corresponding functions in FactManager were updated to be valid and
always use an updated context.
Fixes#3550.
This transformation, given the header of a selection construct with
branching instruction OpBranchConditional, flattens it.
Side-effecting operations such as OpLoad, OpStore and OpFunctionCall
are enclosed within smaller conditionals.
It is applicable if the construct does not contain inner selection
constructs or loops, or atomic or barrier instructions.
The corresponding fuzzer pass looks for selection headers and
tries to flatten them.
Needed for the issue #3544, but it does not fix it completely.
Adds a transformation that inserts a conditional statement with a
boolean expression of arbitrary value and duplicates a given
single-entry, single-exit region, so that it is present in each
conditional branch and will be executed regardless of which branch will
be taken.
Fixes#3614.
This transformation takes an OpSelect instruction and replaces it with
a conditional branch, selecting the correct value using an OpPhi
instruction.
Fixes part of the issue #3544.
Pointer (if VariablePointers is enabled) to find sets of potential
synonyms.
However, some instructions with these types cannot be used in an OpPhi:
- OpFunction cannot be used as a value
- OpUndef should not be used, because it yields an undefined value for
each use
Fixes#3761.
This transformation takes the id of an OpPhi instruction, of a dead
predecessor of the block containing it and a replacement id of
available to use and of the same type as the OpPhi, and changes
the id in the OpPhi corresponding to the given predecessor.
For example, %id = OpPhi %type %v1 %p1 %v2 %p2
becomes %id = OpPhi %type %v3 %p1 %v2 %p2
if the transformation is given %id, %p1 and %v3, %p1 is a dead block,
%v3 is type type and it is available to use at the end of %p1.
The fuzzer pass randomly decides to apply the transformation to OpPhi
instructions for which at least one of the predecessors is dead
Fixes#3726.
A transformation that replaces the use of an irrelevant id with
another id of the same type.
The related fuzzer pass, for every use of an irrelevant id,
checks whether the id can be replaced in that use by another
id of the same type and randomly decides whether to replace
it.
Fixes#3503.
This PR extends CallGraph with functions to return:
- a list of functions in lexicographical order, with respect to
function calls
- the maximum loop nesting depth that a function can be called from
(computed interprocedurally, e.g. if foo() calls bar() at depth 2
and bar() calls baz() at depth 1, the maximum depth of baz() will
be 3).
A transformation that adds new OpPhi instructions to blocks with >=1
predecessors, so that its value depends on previously-defined ids of
the right type, which are all synonymous. This instruction is also
recorded as synonymous to the others.
The related fuzzer pass still needs to be implemented.
Fixes#3592 .
This change adds the notion of "overflow ids", which can be used
during shrinking to facilitate applying transformations that would
otherwise have become inapplicable due to earlier transformations
being removed.
Adds FuzzerPassAddCompositeInserts, which randomly adds new
OpCompositeInsert instructions. Each OpCompositeInsert instruction
yields a copy of an original composite with one subcomponent replaced
with an existing or newly added object. Synonym facts are added for the
unchanged components in the original and added composite, and for the
replaced subcomponent and the object, if possible.
Fixes#2859
For FuzzerPassAddParameters, adds pointer types (that have the storage
class Function or Private) to the pool of available types for new
parameters. If there are no variables of the chosen pointer type, it
invokes TransformationAddLocalVariable / TransformationAddGlobalVariable
to add one.
Part of #3403
`TransformationAddDeadBlock` did not check whether the existing block
(that will become a selection header) dominates its successor block (that
will become its merge block).
This change adds the check.
Fixes#3690.
Improves the code coverage of tests for the following transformations:
1. TransformationAddRelaxedDecoration
2. TransformationReplaceCopyMemoryWithLoadStore
3. TransformationReplaceCopyObjectWithStoreLoad
4. TransformationReplaceLoadStoreWithCopyMemory
5. TransformationReplaceAddSubMulWithCarryingExtended
Support identical predecessors in TransformationPropagateInstructionUp.
A basic block may have multiple identical predecessors as follows:
%1 = OpLabel
OpSelectionMerge %2 None
OpBranchConditional %true %2 %2
%2 = OpLabel
...
This case wasn't supported before.
`TransformationAddTypeFloat` and `TransformationAddTypeInt` did not check whether the required capabilities were present when adding 16-bit, 64-bit, and 8-bit types.
This change adds these checks in the `IsApplicable` method of each transformation.
Fixes#3669.
`TransformationReplaceIdWithSynonym` is careful to avoid replacing id uses that index into a struct with synonyms because the indices must only be `OpConstant` instructions. However, the check only considered `OpAccessChain` instructions, even though the same restriction applies to `OpInBoundsAccessChain`, `OpPtrAccessChain`, etc.
This change extends the check to include all access chain instructions.
Fixes#3671.
Given an instruction (that may use an OpPhi result from the same block as an input operand), try to clone the instruction into each predecessor block, replacing the input operand with the corresponding OpPhi input operand in each case, if necessary.
Fixes#3458.
Replaces OpIAdd with OpIAddCarry, OpISub with OpISubBorrow, OpIMul with
OpUMulExtended or OpSMulExtended and stores the result into a fresh_id
representing a structure. Extracts the first element of the result into
the original result_id. This value is the same as the result of the
original instruction.
Fixes#3577
This PR changes FuzzerPassOutlineFunctions so that it uses some
transformation that make the TransformationOutlineFunction
transformation applicable in more cases. See the discussion in
#3095 for more details.
Fixes#3095.
This PR introduces TransformationAddLoopPreheader, which, given
a loop header and enough fresh ids, adds a loop preheader, updating
all the references so that this new block is the only out-of-loop
predecessor of the header, which branches unconditionally to the
header.
See the discussion in #3095.
Adds a transformation that takes a pair of instruction descriptors to
OpLoad and OpStore that have the same intermediate value and replaces
the OpStore with an equivalent OpCopyMemory.
Fixes#3353.
Right now, TransformationRecordSynonymousConstants requires the type
ids of two candidate constants to be exactly the same.
This PR adds an exception for integer constants, which can be
considered equivalent even if their signedness is different.
This applies to both integers and vector constants.
The IsApplicable method of ReplaceIdWithSynonym is also updated so
that, in the case of two integer constants which don't have the same
type, they can only be swapped in particular instructions (those
that don't take the signedness into consideration).
Fixes#3536.
This PR generalises TransformationAddAccessChain so that dynamic
indices for non-struct composites (with clamping to ensure that
accesses are in-bound) are allowed.
The transformation will add instructions to clamp any index to
a non-struct composite, regardless of whether it is a constant
or not.
Fixes#3179.
Fixes an issue with the shrinker, where the message consumer set for
the shrinker was not being passed on to the replay object that the
shrinker creates. This meant that messages generated during replay
would cause an exception to be thrown.
Adds a transformation that replaces instruction OpCopyMemory with
loading the source variable to an intermediate value and storing this
value into the target variable of the original OpCopyMemory instruction.
Fixes#3352
Adds a transformation that replaces instruction OpCopyObject with
storing into a new variable and immediately loading this variable to
|result_id| of the original OpCopyObject instruction.
Fixes#3351.
Implemented AreEquivalentConstants method to check equivalency of
constants, changing IsApplicable method of
TransformationRecordSynonymousConstants to allow recording equivalence
of composite constants; added some tests to check this.
Tests with arrays and matrices still need to be added.
Fixes#3533.
Add TransformationAddRelaxedDecoration, which adds the RelaxedPrecision decoration to ids of numeric instructions (those yielding 32-bit ints or floats) in dead blocks.
Fixes#3502
In this PR, the classes that represent the adjust branch weights
transformation and fuzzer pass were implemented. This transformation
adjusts the branch weights of a OpBranchConditional instruction.
The outliner would outline regions ending with a loop header, making
the block containing the call to the outlined function serve as the
loop header. This, however, is incorrect in general, since the whole
outlined function -- rather than just the exit block for the region --
would end up getting called every time the loop would iterate.
This change restricts the outliner so that the last block in a region
cannot be a loop header.
It has been resolved that statically out-of-bounds accesses are not
invalid in SPIR-V (they lead to undefind behaviour at runtime but
should not cause a module to be rejected during validation). This
change tolerates such accesses in donated code, clamping them in-bound
as part of making a function live-safe.
The Sample argument of OpImageTexelPointer is sometimes required to be
a zero constant. It thus cannot be replaced with a synonym in
general. This change avoids replacing this argument with a synonym.
The fact manager maintains an equivalence relation on data descriptors
that tracks when one data descriptor could be used in place of
another. An algorithm to compute the closure of such facts allows
deducing new synonym facts from existing facts. E.g., for two 2D
vectors u and v it is known that u.x is synonymous with v.x and u.y is
synonymous with v.y, it can be deduced that u and v are synonymous.
The closure computation algorithm is very expensive if we get large
equivalence relations.
This change addresses this in three ways:
- The size of equivalence relations is reduced by limiting the extent
to which the components of a composite are recursively noted as
being equivalent, so that when we have large synonymous arrays we do
not record all array elements as being pairwise equivalent.
- When computing the closure of facts, equivalence classes above a
certain size are simply skipped (which can lead to missed facts)
- The closure computation is performed less frequently - it is invoked
explicitly before fuzzer passes that will benefit from data synonym
facts. A new transformation is used to control its invocation, so
that fuzzing and replaying do not get out of sync.
The change also tidies up the order in which some getters are declared
in FuzzerContext.
Adds an extra condition on when a region can be outlined to avoid the
case where a region ends with a loop head but such that the loop's
continue target is in the region. (Outlining such a region would mean
that the loop merge is in the original function and the continue target
in the outlined function.)
The function outliner uses a struct to return ids that a region
generates and that are used outside that region. If these ids have
pointer type this would result in a struct with pointer members, which
leads to illegal loading from non-logical pointers if logical
addressing is used. This change bans that outlining possibility.
Provides support for runtime arrays in the code that traverses
composite types when checking applicability of transformations that
replace ids with synonyms.
Demotes the image storage class to Private during donation. Also
fixes an issue where instructions that depended on non-donated global
values would not be handled properly.
The SPIR-V data rules say that all uses of an OpSampledImage
instruction must be in the same block as the instruction, and highly
restrict those instructions that can consume the result id of an
OpSampledImage.
This adapts the transformations that split blocks and create synonyms
to avoid separating an OpSampledImage use from its definition, and to
avoid synonym-creation instructions such as OpCopyObject consuming an
OpSampledImage result id.
The management of equation facts suffered from two problems:
(1) The processing of an equation fact required the data descriptors
used in the equation to be in canonical form. However, during
fact processing it can be deduced that certain data descriptors
are equivalent, causing their equivalence classes to be merged,
and that could cause previously canonical data descriptors to no
longer be canonical.
(2) Related to this, if id equations were known about a canonical data
descriptor dd1, and other id equations known about a different
canonical data descriptor dd2, the equation facts about these data
descriptors were not being merged in the event that dd1 and dd2
were deduced to be equivalent.
This changes solves (1) by not requiring equation facts to be in
canonical form while processing them, but instead always checking
whether (not necessary canonical) data descriptors are equivalent when
looking for corollaries of equation facts, rather than comparing them
using ==.
Problem (2) is solved by adding logic to merge sets of equations when
data descriptors are made equivalent.
In addition, the change also requires elements to be registered in an
equivalence relation before they can be made equivalent, rather than
being added (if not already present) at the point of being made
equivalent.
This change increases the extent to which arbitrary SPIR-V can be used
by the fuzzer pass that donates modules. It handles the case where
various ingredients (such as types, variables and particular
instructions) cannot be donated by omitting them, and then either
omitting their dependencies or replacing their dependencies with
alternative instructions.
The change pays particular attention to allowing code that manipulates
image types to be handled (by skipping anything image-specific).
(1) Runtime arrays are turned into fixed-size arrays, by turning
OpTypeRuntimeArray into OpTypeArray and uses of OpArrayLength into
uses of the constant used for the length of the fixed-size array.
(2) Atomic instructions are not donated, and uses of their results are
replaced with uses of constants of the result type.
The fuzzer pass that constructs composites had an issue where it would
regard isomorphic but distinct structs (similarly arrays) as being
interchangeable when constructing composites. This change fixes the
problem by relying less on the type manager.
Some transformations (e.g. TransformationAddFunction) rely on running
the validator to decide whether the transformation is applicable. A
recent change allowed spirv-fuzz to take validator options, to cater
for the case where a module should be considered valid under
particular conditions. However, validation during the checking of
transformations had no access to these validator options.
This change introduced TransformationContext, which currently consists
of a fact manager and a set of validator options, but could in the
future have other fields corresponding to other objects that it is
useful to have access to when applying transformations. Now, instead
of checking and applying transformations in the context of a
FactManager, a TransformationContext is used. This gives access to
the fact manager as before, and also access to the validator options
when they are needed.
In this PR, the classes that represent the toggle access chain
instruction transformation and fuzzer pass were implemented. This
transformation toggles the instructions OpAccessChain and
OpInBoundsAccessChain between them.
Fixes#3193.
This introduces a new fuzzer pass to add instructions to the module
that define equations, and support in the fact manager for recording
equation facts and deducing synonym facts from equation facts.
Initially the only equations that are supported involve OpIAdd,
OpISub, OpSNegate and OpLogicalNot, but there is scope for adding
support for equations over various other operators.
This change adds a fuzzer pass that sprinkles access chain
instructions into a module at random. This allows other passes to
have a richer set of pointers available to them, in particular the
passes that add loads and stores.
Adds a fuzzer pass that inserts function calls into the module at
random. Calls from dead blocks can be arbitrary (so long as they do
not introduce recursion), while calls from other blocks can only be to
livesafe functions.
The change fixes some oversights in transformations to replace
constants with uniforms and to obfuscate constants which testing of
this fuzzer pass identified.
This change ensures that global and local variables donated from other
modules are always initialized at their declaration in the module
being transformed. This is to help limit issues related to undefined
behaviour that might arise due to accessing uninitialized memory.
The change also introduces some helper functions in fuzzer_util to
make it easier to find the pointee types of pointer types.
This change adds fuzzer passes that sprinkle loads and stores into a
module at random, with stores restricted to occur in either dead
blocks, or to use pointers for which it is known that the pointee
value does not influence the module's overall behaviour.
The change also generalises the VariableValueIsArbitrary fact to
PointeeValueIsIrrelevant, to allow stores through access chains or
object copies of variables whose values are known to be irrelevant.
The change includes some other minor refactorings.
Adds two new fuzzer passes to add variables to a module: one that adds
Private storage class global variables, another that adds Function
storage class local variables.
If the fuzzer object-copies a pointer we would like to be able to
perform loads from the copy (and stores to it, if its value is known
not to matter). Undefined and null pointers present a problem here,
so this change disallows copying them.
This change adds a new kind of fact to the fact manager, which records
when a variable (or pointer parameter) refers to an arbitrary value,
so that anything can be stored to it, without affecting the observable
behaviour of the module, and nothing can be guaranteed about values
loaded from it. Donated modules are the current source of such
variables, and other transformations, such as outlining, have been
adapted to propagate these facts appropriately.
This change allows the generator to (optionally and at random) make
the functions of a module "livesafe" during donation. This involves
introducing a loop limiter variable to each function and gating the
number of total loop iterations for the function using that variable.
It also involves eliminating OpKill and OpUnreachable instructions
(changing them to OpReturn/OpReturnValue), and clamping access chain
indices so that they are always in-bounds.
This change refactors some code for walking access chain indexes to
make it mirror the structure of other code (to improve readability in
the first instance and potentially enable a future refactoring to
extract common code), and fixes a problem related to module donation
and function types.
This adds a new kind of fact to the fact manager that knows whether a
block is dead - i.e. guaranteed to be statically unreachable - and a
new transformation for adding a selection construct to a CFG that
conditionally branches to a fresh, dead block, such that the branch
will never be dynamically taken. Transformations that may create new
blocks ('split block' and 'outline function') are updated to propagate
dead block facts to newly-created blocks where appropriate. A fuzzer
pass randomly adds dead blocks to the module.
Future transformations will be able to exploit the fact that such
blocks are known to be dead.
This change adds a fuzzer pass that allows code from other SPIR-V
modules to be donated into the module under transformation. It also
changes the command-line options of the tools so that, in fuzzing
mode, a file must be specified that contains the names of available
donor modules.
In the context of SPIR-V 1.4 or higher, global variables cannot be
used by an instruction unless they are listed in the interface of all
entry points that might invoke the instruction. This change
conservatively adds new global variables to the interfaces of all
entry points (if the SPIR-V version is 1.4 or higher).
Issue #3111 notes that a more rigorous approach to entry point
interfaces could be taken in spirv-fuzz, which would allow being less
conservative here.
This change prevents the spirv-fuzz function outliner from outlining a
region that uses the result of an OpAccessChain not defined inside the
region. Such accesses were turning into parameters to the outlined
function, and the result of an OpAccessChain cannot be passed as a
function parameter according to the SPIR-V specification.
A new transformation and associated fuzzer pass in spirv-fuzz that
selects single-entry single-exit control flow graph regions and for
each selected region outlines the region into a new function and
replaces the original region with a call to this function.
The passes that add dead breaks and continues suffer from the
challenge that a new control flow graph edge can change dominance
information, leading to the potenital for definitions to no longer
dominate their uses. The attempt at guarding against this was known
to be incomplete. This change calls on the SPIR-V validator to do the
necessary checking: in deciding whether adding such an edge would be
legitimate, we clone the module, add the edge, and use the validator
to check whether the transformed clone is valid.
This strategy is heavy-weight, and should be used sparingly, but seems
like a good option when the validity of transformations is intricate,
to avoid reimplementing swathes of validation logic in the fuzzer.
Fixes#2919.
* Validate that if a construct contains a header and it's merge is
reachable, the construct also contains the merge
* updated block merging to not merge into the continue
* update inlining to mark the original block of a single block loop as
the continue
* updated some tests
* remove dead code
* rename kBlockTypeHeader to kBlockTypeSelection for clarity
Adds an option to run the validator on the SPIR-V binary after each
fuzzer pass has been applied, to help identify when the fuzzer has
made the module invalid. Also adds a helper method to allow dumping
of the sequence of transformations that have been applied to a JSON
file.
Prior to this change, TransformationReplaceIdWithSynonym was designed
to be able to replace an id with some synonymous data descriptor,
possibly necessitating extracting from a composite into a fresh id in
order to get at the synonymous data. This change simplifies things so
that TransformationReplaceIdWithSynonym just allows one id to be
replaced by another id. It is the responsibility of the associated
fuzzer pass - FuzzerPassApplyIdSynonyms - to perform the extraction
operations, using e.g. TransformationCompositeExtract.
Inroduces a new transformation that adds a vector shuffle instruction
to the module, with associated facts about how the result vector of
the shuffle relates to the input vectors.
A fuzzer pass to add such transformations is not yet in place.
When a data synonym fact about two composites is added, data synonym
facts between all sub-components of the composites are also added.
Furthermore, when data synonym facts been all sub-components of two
composites are known, a data synonym fact relating the two composites
is added. Identification of this case is done in a lazy manner, when
questions about data synonym facts are asked.
The change introduces helper methods to get the size of an array type
and the number of elements of a struct type, and fixes
TransformationCompositeExtract to invalidate analyses appropriately.
An equivalence relation is computed by traversing the tree of values
rooted at the class's representative. Children were represented by
unordered sets, meaning that the order of values in an equivalence
class could be nondeterministic. This change makes things
deterministic by representing children using a vector.
The path compression optimization employed in the implementation of
the underlying union-find data structure has the potential to change
the order in which elements appear in an equivalence class by changing
the structure of the tree, so the guarantee of determinism is limited
to being a deterministic function of the manner in which the
equivalence relation is updated and inspected.
