Issue #2919 identifies a problem in spirv-fuzz's ability to determine
when it is safe to add a new control flow edge without breaking
dominance rules. This change adds a (currently disabled) test to
expose the issue, and a comment to document that the current solution
is incomplete.
We want to handle OpKill better. The wrap opkill causes lots of extra
code to be generated, even when they are not needed to avoid the main
problem: OpKill cannot be found directly in a continue construct.
This change will be more selective on which functions the OpKill will be
wrapped and inlining will avoid inlining.
Fixes#2912
Adds a spirv-fuzz option for converting a SPIR-V shader into a shader
that renders red, whilst containing the body of the original shader.
This is for aiding in compiler crash bug reporting.
* Add continue construct analysis to struct cfg analysis
Add the ability to identify which blocks are in the continue construct for a
loop, and to get functions that are called from those blocks, directly or
indirectly.
Part of https://github.com/KhronosGroup/SPIRV-Tools/issues/2912.
There is nothing in the spir-v spec that says the last
instructions in a module cannot be OpLine or OpNoLine.
However, the code that parses the module will simply drop
these instructions.
We add code that will preserve these instructions.
Strip-debug-info is updated to remove these instructions.
Fixes https://crbug.com/1000689.
Because dominance information becomes a bit unreliable when blocks are
unreachable, this change makes it so that the 'dead break'
transformation will not introduce a break to an unreachable block.
Fixes#2907.
The performance of the fuzzer was unacceptable in the 'permute blocks'
transformation, due to dominator analysis being repeatedly invalidated
and recomputed. This change preserves the dominator analysis,
together with the CFG analysis, when a block is moved down.
This change introduces a robust check for whether an index in an
access chain is indexing into a struct, in which case the index needs
to be an OpConstant and cannot be replaced with a synonym.
Fixes#2906.
Issues #2898 and #2900 identify some cases where adding a dead
continue would lead to an invalid module, and these turned out to be
due to the lack of sensible dominance information when a continue
target is unreachable. This change requires that the header of a loop
dominates the loop's continue target if a dead continue is to be
added.
Furthermore, issue #2905 identified a shortcoming in the algorithm
being used to identify when it is OK, from a dominance point of view,
to add a new break/continue edge to a control flow graph. This change
replaces that algorithm with a simpler and more obviously correct
algorithm (that incidentally does not require the new edge to be a
break/continue edge in particular).
Fixes#2898.
Fixes#2900.
Fixes#2905.
Before this change, spirv-fuzz would replace a pointer argument to a
function call with a synonym, which is problematic when the synonym is
not a memory object declaration, since function call arguments are
required to be memory object declarations. This change adds a check
to ensure that such a replacement is not made.
Fixes#2896.
Before this change, spirv-fuzz would replace a constant boolean
argument to an OpPhi with the result of a binary operation, inserting
the instruction to compute the binary operation right before the
OpPhi, leading to an invalid module. This change conservatively
disallows replacing OpPhi arguments. Issue #2902 notes that there is
scope for being less conservative.
Fixes#2897.
* Handle extract with no indexes
It is possible that OpCompositeExtract instructions will not have any
indexes. This is not handled well by scalar replacement and instruction
folding.
Fixes https://crbug.com/1006435
* Fix typo.
This change to spirv-fuzz uses ideas from "Swarm Testing" (Groce et al. 2012), so that a random subset of fuzzer passes are enabled. These passes are then applied repeatedly in a randomized fashion, with the aggression with which they are applied being randomly chosen per pass.
There is plenty of scope for refining the probabilities introduce in this change; this is just meant to be a reasonable first effort.
* Use OpReturn* in wrap-opkill
The warp-opkill pass is generating incorrect code. It is placing an
OpUnreachable at the end of a basic block, when the block can be
reached. We can't reach the end of the block, but we can reach the end.
Instead we will add a return instruction.
Fixes#2875.
To aid in debugging issues in spirv-fuzz, this change adds an option whereby the SPIR-V module is validated after each transformation is applied during replay. This can assist in finding a transformation that erroneously makes the module invalid, so that said transformation can be debugged.
The warp-opkill pass is generating incorrect code. It is placing an
OpUnreachable at the end of a basic block, when the block can be
reached. We can't reach the end of the block, but we can reach the end.
Instead we will add a return instruction.
Fixes#2875.
Many of the places in copy propagate arrays assumes that integer constant will be defined by an OpConstant instruction. That is not always true. We fix these spots by allowing for an OpConstantNull.
spirv-fuzz generates protobuf sources in a 'protobuf' directory. When
building with Unix Makefiles, compilation would fail due to to this
directory not existing. This change causes the directory to be
created when the build is prepared.
If the fuzzer's fact manager knows that ids A and B are synonymous, it
can replace a use of A with a use of B, so long as various conditions
hold (e.g. the definition of B must dominate the use of A, and it is
not legal to replace a use of an OpConstant in a struct's access chain
with a synonym that is not an OpConstant).
This change adds a fuzzer pass to sprinke such synonym replacements
through the module.
* When input or result is a pointer type also allow 32-bit integer
vectors for the other type
* Relaxation only applies to SPIR-V 1.5 or in the presence of
SPV_KHR_physical_storage_buffer
* new tests
* Vulkan specific checks
* storage buffer variables must be structs or arrays of structs
* storage buffer struct must be Block decorated
* uniform struct must be Block or BufferBlock decorated
* new tests
* Ensure same enum values have consistent extension lists
* val: fix checking of capabilities
The operand for an OpCapability should only be
checked for the extension or core version.
The InstructionPass registers a capability, and all its implied
sub-capabilities before actually checking the operand to an
OpCapability.
* Add basic support for SPIR-V 1.5
- Adds SPV_ENV_UNIVERSAL_1_5
- Command line tools default to spv1.5 environment
- SPIR-V 1.5 incorporates several extensions. Now the disassembler
prefers outputing the non-EXT or non-KHR names. This requires
updates to many tests, to make strings match again.
- Command line tests: Expect SPIR-V 1.5 by default
* Test validation of SPIR-V 1.5 incorporated extensions
Starting with 1.5, incorporated features no longer require
the associated OpExtension instruction.
A new fuzzer pass that randomly introduces OpCopyObject instructions
that make copies of ids, and uses the fact manager to record the fact
that an id %id is synonymous with an id generated by an OpCopyObject
applied to %id. (A future pass will exploit such synonym facts.)
If an OpKill instruction is inlined into a continue construct, then the
spir-v is no longer valid. To avoid this issue, we do inline into an
OpKill at all. This method was chosen because it is difficult to keep
track of whether or not you are in a continue construct while changing
the function that is being inlined into. This will work well with wrap
OpKill because every will still be inlined except for the OpKill
instruction itself.
Fixes#2554Fixes#2433
This reverts commit aa9e8f5380.
Before this change there was quite a lot of duplication in the code
being used to choose random percentages, and some of it was incorrect
so that a percentage chance of (100-N)% instead of N% was being used.
Also there was a lot of duplicate code to choose a random index into a
vector. This change eliminates that duplication (fixing up the
percentage problem), and gets rid of direct access to the random
number generator being used for fuzzing, so that all randomization
requests must go through the FuzzerContext class, discouraging future
ad-hoc uses of the random number generator.
The implementation of these passes had overlooked the fact that adding
a new edge to a control flow graph can change dominance information.
Adding a dead break/continue risks causing uses to no longer be
dominated by their definitions. This change introduces various tests
to expose such scenarios, and augments the preconditions for these
transformations with checks to guard against the situation.
* Handle id overflow in the ssa rewriter.
Remove LocalSSAElim pass at the same time. It does the same thing as the SSARewrite pass. Then even share almost all of the same code.
Fixes crbug.com/997246
As far as I know, it is legal to have multiple decoration adding the
same decoration to the same id. The validator registers all of these
decoration as if they were distinct decorations. This can cause poor
memory usage and performance in some cases.
This fix is to make sure that duplicates are not registers.
I keep the type of the decoration list as an std::vector because I
expect it to be small enough in most cases that the linear search will
still be faster that using some type of map.
No tests are added because we do not have a mechanism to test memory
usage in our unit tests.
Fixes#2837. The total memory usage drop to 14,236KB.
The first pass applies the RelaxedPrecision decoration to all executable
instructions with float32 based type results. The second pass converts
all executable instructions with RelaxedPrecision result to the equivalent
float16 type, inserting converts where necessary.
Add the first steps to removing the AMD extension VK_AMD_shader_ballot.
Splitting up to make the PRs smaller.
Adding utilities to add capabilities and change the version of the
module.
Replaces the instructions:
OpGroupIAddNonUniformAMD = 5000
OpGroupFAddNonUniformAMD = 5001
OpGroupFMinNonUniformAMD = 5002
OpGroupUMinNonUniformAMD = 5003
OpGroupSMinNonUniformAMD = 5004
OpGroupFMaxNonUniformAMD = 5005
OpGroupUMaxNonUniformAMD = 5006
OpGroupSMaxNonUniformAMD = 5007
and extentend instructions
WriteInvocationAMD = 3
MbcntAMD = 4
Part of #2814
If they are not aliased, the function will always print the message:
"Binary unexpectedly changed despite optimizer saying there was no change"
Which is (usually) totally bogus.
Fixes#2798
* Refactor instruction folders
We want to refactor the instruction folder to allow different sets of
rules to be added to the instruction folder. We might want different
sets of rules in different circumstances.
We also need a way to add rules for extended instructions. Changes are
made to the FoldingRules class and ConstFoldingRules class to enable
that.
We added tests to check that we can fold extended instructions using the
new framework.
At the same time, I noticed that there were two tests that did not tests
what they were suppose to. They could not be easily salvaged. #2813 was
opened to track adding the new tests.
Adds a reduction pass that removes OpDecorate and OpMemberDecorate
instructions that annotate instructions and members with
RelaxedPrecision. As well as being useful in its own right, removing
such references allows other passes to remove further instructions.
Now we need to handle id overflow when we overflow while replacing uses of the variable. While looking at this code, I noticed an error in the way we handle access chains that cannot be replaced because of overflow. Name it will make some change, and then give up by returning SuccessWithoutChange. But it was changed.
This is fixed up by returning Failure if we notice the error at the time of rewriting the users. This is for both id overflow or out-of-bounds accesses.
Code is added to "CheckUses" to remove variables that have out-of-bounds accesses from the candidate list, so we don't even try to rewrite its uses.
Fixes https://crbug.com/995032
If we run out of ids when creating a new variable, sroa does not recognize
the error, and continues doing work. This leads to segmentation faults.
Fixes https://crbug/969655
`#include <source/util/string_utils.h>` works only when we specify
`include_directories(${CMAKE_CURRENT_SOURCE_DIR}/)` in
cmake. It is hard to set the source directory as a include path
in some build systems e.g., bazel. Using the relative path easily
solves this issue. This commit uses
`#include "source/util/string_utils.h"` instead of
`#include <source/util/string_utils.h>`.
Fixes#2793
* Don't special case matrix validation compared to other composites
* just check the constituents are constants or undefs
* later checking validates the column type
* new test
We are no able to inline OpKill instructions into a continue construct.
See #2433. However, we have to be able to inline to correctly do
legalization. This commit creates a pass that will wrap OpKill
instructions into a function of its own. That way we are able to inline
the rest of the code.
The follow up to this will be to not inline any function that contains
an OpKill.
Fixes#2726
This also fixes ADCE to not remove possibly needed OpTypeForwardPointer.
The bug, its fix and the corresponding test have a circular dependency
with the extension, so they are packaged together.
If a member of a struct has a relaxed precision, sroa will not split the
struct. This means we do not get all cases. This commit handles these
cases. The other part is that the decoration needs to be passed on to
the new variables.
Fixes#2786
This transformation can introduce an instruction that uses
OpCopyObject to make a copy of some other result id. This change
introduces the transformation, but does not yet introduce a fuzzer
pass to actually apply it.
Fixes#2768
* In scalar replacement, interpret access chain indexes as signed counts
* Use Constant::GetSignExtendedValue and Constant::GetZeroExtendedValue
where appropriate
* new tests
spirv-opt: Add --graphics-robust-access
Clamps access chain indices so they are always
in bounds.
Assumes:
- Logical addressing mode
- No runtime-array-descriptor-indexing
- No variable pointers
Adds stub code for clamping coordinate and samples
for OpImageTexelPointer.
Adds SinglePassRunAndFail optimizer test fixture.
Android.mk: add source/opt/graphics_robust_access_pass.cpp
Adds Constant::GetSignExtendedValue, Constant::GetZeroExtendedValue
This makes it symmetric with the result type of ...->element_type which
returns a const Type.
So now we can write code like this:
analysis::Vector v = ...
analysis::Vector(v->element_type(), 2);
This fixes#2608.
The original test case had an out-of-bounds reference that ended up
folding into OpCompositeExtract that was indexing right outside the
constant composite.
The returned constant would then cause a segfault during constant
propagation.
Fixes#2764
* Don't replace all uses when simplifying instructions, instead only
update non-debug, non-decoration uses
* added a test
* Add a new version of RAUW that takes a predicate to decide whether to
replace the use or not
* used in simplification pass
* Fix#2609 - Handle out-of-bounds scalar replacements.
When SROA tries to do a replacement for an OpAccessChain that is exactly
one element out of bounds, the code was trying to access its internal
array of replacements and segfaulting.
This protects the code from doing this, and it additionally fixes the
way SROA works by not returning failure when it refuses to do a
replacement. Instead of failing the optimization pass, SROA will now
simply refuse to do the replacement and keep going.
Additionally, this patch fixes the SROA logic to now return a proper status so we can
correctly state that the pass made no changes to the IR if it only found
invalid references.
Merge return expects unreachable merge block to look a certain way, and
unreachable continue blocks to look a certain way. What if an
unreachable block is both a merge and a continue? The continue is
suppose to take precedent, but merge-return implements it with the merge
taking precedent. This change flips that around.
Fixes#2746
Similar to the existing 'add dead breaks' pass, this adds a pass to
add dead continues to blocks in loops where such a transformation is
viable. Various functionality common to this new pass and 'add dead
breaks' has been factored into 'fuzzer_util', and some small
improvements to 'add dead breaks' that were identified while reviewing
that code again have been applied.
Fixes#2719.
* Process OpDecorateId in ADCE
When there is an OpDecorateId instruction that is live,
the ids that is references must be kept live. This change
adds them to the worklist.
I've also updated a validator check to allow OpDecorateId
to be able to apply to decoration groups.
Fixes#1759.
* Remove dead code.
In merge return, we need to know the original dominator for a block in order to
traverse code from the original dominator to the new dominator and add
appropriate Phi nodes. The current code gets this wrong because the dominator
tree is build as needed. The first time we get the immediate dominator for a
function we just built the dominator tree and it takes into account that a
block has been split. The second time it does not.
This inconsistency needs to be fixed. We do that by recording the original
dominator for all blocks at the start of the pass.
If we were to record just the basic block, that could change if the block is
split. We want to traverse the code in the body of the original dominator,
whatever block it ends up in. To make this easy to track, we not save the
terminator instruction to represent the original dominator.
Fixes#2745
When a phi candidate is marked as trivial, we are suppose to update all
of its uses to the reference the value that it is being folded to.
However, the code updates the uses misses `defs_at_block_`. So at a
later time, the id for the trivial phi can reemerge.
Fixes#2744
* Bindless Instrument: Make init check depend solely on input_init_enabled
Previously was dependent on presense of descriptor_indexing extension
in SPIR-V, but this missed some cases. Tests updated to refect this new
policy.
* Fix format.
This change refactors all storage class validation for atomics
to reflect the similar refactoring in the specification.
It is currently not possible to write a test for the check
rejecting Generic in an OpenCL 1.2 environment as the required
GenericPointer capability isn't allowed there. I've decided
to keep the check nonetheless to guard against the capability
becoming available without the rules for atomics being updated.
The ID changes in existing tests aren't ideal but introducing
names drags in a substantial refactoring of this file.
Contributes to #2595.
Signed-off-by: Kevin Petit <kevin.petit@arm.com>
* Fix bug in merge return
The merge return pass seems to assume that the only new edges in the cfg
are from return block to merge blocks. However, it is possible that a
merge block branches to a merge block when it did not before.
This change add a new variable to track all of the new edges. It also
renames some other variables and cleans us the code to make it a bit
easier to read.
Fixes#2702.
Dead branch elimination needs to know about the constructs that a block is contained it when determining what to do with its merge instruction. We currently fold branches in block as we see them, which is parent constructs before their children. This causes the struct cfg analysis to crash because it tries to get the parent construct for a block after the parent has been folded.
This can be fixed by folding the branch of the children before the parents.
Fixes#2667.
There are a couple spots where we are not looking at decorations when we should.
1. Value numbering is suppose to assign a different value number to ids if they have different decorations. However that is not being done for OpCopyObject and OpPhi.
1. Instruction simplification is propagating OpCopyObject instruction without checking for decorations. It should only do that if no decorations are being lost.
Add a new function to the decoration manager to check if the decorations of one id are a subset of the decorations of another.
Fixes#2715.
Fixes#2669
* Check capabilities when validating variables
* validate load and store types
* Constant check
* Don't checks pointers for stores, constants and loads
* Validate composite instructions
* Validate conversions for 8- and 16-bit limited types
* Unified tests and expanded them
* Disallow OpCopyMemory
* new tests and update old tests
Adds to spirv-fuzz the option to shrink a sequence of transformations
that lead to an interesting binary to be generated, to find a smaller
sub-sequence of transformations that still lead to an interesting (but
hopefully simpler) binary being generated. The notion of what counts
as "interesting" comes from a user-provided script, the
"interestingness function", similar to the way the spirv-reduce tool
works. The shrinking process will give up after a maximum number of
steps, which can be configured on the command line.
Tests for the combination of fuzzing and shrinking are included, using
a variety of interestingness functions.
Inlining does not inline functions that have a single return that is in a loop. This is because the return cannot be replaced by a branch outside of the loop easily. Merge return knows how to rewrite the function so the return is replaced by a branch.
Fixes#2038.
It is illegal to inline an OpKill instruction into a continue construct because the continue header will no longer dominate the backedge.
This commit adds a check for this, and does not inline.
If we still want to be able to inline a function that contains an OpKill, we can add a new pass that will wrap OpKill instructions into its own function with just the single instruction.
I do not believe that this is a common case right now, so I will not do that yet.
Fixes#2433.
When working on descriptor indexing validation for compute shaders, the
gl_GlobalInvocationID builtin was being loaded as uint which would cause
compute shaders instrumented by the bindless check pass to have:
%83 = OpLoad %uint %gl_GlobalInvocationID
%84 = OpCompositeExtract %uint %83 0
%85 = OpCompositeExtract %uint %83 1
%86 = OpCompositeExtract %uint %83 2
which results in validation failures:
error: line 127: Reached non-composite type while indexes still remain
to be traversed.
%84 = OpCompositeExtract %uint %83 0
for trying to extract a uint from a uint.
Fixes#2621.
Instead of aborting when an invalid input fact is provided, the tool
now warns about the invalid fact and then ignores it. This is
convenient for example if facts are specified about uniforms with
descriptor sets and bindings that happen to not be present in the
input binary.
Fixes#2695. Allowing unreachable blocks to be moved can lead to an
unreachable block A getting placed after an unreachable successor B,
which is a problem if B uses ids that A generates.
* Replace global static map with an array of pairs
\#2687 introduced a global static map, which isn't allowed by
the style guide and caused an issue in DXC.
This change replaces it with an array of pairs.
Signed-off-by: Kévin Petit <kpet@free.fr>
* Replace constexpr with const
Signed-off-by: Kévin Petit <kpet@free.fr>
Several tools take a --target-env option to specify the SPIR-V
environment to use. They all use spvParseTargetEnv to parse
the user-specified string and select the appropriate spv_target_env
but all tools list only _some_ of the valid values in their help
text.
This change makes the help text construction automatic from the
full list of valid values, establishing a single source of truth
for the values printed in the help text. The new utility function
added allows its user to specify padding and wrapping constraints
so the produced strings fits well in the various help texts.
Signed-off-by: Kévin Petit <kpet@free.fr>
* Represent uniform facts via descriptor set and binding.
Previously uniform facts were expressed with resepect to the id of a
uniform variable. Describing them with respect to a descriptor set
and binding is more convenient from the point of view of expressing
facts about a shader without requiring analysis of its SPIR-V.
* Fix equality testing for uniform buffer element descriptors.
The equality test now checks that the lengths of the index vectors
match. Added a test that exposes the previous omission.
Adds a new transformation that can replace a constant with a uniform known to have the same value, and adds a fuzzer pass that (a) replaces a boolean with a comparison of literals (e.g. replacing "true" with "42 > 24"), and then (b) obfuscates the literals appearing in this comparison by replacing them with identically-valued uniforms, if available.
The fuzzer_replayer test file has also been updated to allow initial facts to be provided, and to do error checking of the status results returned by the fuzzer and replayer components.
* Can only be used with Vulkan memory model
* Can only be used with atomics
* Bit setting must match for compare exchange opcodes
* Updated memory semantics checks to allow constant instructions
generally with CooperativeMatrixNV
The replayer takes an existing sequence of transformations and applies
them to a module. Replaying a sequence of transformations that were
obtained via fuzzing should lead to an identical module to the module
that was fuzzed. Tests have been added to check for this.
Adds a new (and first) kind of fact to the fact manager, which is that
a specific uniform value is guaranteed to be equal to a specific
constant. The point of this is that such information (if known to be
true by some external source) can be used by spirv-fuzz to transform
the module in interesting ways that a static compiler cannot reverse
via compile-time analysis.
This change introduces protobuf messages for the fact, and adds
capabilities to the fact manager to store this kind of fact and
provide information about it.
The transformation can, for example, replace "true" with "12.0 > 6.0",
if constants for those floating-point values are available.
This introduces a new 'id use descriptor' structure, which provides a
way to describe a particular use of an id, and which will be heavily
used in future transformations. Describing an id use is trivial if
the use occurs in an instruction that itself generates an id, but is
less straightforward if the id of interest is used by an instruction
such as OpStore that does not have a result id. The 'id use
descriptor' structure caters for such cases.
Also add a Builtin test generator variant that takes
capabilities and extensions.
Tests
- verify that the SMCountNV, SMIDNV, WarpsPerSMNV, and WarpIDNV Builtins are
accepted as Inputs in Vertex, Fragment, TessControl, TessEval, Geometry,
and Compute.
- verify that the SMCountNV, SMIDNV, WarpsPerSMNV, and WarpIDNV Builtins are
accepted as Inputs in MeshNV and TaskNV shaders.
- verify that the SMCountNV, SMIDNV, WarpsPerSMNV, and WarpIDNV Builtins are
accepted as Inputs in the 6 ray tracing stages
- verify that the SMCountNV, SMIDNV, WarpsPerSMNV, and WarpIDNV Builtins are
NOT accepted as Outputs.
- verify that the SMCountNV, SMIDNV, WarpsPerSMNV, and WarpIDNV Builtins are
NOT accepted as non-scalar integers (f32, uvec3)
- verify that the SMCountNV, SMIDNV, WarpsPerSMNV, and WarpIDNV Builtins are
NOT accepted as non-32-bit integers (u64)
There turned out to be a bug in the 'split blocks' transformation due
to blocks being split while they were being iterated over. This
change fixes that issue, and adds tests that were able to expose the
issue by running the fuzzer on some example shaders.
When it's an OpConstant or OpSpecConstant, then the literal
values are compared. If the OpSpecConstant also has a SpecId
decoration, then that's also compared.
Otherwise, it's an OpSpecConstantOp and we only compare the
ID of the OpSpecConstantOp instruction itself.
Fixes#2649
