Some transformations (e.g. TransformationAddFunction) rely on running
the validator to decide whether the transformation is applicable. A
recent change allowed spirv-fuzz to take validator options, to cater
for the case where a module should be considered valid under
particular conditions. However, validation during the checking of
transformations had no access to these validator options.
This change introduced TransformationContext, which currently consists
of a fact manager and a set of validator options, but could in the
future have other fields corresponding to other objects that it is
useful to have access to when applying transformations. Now, instead
of checking and applying transformations in the context of a
FactManager, a TransformationContext is used. This gives access to
the fact manager as before, and also access to the validator options
when they are needed.
This change adds a fuzzer pass that allows code from other SPIR-V
modules to be donated into the module under transformation. It also
changes the command-line options of the tools so that, in fuzzing
mode, a file must be specified that contains the names of available
donor modules.
Add support for SPV_KHR_non_semantic_info
This entails a couple of changes:
- Allowing unknown OpExtInstImport that begin with the prefix `NonSemantic.`
- Allowing OpExtInst that reference any of those sets to contain unknown
ext inst instruction numbers, and assume the format is always a series of IDs
as guaranteed by the extension.
- Allowing those OpExtInst to appear in the types/variables/constants section.
- Not stripping OpString in the --strip-debug pass, since it may be referenced
by these non-semantic OpExtInsts.
- Stripping them instead in the --strip-reflect pass.
* Add adjacency validation of non-semantic OpExtInst
- We validate and test that OpExtInst cannot appear before or between
OpPhi instructions, or before/between OpFunctionParameter
instructions.
* Change non-semantic extinst type to single value
* Add helper function spvExtInstIsNonSemantic() which will check if the extinst
set is non-semantic or not, either the unknown generic value or any future
recognised non-semantic set.
* Add test of a complex non-semantic extinst
* Use DefUseManager in StripDebugInfoPass to strip some OpStrings
* Any OpString used by a non-semantic instruction cannot be stripped, all others
can so we search for uses to see if each string can be removed.
* We only do this if the non-semantic debug info extension is enabled, otherwise
all strings can be trivially removed.
* Silence -Winconsistent-missing-override in protobufs
Adds an option to run the validator on the SPIR-V binary after each
fuzzer pass has been applied, to help identify when the fuzzer has
made the module invalid. Also adds a helper method to allow dumping
of the sequence of transformations that have been applied to a JSON
file.
* Remove Impl struct in Reducer; we can re-add it later (in a cleaner fashion) if we need to.
* Add cleanup passes in Reducer; needed so that removal of constants can be disabled during the main passes, and then enabled during cleanup passes, otherwise some main passes can perform worse due to lack of available constants.
* Delete passes: remove op name, remove relaxed precision. And delete associated tests.
* Add more tests for remove unreferenced instructions.
* Always return and write the output file, even if there was a reduction failure.
* Only exit with 0 if the reduction completed or we hit the reduction step limit.
Adds a spirv-fuzz option for converting a SPIR-V shader into a shader
that renders red, whilst containing the body of the original shader.
This is for aiding in compiler crash bug reporting.
* reduce: add -o.
* reduce: add --temp-file-prefix.
* reduce: add interestingness test args.
* Detect bad args with one dash e.g. -a.
* reduce: fix validator args.
* Add = to args that require it.
* More consistent naming/style across fuzz/reduce.
* Change some 0 exit codes to 1.
To aid in debugging issues in spirv-fuzz, this change adds an option whereby the SPIR-V module is validated after each transformation is applied during replay. This can assist in finding a transformation that erroneously makes the module invalid, so that said transformation can be debugged.
* Ensure same enum values have consistent extension lists
* val: fix checking of capabilities
The operand for an OpCapability should only be
checked for the extension or core version.
The InstructionPass registers a capability, and all its implied
sub-capabilities before actually checking the operand to an
OpCapability.
* Add basic support for SPIR-V 1.5
- Adds SPV_ENV_UNIVERSAL_1_5
- Command line tools default to spv1.5 environment
- SPIR-V 1.5 incorporates several extensions. Now the disassembler
prefers outputing the non-EXT or non-KHR names. This requires
updates to many tests, to make strings match again.
- Command line tests: Expect SPIR-V 1.5 by default
* Test validation of SPIR-V 1.5 incorporated extensions
Starting with 1.5, incorporated features no longer require
the associated OpExtension instruction.
The first pass applies the RelaxedPrecision decoration to all executable
instructions with float32 based type results. The second pass converts
all executable instructions with RelaxedPrecision result to the equivalent
float16 type, inserting converts where necessary.
Add the first steps to removing the AMD extension VK_AMD_shader_ballot.
Splitting up to make the PRs smaller.
Adding utilities to add capabilities and change the version of the
module.
Replaces the instructions:
OpGroupIAddNonUniformAMD = 5000
OpGroupFAddNonUniformAMD = 5001
OpGroupFMinNonUniformAMD = 5002
OpGroupUMinNonUniformAMD = 5003
OpGroupSMinNonUniformAMD = 5004
OpGroupFMaxNonUniformAMD = 5005
OpGroupUMaxNonUniformAMD = 5006
OpGroupSMaxNonUniformAMD = 5007
and extentend instructions
WriteInvocationAMD = 3
MbcntAMD = 4
Part of #2814
spirv-opt: Add --graphics-robust-access
Clamps access chain indices so they are always
in bounds.
Assumes:
- Logical addressing mode
- No runtime-array-descriptor-indexing
- No variable pointers
Adds stub code for clamping coordinate and samples
for OpImageTexelPointer.
Adds SinglePassRunAndFail optimizer test fixture.
Android.mk: add source/opt/graphics_robust_access_pass.cpp
Adds Constant::GetSignExtendedValue, Constant::GetZeroExtendedValue
Adds to spirv-fuzz the option to shrink a sequence of transformations
that lead to an interesting binary to be generated, to find a smaller
sub-sequence of transformations that still lead to an interesting (but
hopefully simpler) binary being generated. The notion of what counts
as "interesting" comes from a user-provided script, the
"interestingness function", similar to the way the spirv-reduce tool
works. The shrinking process will give up after a maximum number of
steps, which can be configured on the command line.
Tests for the combination of fuzzing and shrinking are included, using
a variety of interestingness functions.
Fixes#2621.
Instead of aborting when an invalid input fact is provided, the tool
now warns about the invalid fact and then ignores it. This is
convenient for example if facts are specified about uniforms with
descriptor sets and bindings that happen to not be present in the
input binary.
Several tools take a --target-env option to specify the SPIR-V
environment to use. They all use spvParseTargetEnv to parse
the user-specified string and select the appropriate spv_target_env
but all tools list only _some_ of the valid values in their help
text.
This change makes the help text construction automatic from the
full list of valid values, establishing a single source of truth
for the values printed in the help text. The new utility function
added allows its user to specify padding and wrapping constraints
so the produced strings fits well in the various help texts.
Signed-off-by: Kévin Petit <kpet@free.fr>
The replayer takes an existing sequence of transformations and applies
them to a module. Replaying a sequence of transformations that were
obtained via fuzzing should lead to an identical module to the module
that was fuzzed. Tests have been added to check for this.
The current tool can parse basic command-line argument, but generates
a binary identical to the input binary, since no transformations are
yet implemented.
* Make pointers to logically matching types interchangeable with option.
DXC will be generating code where the function parameters will be a more
generic type that the actual parameter. They should be logically
matching and the decorations of the actual parameter must be a superset
of the decorations of the formal parameter.
We want to accept this code with an options so that spirv-opt can then
inline and fix the type mismatch. We will accept this under a new
options `--before-hlsl-legalization`.
The new option will also imply `relax-logical-pointer` so that HLSL
frontends will need to use just the one more generic option.
Moved the |LogicallyMatches| to the validation state to make it
available in more places. Also added a parameter to have it check the
decorations. I did not do a separate function for the decorations
because checking the decorations involves making sure the types
logically match anyway.
Fixes#2535
* SPIR-V 1.4 headers, add SPV_ENV_UNIVERSAL_1_4
* Support --target-env spv1.4 in help for command line tools
* Support asm/dis of UniformId decoration
* Validate UniformId decoration
* Fix version check on instructions and operands
Also register decorations used with OpDecorateId
* Extension lists can differ between enums that match
Example: SubgroupMaskEq vs SubgroupMaskEqKHR
* Validate scope value for Uniform decoration, for SPIR-V 1.4
* More unioning of exts
* Preserve grammar order within an enum value
* 1.4: Validate OpSelect over composites
* Tools default to 1.4
* Add asm/dis test for OpCopyLogical
* 1.4: asm/dis tests for PtrEqual, PtrNotEqual, PtrDiff
* Basic asm/Dis test for OpCopyMemory
* Test asm/dis OpCopyMemory with 2-memory access
Add asm/dis tests for OpCopyMemorySized
Requires grammar update to add second optional memory access operand
to OpCopyMemory and OpCopyMemorySized
* Validate one or two memory accesses on OpCopyMemory*
* Check av/vis on CopyMemory source and target memory access
This is a proposed rule. See
https://gitlab.khronos.org/spirv/SPIR-V/issues/413
* Validate operation for OpSpecConstantOp
* Validate NonWritable decoration
Also permit NonWritable on members of UBO and SSBO.
* SPIR-V 1.4: NonWrtiable can decorate Function and Private vars
* Update optimizer CLI tests for SPIR-V 1.4
* Testing tools: Give expected SPIR-V version in message
* SPIR-V 1.4 validation for entry point interfaces
* Allow only unique interfaces
* Allow all global variables
* Check that all statically used global variables are listed
* new tests
* Add validation fixture CompileFailure
* Add 1.4 validation for pointer comparisons
* New tests
* Validate with image operands SignExtend, ZeroExtend
Since we don't actually know the image texel format, we can't fully
validate. We need more context.
But we can make sure we allow the new image operands in known-good
cases.
* Validate OpCopyLogical
* Recursively checks subtypes
* new tests
* Add SPIR-V 1.4 tests for NoSignedWrap, NoUnsignedWrap
* Allow scalar conditions in 1.4 with OpSelect
* Allows scalar conditions with vector operands
* new tests
* Validate uniform id scope as an execution scope
* Validate the values of memory and execution scopes are valid scope
values
* new test
* Remove SPIR-V 1.4 Vulkan 1.0 environment
* SPIR-V 1.4 requires Vulkan 1.1
* FIX: include string for spvLog
* FIX: validate nonwritable
* FIX: test case suite for member decorate string
* FIX: test case for hlsl functionality1
* Validation test fixture: ease debugging
* Use binary version for SPIR-V 1.4 specific features
* Switch checks based on the SPIR-V version from the target environment
to instead use the version from the binary
* Moved header parsing into the ValidationState_t constructor (where
version based features are set)
* Added new versions of tests that assemble a 1.3 binary and validate a
1.4 environment
* Fix test for update to SPIR-V 1.4 headers
* Fix formatting
* Ext inst lookup: Add Vulkan 1.1 env with SPIR-V 1.4
* Update spirv-val help
* Operand version checks should use module version
Use the module version instead of the target environment version.
* Fix comment about two-access form of OpCopyMemory
Renames the existing flag '--webgpu-mode' to '--vulkan-to-webgpu' for
the Vulkan->WebGPU operation, and adds a new flag '--webgpu-to-vulkan'
for the WebGPU->Vulkan operation.
Currently '--webgpu-to-vulkan' doesn't have any passes associated with
it yet, but further patches will implement them.
Fixes#2495
Fix#2475. Fix#2476.
* Improve reducer algorithm: shrink granularity, remove an early return, no lazy initialization, notify pass if binary is interesting, add comments.
* Add fail-on-validation-error option to fail a reduction if an invalid state is reached; useful for tests.
* Set fail-on-validation-error in tests.
* Improve some documentation comments.
* Add Reducer::AddDefaultReductionPasses so tests (and other library consumers) can add the default reduction passes.
* Add CLIMessageConsumer in test_reduce so we can see messages for tricky tests.
* Remove test RemoveUnreferencedInstructionReductionPassTest_ApplyReduction because it was indirectly testing the reduction algorithm, not the RemoveUnreferencedInstruction pass.
* Tweak tests where needed.
Fix#2396
* Check that initial state is valid. Add kInitialStateInvalid.
* Fix RemoveOpnameAndRemoveUnreferenced test; turns out the original shader is invalid, but we never notice because we don't check this and the reduced shader is valid; fix original shader. Assert reduction status is kComplete.
* Always check return value from `Reducer::Run`.
* Change Reducer::Run to *not* immediately copy the input binary.
Adds an optimization pass to remove usages of AtomicCounterMemory
bit. This bit is ignored in Vulkan environments and outright forbidden
in WebGPU ones.
Fixes#2242
* Fixes#2358. Added to the reducer the ability to remove a function that is not directly called. Factored out some code from the optimizer to help with this.
