* linker: Address conversion error introduced in earlier rework
Also rework the code slightly to first compute the new ID bound and
validate it, and only then, offset all existing IDs.
This makes those two steps clearer, and avoids potentially overflowing
the IDs within a module (even if that would have been caught later on).
Fixes: c3849565 ("Linker improvements (#4679)")
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4684
* test/linker: Disable IdsLimit tests for now
They are taking too long to run and results in the CI jobs timing out.
* test/linker: Code factorisation and small tweaks
* linker: Do not fail when going over limits
The limits are minima and implementations or APIs might support higher
limits, so just warn the user about it. And only check for the limits
right before emitting the binary, as limits might change earlier when
removing duplicate instructions, function prototypes, etc.
The only check performed right before merging, is making sure the ID
bound will not overflow the 32 bits following the merge.
Also, use the defines for the limits instead of hard-coding them.
* linker: Require a memory model in each input module
The existing code could run into weird situation. For example, if the
first module had no memory model, it would not emit any memory model
(sort of reasonable) and would accept without complains all possible mix
from later modules as it would not verify them.
* linker: Replace hex version with SPV_SPIRV_VERSION_WORD
* linker: Error out when linking together different versions
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4135
* tools/linker: Do not write to disk if linking failed
Also, do not consider warnings as errors.
* tools/linker: Fix formatting in help message
* tools/linker: Further clarify the use of --target-env
Also update the text for the default version to reflect the change made
in 7d768812 ("Basic support for SPIR-V 1.6 (#4663)").
The test harness for the opt fuzzer was failing to consider that the
input might use a very large id bound, despite no id approaching this
bound actually being used.
This change modifies the test harness to use the module's id bound,
rather than looking through the module for large ids.
Fixes: oss-fuzz:42386
For a shader input/output interface variable of structure type:
If the structure has BuiltIn members, then the structure type
must be Block decorated.
Otherwise, the variable, or the struct members must have locations,
but nothing can have both a location be a BuiltIn.
Implements validation needed to reject the example in
https://github.com/KhronosGroup/SPIRV-Registry/issues/134
* Basic support for SPIR-V 1.6
* Update SPIRV-Headers deps
* Add new environment enum for SPIR-V 1.6
* Make default environment 1.6 for most tools
* Update tests
* Disallow conditional branch with duplicate labels
* Disallow Dim=Buffer with sampled images
* Do not require the non-semantic extension after SPIR-V 1.5
The pass to remove the nonsemantic information and instructions
is used for drivers or tools that may not support them. Debug
information was only partially handle, which is causing a
problem. We need to either fully remove debug information or
not remove it all. Since I can see it being useful to keep the
debug information even when the nonsemantic instructions are
removed, I propose we do not remove debug info.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4269
In https://github.com/KhronosGroup/SPIRV-Tools/pull/3110, the strip reflect
pass was changed to also remove all explicitly nonsemantic instructions. This
makes it so that the name of the pass no longer reflects what the pass actually
does. This change renames the pass so that it reflects what the pass actaully does.
Use a very large id bound when fuzzing the optimizer, and check that the
input does not ids that are too close to this bound. This should make it
impossible in practice for an id overflow to occur.
Fixes#4657.
* Fix endianness of string literals
To get correct and consistent encoding and decoding of string literals
on big-endian platforms, use spvtools::utils::MakeString and MakeVector
(or wrapper functions) consistently for handling string literals.
- add variant of MakeVector that encodes a string literal into an
existing vector of words
- add variants of MakeString
- add a wrapper spvDecodeLiteralStringOperand in source/
- fix wrapper Operand::AsString to use MakeString (source/opt)
- remove Operand::AsCString as broken and unused
- add a variant of GetOperandAs for string literals (source/val)
... and apply those wrappers throughout the code.
Fixes #149
* Extend round trip test for StringLiterals to flip word order
In the encoding/decoding roundtrip tests for string literals, include
a case that flips byte order in words after encoding and then checks for
successful decoding. That is, on a little-endian host flip to big-endian
byte order and then decode, and vice versa.
* BinaryParseTest.InstructionWithStringOperand: also flip byte order
Test binary parsing of string operands both with the host's and with the
reversed byte order.
This prevents CCP from making constant -> constant transitions when
evaluating instruction values. In this case, FClamp is evaluated twice.
On the first evaluation, if computes FClamp(0.5, 0.5, -1) which returns
-1. On the second evaluation, it computes FClamp(0.5, 0.5, VARYING)
which returns 0.5.
Both fold() computations are correct given the semantics of FClamp() but
this causes a lateral transition in the constant lattice which was not
being considered VARYING by CCP.
Along with OpDecorate, also clone the OpDecorateString instructions for
variables created in the descriptor scalar replacement pass.
Fixesmicrosoft/DirectXShaderCompiler#3705
* https://github.com/KhronosGroup/Vulkan-Docs/issues/666 clearly
specified that interfaces do not require an input if there is an
associated output
* ADCE can now remove unused input variables (though they are kept if
the preserve interfaces option is used)
Fixes#4469
* Checks that decorations only usable with structure members are not
used by OpDecorate or OpDecorateId
* Checks that decorations not allowed on structure members are not used
with OpMemberDecorate
* Checks decoration targets for most core decorations
* Performs some Vulkan specific validation on deorations
* Add wasm build
* Run wasm ci on push
* Add copyright notice to wasm files
* [wasm] Update Emscripten
* [wasm] Change global lambda to regular function
* [wasm] Show detected core count during build
* [wasm] Set JS version from CHANGES, GITHUB_RUN_ID
Also remove custom docker emscripten build with brotli, as not used
* [wasm] Change github actions to use npm-publish
* [wasm] Us docker-compose up for CI
* [wasm] pass GITHUB_RUN_ID to docker
* [wasm] Change GITHUB_RUN_ID to GITHUB_RUN_NUMBER
* [wasm] Fix GITHUB_RUN_NUMBER in docker-compose.yml
If the ids overflow when creating an integer constant in the ir_builder, there will be a nullptr dereference. This is happening from inside merge return.
We need to propagate the error up, and make sure it is handled appropriately.
Consider the new test case. The conditional branch in the continue
block is never marked as live. However, `IsDead` will say it is not
dead, so it does not get deleted. Because it was never marked as live,
`%false` was not mark as live either, but it gets deleted. This results
in invalid code.
To fix this properly, we had to reconsider how branches are handle. We
make the following changes:
1) Terminator instructions that are not branch or OpUnreachable must be
kept, so they are marked as live when initializing the worklist.
2) Branches and OpUnreachable instructions are marked as live if
a) the block does not have a merge instruction and another instruction
in the block is marked as live, or
b) the merge instruction in the same block is marked as live.
3) Any instruction that is not marked as live is removed.
4) If a terminator is to be removed, an OpUnreachable is added. This
happens when the entire block is dead, and the block will be removed.
The OpUnreachable is generated to make sure the block still has a
terminator, and is valid.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4509.
When checking the OpBranchConditional for selection headers,
we intend to register both the true and false targets.
Short circuiting was getting in the way.
Co-authored-by: Steven Perron <stevenperron@google.com>
Spirv-opt has not had to handle module with function declarations. This
lead many passes to assume that every function has a body. This is not
always true. This commit will modify a number of passes to handle
function declarations.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4443
When setting default value for spec constants, for numeric bit types smaller
than 32 bits, follow the SPIR-V rules for narrow literals:
- signed integers are sign-extended
- otherwise, upper bits are zero.
Followup to #4588
* test: add a test to show 8/16-bit
* opt/spec_constants: fix bit pattern width checks.
The input bit patterns are always at least 32-bits, so let the test
pass for 8/16-bit values as well. This shouldn't have any effect on the
64-bit patterns I assume this was introduced for.
The OSS-Fuzz i386 build has been failing due to errors about
64-to-32-bit conversions, relating to random generation code. This
changre fixes the problem by explicitly using a 64-bit random generator,
and by adding a cast to size_t to avoid an implicit conversion.
Do this if Constant or DefUse managers are invalid. Using the
ConstantManager attempts to regenerate the DefUseManager
which is not valid during inlining.
* Account for strided components in arrays
Fixes#4567
* If the element type of an array takes less than a location in size,
calculate the location usage in a strided manner
* formatting
According to spec this opcode is a constant instruction - that's it
can appear outside of function bodies.
Co-authored-by: DmitryBushev <dmitry.bushev@intel.com>
Instead calculate a hash based on the input and use that as a seed
into random data generation for the target env.
Also fixes issue where input data was not actually being fed into
one fuzzer.
Fixes#4450
* Don't eliminate dead members from StructuredBuffer as layout(offset) qualifiers cannot be applied to structure fields.
* Traverse arrays when marking structs as fully used.
Co-authored-by: Steven Perron <stevenperron@google.com>
Debug[No]Line are tracked and optimized using the same mechanism that tracks
and optimizes Op[No]Line.
Also:
- Fix missing DebugScope at top of block.
- Allow scalar replacement of access chain in DebugDeclare
Pending a more general solution for constructing a target environment
based on the bytes of a test input, this change avoids a UBSan error
caused by the existing approach.
Fixes https://crbug.com/38087
* Fix extract with out-of-bounds index
When folding a OpCompositeExtract that is fed by an
OpCompositeConstruct, we handle and out of bounds
index, but only in the case where the result of the
OpCompostiteConstruct is a struct. This change
refactors that folding rule and then improves it to
handle an out-of-bounds access when the result of the
OpCompositeConstruct is a vector.
Includes:
- Shift to use of spirv-header extinst.nonsemantic.shader grammar.json
- Remove extinst.nonsemantic.vulkan.debuginfo.100.grammar.json
- Enable all optimizations for Shader.DebugInfo
Also fixes scalar replacement to only insert DebugValue after all
OpVariables. This is not necessary for OpenCL.DebugInfo, but it is
for Shader.DebugInfo.
Likewise, fixes Private-to-Local to insert DebugDeclare after all
OpVariables.
Also fixes inlining to handle FunctionDefinition which can show up
after first block if early return processing happens.
Co-authored-by: baldurk <baldurk@baldurk.org>
Makes the fuzzer pass and transformation that wraps vector synonyms
aware of the fact that integer operations can have arguments that
differ in signedness, and that the result type of such an operation
can have different sign from the argument types.
Fixes#4413.
In SPIR-V, integers use 2s complement representation, so that signed
integer overflow and underflow is well defined. However, the constant
folder was causing overflow / underflow at the C++ level. This change
avoids such overflows by performing constant folding for IAdd, ISub and
IMul in the context of unsigned values, which works because signedness
is irrelevant according to the SPIR-V semantics for these instructions.
Fixes#4510.
* Fix infinite loop in validation
Fixes https://crbug.com/38548
* Fixes an issue in structured exit checking where an invalid merge
could result in an infinite traversal
* formatting
A test has been removed which depends on casting to spv_target_env from a value
outside the range of that enum. This is an undefined behaviour, thus the
test is invalid.
It is possible that other optimization will propagate
a value into an OpCompositeExtract or OpVectorShuffle
instruction that is larger than the vector size.
Vector DCE has to be able to handle it.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4513.
- The binary exponent must have some decimal digits
- A + or - after the binary exponent digits should not be interpreted as
part of the binary exponent.
Fixes: #4500
With OSS-Fuzz, the build system should not directly set options such as
-fsanitize=fuzzer. Instead, these are set by OSS-Fuzz, and
linker options are provided via the LIB_FUZZER_OPTIONS environment
variable. This change allows the fuzzers to be build stand-alone,
outside of OSS-Fuzz, in the way that was already supported, as well as
inside OSS-Fuzz, when the LIB_FUZZER_OPTIONS environment variable is
set.
ADCE does not handle exported functions. This was an explicit decision
because we did not believe that the linkage attribute could be used in
shaders, but it can now. This change has been made.
While fixing this error, I noticed that the OpName for labels is
sometimes removed because the label instructions are not marked
explicitly marked as live. This has able been fixed.
Currently, handles promotion of divergence due to reconvergence rules, but doesn't handle "late merges" caused by a later-than-necessary declared merge block.
Co-authored-by: Jakub Kuderski <kubak@google.com>
convert-to-sampled-image pass converts images and/or samplers with
given pairs of descriptor set and binding to sampled image.
If a pair of an image and a sampler have the same pair of descriptor
set and binding that is one of the given pairs, they will be
converted to a sampled image. In addition, if only an image has the
descriptor set and binding that is one of the given pairs, it will
be converted to a sampled image as well.
For example, when we have
%a = OpLoad %type_2d_image %texture
%b = OpLoad %type_sampler %sampler
%combined = OpSampledImage %type_sampled_image %a %b
%value = OpImageSampleExplicitLod %v4float %combined ...
1. If %texture and %sampler have the same descriptor set and binding
%combine_texture_and_sampler = OpVaraible %ptr_type_sampled_image_Uniform
...
%combined = OpLoad %type_sampled_image %combine_texture_and_sampler
%value = OpImageSampleExplicitLod %v4float %combined ...
2. If %texture and %sampler have different pairs of descriptor set and binding
%a = OpLoad %type_sampled_image %texture
%extracted_image = OpImage %type_2d_image %a
%b = OpLoad %type_sampler %sampler
%combined = OpSampledImage %type_sampled_image %extracted_image %b
%value = OpImageSampleExplicitLod %v4float %combined ...
* Disallow loading a runtime-sized array
Fixes#4472
* Disallow loading a runtime-sized array or a composite containing one
* Refactor type traversal into a separate function used by both runtime
array checks and sized int/float checks
* Update invalid tests
This PR adds a generic dataflow analysis framework to SPIRV-opt, with the intent of being used in SPIRV-lint. This may also be useful for SPIRV-opt, as existing ad-hoc analyses can be rewritten to use a common framework, but this is not the target of this PR.
This PR adds a new executable spirv-lint with a simple "Hello, world!"
program, along with its associated library and a dummy unit test.
For now, only adds to CMake and Bazel; other build systems will be added
in a future PR.
Issue: #3196
Testing on ClusterFuzz has revealed that the fuzzer sometimes goes
wrong when a shader is very simple - e.g., there have been bugs where
a fuzzer pass has assumed that at least one basic type exists in the
module. This change adds an almost empty SPIR-V example to the shaders
used for testing, to help catch such cases locally.
spirv-fuzz features transformations that should be applicable by
construction. Assertions are used to detect when such transformations
turn out to be inapplicable. Failures of such assertions indicate bugs
in the fuzzer. However, when using the fuzzer at scale (e.g. in
ClusterFuzz) reports of these assertion failures create noise, and
cause the fuzzer to exit early. This change adds an option whereby
inapplicable transformations can be ignored. This reduces noise and
allows fuzzing to continue even when a transformation that should be
applicable but is not has been erroneously created.
Control dependence analysis constructs a control dependence graph,
representing the conditions for a block's execution relative to the
results of other blocks with conditional branches, etc.
This is an analysis pass that will be useful for the linter and
potentially also useful in opt. Currently it is unused except for the
added unit tests.
Adaps the transformations that add OpConstantUndef and OpConstantNull
to a module so that pointer undefs are not allowed, and null pointers
are only allowed if suitable capabilities are present.
Fixes#4357.
Adds a new transformation that rewrites a scalar operation (like
OpFAdd, opISub) as an equivalent vector operation, adding a synonym
between the scalar result and an appropriate component of the vector
result.
Fixes#4195.
This change is responsible for avoiding the replacement of constant
operands with another one not constant, in the context of atomic
operations. The related rule from the SPIR-V spec is: "All used for
Scope and Memory Semantics in shader capability must be of an
OpConstant."
Fixes#4346.
This change allows spriv-reduce to get rid of a selection, switch or
loop construct if none of the instructions defined in the construct
are used outside the construct.
The new pass will removed interface variable on the OpEntryPoint instruction when they are not statically referenced in the call tree of the entry point.
It can be enabled on the command line using the options `remove-unused-interface-variables`.
This change allows the reducer to merge together blocks even when they
are unreachable, but keeps the restriction of reachability in place
for the optimizer.
Fixes#4302.
* Initial support for SPV_KHR_integer_dot_product
- Adds new operand types for packed-vector-format
- Moves ray tracing enums to the end
- PackedVectorFormat is a new optional operand type, so it requires
special handling in grammar table generation.
- Add SPV_KHR_integer_dot_product to optimizer whitelists.
- Pass-through validation: valid cases pass validation
Validation errors are not checked.
- Update SPIRV-Headers
Patch by David Neto <dneto@google.com>
Rebase and minor tweaks by Kevin Petit <kevin.petit@arm.com>
Signed-off-by: David Neto <dneto@google.com>
Signed-off-by: Kevin Petit <kevin.petit@arm.com>
Change-Id: Icb41741cb7f0f1063e5541ce25e5ba6c02266d2c
* format fixes
Change-Id: I35c82ec27bded3d1b62373fa6daec3ffd91105a3
Fixes#4170, by checking the signedness of bitwise operands in
TransformationAddBitInstructionSynonym, to avoid an "Expected Base
Type to be equal to Result Type" validation error.
PR #4118 (d71ac38b8e) let spirv-val report a validation error when we
use offset for an OpImage* instruction instead of ConstantOffset. Since
some compilers like DXC rely on spirv-opt for function inlining or loop
unrolling, the spirv-val change broke some working shaders when the
shader developers disable the optimization (spirv-opt).
For example, DXC recently got this issue from a few users e.g.,
https://github.com/microsoft/DirectXShaderCompiler/issues/3807
Since this error is reported only when the spirv-opt is disabled, it
looks like the exact case that we have to skip spirv-val when
`--before-legalize-hlsl` is given. Moreover, avoiding the error using
`--before-legalize-hlsl` on DXC is exactly what FXC and DXC's DXIL
do (they do not report the error if the offset becomes a constant after
function inlining or loop unrolling).
This change prevents TransformationOutlineFunction from outlining a
region of blocks if some block in the region has an unreachable
predecessor. This avoids a bug whereby the region would be outlined,
and the unreachable predecessors would be left behind, referring to
blocks that are no longer in the function.
The def-use manager was being incorrectly updated in
TransformationPermutePhiOperands, and this was causing future
transformations to go wrong during fuzzing. This change updates the
def-use manager in a correct manner, and adds a test exposing the
previous bug.
Fixes#4300.
The Transformation class tests did not cover the (trivial) ToMessage
methods of each transformation, nor the constructors that take a
protobuf message. This lac of coverage makes it hard to see which more
interesting pieces of code are not covered when looking at coverage
percentages. This change adapts the helper function for applying a
transformation and checking fresh ids so that it turns a
transformation into a protobuf message and back, thus covering
ToMessage and the protobuf constructor for every transformation. The
runtime overhead of doing this is very small.
Fixes https://crbug.com/tint/793
* When a loop has an empty loop construct, the loop construct and
continue construct share the same header so don't disallow the loop
header for the continue construct
Fix dangling phi bug from loop-unroll
When unrolling the following loop:
```
%const0 = OpConstant ...
%const1 = OpConstant ...
...
%LoopHeader = OpLabel
%phi0 = OpPhi %float %const0 %PreHeader %phi1 %Latch
%phi1 = OpPhi %float %const1 %PreHeader %x %Latch
...
%LoopBody = OpLabel
%x = OpFSub %float %phi1 %phi0
...
```
the loop-unroll pass sets the value of `%phi0` as `%phi1` for the second
copy of the loop body. For example, the second copy of
`%x = OpFSub %float %phi1 %phi0` will be
`%y = OpFSub %float %x %phi1`.
Since all phi instructions for inductions will are removed after the
loop unrolling, `%phi1` will be a dead dangling phi.
It happens only for the phi values of the first loop iteration. Replacing those
dangling phis with their initial values fixes this issue.
For example, the second copy of `%x = OpFSub %float %phi1 %phi0` should be
`%y = OpFSub %float %x %const1` because the value of `%phi1` from the
first loop iteration is `%const1`.
This pass converts an internal form of GLSLstd450 Interpolate ops
to the externally valid form. The external form takes the lvalue
of the interpolant. The internal form can do a load of the interpolant.
The pass replaces the load with its pointer. The internal form is
generated by glslang and possibly other frontends for HLSL shaders.
The new pass is called as part of HLSL legalization after all
propagation is complete.
Also adds internal interpolate form to pre-legalization validation
To help ensure that optimizations that do less cautious invalidation
of analyses are implemented correctly, this change adds checks to the
tests of various transformations to ensure that analyses such as
def-use are up to date.
This allows the GPU-AV layer to differentiate between errors with
uniform buffers versus storage buffers and map these to the relevant
VUIDs.
This is a resubmit of a previously reverted commit. The revert was
done as someone erroneously attempted to build the latest validation
layers with a TOT spirv-tools. The validation layers must be built with
their known-good glslang and its known-good spirv-tools and spirv-headers.
* Mark module as modified if convert-to-half removes decorations.
If the convert-to-half pass does not change the body of the function,
but removes decorations, it returns that nothing changed. This is
incorrect, and will be fixed.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4117
* Update comment for RemoveDecorationsFrom
The existing spirv-opt `DebugInfoManager::AddDebugValueForDecl()` sets
the scope and line info of the new added DebugValue using the scope and
line of DebugDeclare. This is wrong because only a single DebugDeclare
must exist under a scope while we have to add DebugValue for all the
places where the variable's value is updated. Therefore, we have to set
the scope and line of DebugValue based on the places of the variable
updates.
This bug makes
https://github.com/google/amber/blob/main/tests/cases/debugger_hlsl_shadowed_vars.amber
fail. This commit fixes the bug.
* Validate SPV_KHR_workgroup_memory_explicit_layout
* Check if SPIR-V is at least 1.4 to use the extension.
* Check if either only Workgroup Blocks or only Workgroup non-Blocks
are used.
* Check that if more than one Workgroup Block is used, variables are
decorated with Aliased.
* Check layout decorations for Workgroup Blocks.
* Implicitly use main capability if the ...8BitAccess or
...16BitAccess are used.
* Allow 8-bit and 16-bit types when ...8BitAccess and ...16BitAccess
are used respectively.
* Update SPIRV-Headers dependency
Bump it to include SPV_KHR_workgroup_memory_explicit_layout.
* Add option to validate Workgroup blocks with scalar layout
Validate the equivalent of scalarBlockLayout for Workgroup storage
class Block variables from SPV_KHR_workgroup_memory_explicit_layout.
Add option to the API and command line tool.
Propagating the OpLine/OpNoLine to preserve the debug information
through transformations results in integrity check failures because of
the extra line instructions. This commit lets spirv-opt skip the
integrity check when the code contains OpLine or OpNoLine.
When there is an array of strutured buffers, desc sroa will only split
the array, but not a struct type in the structured buffer. However,
the calcualtion of the number of binding a struct requires does not take
this into consideration. This commit will fix that.
Avoid generating OpPhi on void types, and allow the transformation to
take place on regions that produce pointer and sampled image result
ids if such ids are not used after the region.
Fixes#3787.
* validation: tighter validation of multisampled images
- if MS=1, then Sample image operand is required
- Sampling operations are not permitted when MS=1
Fixes#4057, #4058
* Fail early for multisampled image for sampling, dref, gather
* validate StorageImageMultisampled capability
The StorageImageMultisampled capability is required when declaring
an image type with with Multisampled==1 and it's a storage image (Sampled == 2)
Fixes#4061
* Allow SubpassData with Multisampled and Sampled==2
