The control barrier instruction was allowed in a limiteted set of shader types.
Part of the HLSL legalization, we use to remove the instructions when it was is
a shader in which it was not allowed. As of spv1.3 that restriction is not long
there.
This change modifies replaced invalid opc to no longer remove it.
Fixes#4999.
spirv_target_env.cpp uses std::pair, which is defined in utility.
Right now, this compiles because utility is provided via transitive
includes in other C++ standard library headers provided by the
implementation. However, these transitive includes are not guaranteed to
exist, and won't exist in certain cases (e.g. compiling against LLVM's
libc++ with modules enabled.)
GetExtractOperandsForElementOfCompositeConstruct() states "Returns the
empty vector if |result_index| is out-of-bounds", but violates that
contract for non-vector result types.
Adding a new type instruction required making the same change in two
places.
Also remove IsCompileTimeConstantInst that was used in a single place
and replace its use by an expression that better conveys the intent.
Change-Id: I49330b74bd34a35db6369c438c053224805c18e0
Signed-off-by: Kevin Petit <kevin.petit@arm.com>
This commit adds a C++ wrapper above the current spvBinaryParse
function. I tried to match it 1:1, except for 2 things:
- std::function<>& are used. No more function pointers, allowing
context capture.
- spv_result_t replaced with a boolean, to match other C++ apis.
Callbacks still return a spv_result_t because the underlying implem
relies on that. The convertion from spv_result_t to boolean is only done
at the boundary.
Signed-off-by: Nathan Gauër <brioche@google.com>
* Update fuzz tests to not use invalid combinations of LoopMerge + BranchConditional
New IDs were selected to make clear the transformation being done here, instead
of reordering all IDs in between or refactoring the SPIR-V in the test.
* spirv-val: Conditional Branch without an exit is invalid in loop header
From 2.16.2, for CFG:
Selections must be structured. That is, an OpSelectionMerge
instruction is required to precede:
- an OpSwitch instruction
- an OpBranchConditional instruction that has different True Label
and False Label operands where neither are declared merge blocks
or Continue Targets.
* Fix null pointer in FoldInsertWithConstants.
Struct types are not supported in constant folding yet.
* Added 'Test case 16' to fold_test.
Tests OpCompositeInsert not to be folded on a struct type.
Contributes to https://github.com/KhronosGroup/glslang/issues/2439
* When OpTypeStruct is used in Vulkan env and its last member
is a RuntimeArray, check if the struct is decorated with
Block or BufferBlock, as required by VUID-...-04680.
-Make more use of InstructionBuilder instruction helper methods
-Use MakeUnique<>() rather than new
-Add InstrumentPass::GenReadFunctionCall() which optimizes function
calls in a loop with constant arguments and no side effects.
This is a prepatory change for future work on the instrumentation
code which will add more generated functions.
The CHANGES file was an alternative source of truth that was trying
to duplicate/replace the git history truth.
This allow us to change the way we handle releases so we don't have to make
sure our CHANGES PR are linked to the tag and tested PR, simplifying the
process.
Kokoro clones repos with a different user used to run the build steps,
meaning if some git command must be run at build time, they will fail
because of this dubious ownership issue.
Running some git commands makes only sense with history, so changing
checkout depth so we can run them and get the true result.
This is a known Kororo issue.
Fixing this is required to generate the version file using git history.
Signed-off-by: Nathan Gauër <brioche@google.com>
Signed-off-by: Nathan Gauër <brioche@google.com>
Kokoro clones repos with a different user used to run the build steps,
meaning if some git command must be run at build time, they will fail
because of this dubious ownership issue.
Running some git commands makes only sense with history, so changing
checkout depth so we can run them and get the true result.
This is a known Kororo issue.
Fixing this is required to generate the version file using git history.
Signed-off-by: Nathan Gauër <brioche@google.com>
Avoid using OpConstantNull with types that do not allow it.
Update existing tests for slight changes in code generation.
Add new tests based on the Vulkan Validation layer test case
that exposed this problem.
* Validate version 5 of clspv reflection
* Add validation for instructions in versions 2 through 5
* Change the instruction reported for remaining indices on an access
chain to the access chain itself for clarity
* update spirv-headers
* Update minimum required CMake to 3.17.2
- For Wasm build, update to emscripten/emsdk:3.1.28 which has 3.22.1
- Move the docker-compose.yml down to the source/wasm directory.
Fixes: #5040
* Fix working directory for invocation of wasm build
When the parser saw more significant hex digits than fit in
the target type, it would compute a nonsensical shift amount, resulting
in undefined behaviour.
Now, drop the excess bits, effectively truncating the significand.
Also guard against overflow of the exponent in the extraordinary (and untested)
case where we see more than, for example, 2**(32-4+1) significant hex digits
for a 32-bit float, or 2**(16-4+1) significant hex digits for a 16-bit
float.
Also guard against overflow of the indexing counting the number of
significant bits. When that would occur silently drop any further
significant bits. (Untested)
Avoid hex floats in C++ code. It's a C++17 feature.
Fixes: #4724
* Fix layout validation
Fixes#5010
* Unless scalar block layout is enabled, no member can reside at an
offset between the end of the previous member that is a struct or
array and the next multiple of that alignment
* Remove a dead if that was introduced
Co-authored-by: David Neto <dneto@google.com>
This can cause interface incompatibility and should only be done
if ADCE has been applied to the following shader in the pipeline.
For this reason this capability is not available through the CLI
but rather only non-default through the API. This functionality is
intended as part of a larger cross-shader dead code elimination
sequence.
Constexpr guaranteed no runtime init in addition to const semantics.
Moving all opt/ to constexpr.
Moving all compile-unit statics to anonymous namespaces to uniformize
the method used (anonymous namespace vs static has the same behavior
here AFAIK).
Signed-off-by: Nathan Gauër <brioche@google.com>
Safe version will only optimize vertex shaders. All other shaders will
succeed without change.
Change --eliminate-dead-input-components to use new safe version.
Unsafe version (allowing non-vertex shaders) currently only available
through API. Should only be used in combination with other optimizations
to keep interfaces consistent. See optimizer.hpp for more details.
Add a flags field at the first offset within this buffer.
Define flags to allow buffer OOB checking to be enabled or
disabled at run time. This is to support VK_EXT_pipeline_robustnes.
This pass eliminates components of output variables that are not stored
to. Currently this just eliminates trailing components of arrays and
structs, all of which are dead.
WARNING: This pass is not designed to be a standalone pass as it can
cause interface incompatibiliies with the following shader in the
pipeline. See the comment in optimizer.hpp for best usage. This pass is
currently available only through the API; it is not available in the CLI.
This commit also fixes a bug in CreateDecoration() which is part of the
system of generating SPIR-V from the Type manager.
This adds two passes to accomplish this: one pass to analyze a shader
to determine the input slots that are live. The second pass is run on
the preceding shader to eliminate any stores to output slots that are
not consumed by the following shader.
These passes support vert, tesc, tese, geom, and frag shaders.
These passes are currently only available through the API.
These passes together with dead code elimination, and elimination of
dead input and output components and variables (WIP), will allow users
to do dead code elimination across shader boundaries.
Fixes#4918
* Prevent block merging from producing an invalid case construct by
merging a switch target/default with another construct's merge or
continue block
* This is to satisfy the structural dominance requirement between the
switch header and the case constructs
Fixes#4671
Fixes https://crbug.com/oss-fuzz/43265
* Only validate full layout in a vulkan environment
* Universal validation still checks that the right decorations are
present, but the values are only considered for vulkan
* One exception is that invalid overlaps are only checked for vulkan
* This is a pragmatic choice as SPIR-V doesn't define the size of
types so the amount of universal checking would be quite limited
* Removed redundant check for GLSLShared and GLSLPacked decorations
* Should never have been validated as part of universal validation
* make conditionals independent
Fixes https://crbug.com/oss-fuzz/48553
* Assign a reflexive dominator if no other dominator can be found using
forward traversals
* This prevents a null dereference of a pointer in the sorting of the
output
* Support Narrow Types in BitCast Folding Rule
This change adds support for narrow types in the BitCastScalarOrVector
folding rule. According to Section 2.2.1 of the SPIR-V spec, types that
are narrower than 32 bits are automatically either sign extended, or
zero extended depending on the type. With that guaranteed, we should
be able to use the first 32-bit word of any narrow type for the folding
logic without performing any special conversions.
In order to reduce code duplication, this change moves the
GetU32BitValue and GetU64BitValue functions from IntConstant to
ScalarConstant. Without this move, we would have needed an identical
version of GetU32BitValue on FloatConstant.
* Add Tests for 16-bit BitCast Folding
This change adds several new test cases to the
IntegerInstructionFoldingTest which trigger the 16-bit BitCast logic.
The logic for half types was also added to the integer case since we
can't easily validate half float types in C++ code. It's easier to
validate them as unsigned integers instead. Pllus this also allows us
to verify the SPIR-V constant sign extension logic too.
* Add 8-Bit Folding Test Cases
This change adds a couple more test cases to the integer instruction
folding test suite in order to ensure that the BitCast logic also
works correctly with the Int8 shader capability.
The always-friendly messages make it harder to debug when the
disassembly is later generated without friendly names.
Additionally, the friendly-name-mapper is slow. Disabling it improves
performance of an ANGLE test that creates numerous shaders by ~5%.
Half the messages used to output 'id[%name]' and half id[%name]. With
this change, all messages consistently output 'id[%name]'. Some typos
are also fixed in the process.
This was spotted in the Validation Layers where OpSpecConstantOp %x CompositeExtract %y 0 was being folded to a constant, but anything that was using it wasn't recognizing it as a constant, the simple fix was to add a const_mgr->MapInst(new_const_inst); so the next instruction knew it was a const
* Remove `spvOpcodeTerminatesExecution`
This function is the same as `spvOpcodeIsAbort` except for
OpUnreachable. The names are so close in meaning that it is hard to
distinguish them. I've removed `spvOpcodeTerminatesExecution` since it
is used in only a single place. I've special cased OpUnreachable in
that location.
At the same time, I fixed up some comments related to the use of the
TerminatesExecution and IsAbort functions.
Following up on #4930.
* Fix comments
Removed now unused DebugDeclare visibility logic for generating
DebugValue.
Also eliminated the phi sort introduced in 272e4b3. This should have
been removed in the first commit.
Changed a couple small parts of the algorithm to reduce time to build
the dominator trees. There should be no visible changes.
Add a depth first search algorithm that does not run a function on
backedges. The check if an edge is a back edge is time consuming, and
pointless if the function run on it is a nop.
Add name annotations to the generated instrumentation code to
make it easier to understand. Example spirv-cross output:
vec4 _140;
if (0u < inst_bindless_direct_read_4(0u, 0u, 1u, uint(_19)))
{
_140 = texture(textures[nonuniformEXT(_19)], inUV);
}
else
{
inst_bindless_stream_write_4(50u, 1u, uint(_19), 0u);
_140 = vec4(0.0);
}
* Improve algorithm to reorder blocks in a function
In dead branch elimination, blocks can end up in a the wrong order, so
there is code to reorder the blocks in structured order. The problem is
that the algorithm to do that is very poor. It involves many searchs in
the function for the correct position to place the block, as well as
moving many block in the vector.
The solution is to write a specialized function in the function class
that will reorder the blocks in structured order. After computing the
structured order, reordering the block can be done in linear time, with
very little overhead.
* spirv-opt: fix copy-propagate-arrays index opti on structs.
As per SPIR-V spec:
OpAccessChain indices must be OpConstant when indexing into a structure.
This optimization tried to remove load cascade. But in some scenario
failed:
```c
cbuffer MyStruct {
uint my_field;
};
uint main(uint index) {
const uint my_array[1] = { my_field };
return my_array[index]
}
```
This is valid as the struct is indexed with a constant index, and then
the array is indexed using a dynamic index.
The optimization would consider the local array to be useless and
generated a load directly into the struct.
* spirv-opt: prevent creation of unused instructions
Copy-propagate-arrays optimization pass would create unused constants,
even if the optimization not completed.
This was caused by the way we handled OpAccessChain squashing: we
only referenced constants, and had to create them upfront.
Fixes#4887
Signed-off-by: Nathan Gauër <brioche@google.com>
Specifically, DebugSourceContinued, DebugCompilationUnit, and
DebugEntryPoint. These instructions are top-level instructions
which do not or may not have a user except for the tool and so
should not be eliminated.
When folding a vector shuffle with an undef literal, it is possible that the
literal is adjusted so that it will then be interpreted as an index into
the input operands. This is fixed by special casing that case, and not
adjusting those operands.
Fixes#4859
Disassembler was called with non-default params, loosing FRIENDLY_NAMES.
This commit changes the call options to allow the spirv-opt to show
friendly names instead of raw-ids. Might be more helpful when reading
the SPIRV-opt output.
Fixes#4882
Signed-off-by: Nathan Gauër <brioche@google.com>
An access chain instruction interpretes its index operands as signed.
The composite insert and extract instruction interpret their index
operands as unsigned, so it is not possible to represent a negative
number.
This commit adds a check to the local-access-chain-convert pass to check
for a negative number in the access chain and to not do the conversion.
Fixes#4856
These asserts are not valid for string literals, which may
contain several words:
assert(a_operand.words.size() == 1);
assert(b_operand.words.size() == 1);
It looks like they only make sense for the default case, which
assumes that both operands contain a single word.
Running a debug version of spirv-diff on any shader containing
a string literal will hit the original asserts.
A recent libc++ roll in Chrome warned of a deprecated copy. We're
still looking if this is a bug in libc++ or a valid warning, but
removing the redundant line is a safe workaround or fix in either
case.
See discussion in https://crrev.com/c/3791771
* spirv-as: Avoid overflow when parsing exponents on hex floats
When an exponent is so large that it would overflow the int
type in the parser, saturate the exponent.
This allows extremely large exponents, and saturates
to infinity when the exponent is positive, and zero when the exponent
is negative.
Fixes#4721.
* Avoid unexpected narrowing conversions from arithmetic operations
Co-authored-by: Alastair F. Donaldson <alastair.donaldson@imperial.ac.uk>
Excessive whitespace can lead to stack overflow during parsing as each
character of skipped whitespace involves a recursive call. An
iterative solution avoids this.
Fixes#4729.
This reverts commit d18d0d92e5.
This is reverted because it causes a 7X slowdown when legalizing
SPIR-V with NonSemantic.Shader.DebugInfo.100 instructions.
This is due to the creation of very large UseLists for several
heavily used operands for this extension combined with the fact
that the original commit changed the performance of Uselists to O(n).
Fix code that is traversing def-use user structure at the same time
that it is changing it. This is dicey at best and error prone at worst.
This was uncovered making a change to the id_to_user representation.
Fixes https://crbug.com/oss-fuzz48578
* Adds structural reachability to basic blocks
* calculated in same manner as reachability, but using structural
successors
* Change structured CFG validation to use structural reachability
instead of reachability
* Fix some invalid reducer cases
The fuzzer cretes code with very large array, and scalar replacement
times out. Adding a limit on the size of the composites that will be
split when fuzzing.
Fixes https://crbug.com/oss-fuzz/48630
Arrays do not have to have a size that is known at compile time. It
could be a spec constant. In these cases, treat the array
as if it is arbitrarily long. This commit will treat it like it is an
array of size UINT32_MAX.
Fixes https://crbug.com/oss-fuzz/47397.
If the `instruction` operand in an extended instruction instruction is
too large, it causes undefined behaviour when that value is cast to the
enum for the corresponding set. This is done with the
NonSemanticDebug100 instruction set. We need to avoid the undefined
behaviour.
Fixes#4727
Which functions are processed is determined by which ones are on the
call tree from the entry points before dead code is removed. So it is
possible that a function is process because it is called from an entry
point, but the CFG is not cleaned up because the call to the function
was removed.
The fix is to process and cleanup every function in the module. Since
all of the dead functions would have already been removed in an earlier
step of DCE, it should not make a different in compile time.
Fixes#4731
* ANGLE builds with -Werror,-Wunreachable-code-loop-increment which had
a problem with grabbing the first construct in a loop
* Change the code to just use begin instead
* Structural dominance introduced in SPIR-V 1.6 rev2
* Changes the structured cfg validation to use structural dominance
* structural dominance is based on a cfg where merge and continue
declarations are counted as graph edges
* Basic blocks now track structural predecessors and structural
successors
* Add validation for entry into a loop
* Fixed an issue with inlining a single block loop
* The continue target needs to be moved to the latch block
* Simplify the calculation of structured exits
* no longer requires block depth
* Update many invalid tests
When the `SPV_BINARY_TO_TEXT_OPTION_PRINT` option is specified, `spvtext` will not be created (see 37d2396cab/source/disassemble.cpp (L117)), and the attempt to dereference into its members (`spvtext->str` and `spvtext->length`) results in segmentation fault.
This is fixed by first checking if `spvtext` is nulll.
Co-authored-by: Steven Perron <stevenperron@google.com>
We currently build the structured order for all nodes reachable from the
loop header when unrolling a loop. However, unrolling only needs the
nodes in the loop and possibly the merge node.
To avoid needless computation, I have implemented a search that will
stop at the merge node.
Fixes#4827
Uses a preprocessor macro to bail out of unrolling loops with large
iteration counts during fuzzing, to reduce the number of
timeouts/memouts that arise.
Related issue: #4728.
The code in `CFG::SplitLoopHeader` assumes the loop header is not the
latch. This leads to it not being able to find the latch block. This
has been fixed, and a test added.
Fixes#4527
If the predecessor blocks are the same, then there is only 1 value for the
OpPhi. The simplition pass will simplify it, and it causes problems for
if-conversion. In these cases, if-conversion can just punt.
Fixes#3554.
There is an assert that verifies that the binary did not change when the
optimizer said that it did not. However, if the input binary is in big endian
format, the optimizer will encode the optimized binary in little endian. This
causes the assert to fail. Since we do not believe that anybody cares about a
big endien formate, we will disable the assert in that case.
Fixes#4722
The Reciprocal function expects a divide-by-0 to return nan,and then Reciprocal will return 0.
Since the divide-by-0 is actually undefined, we will identify this case early, and return 0.
No new tests are needed because we already tests folding divide-by-0.
Fixes#4715
An access chain could have a constant index that is an out of bounds
access. This is valid spir-v, even if it can cause problems at runtime.
However, it is not valid to have an OpCompositeExtract with an out of
bounds access. This means we have to stop local-access-chain-convert
from making that change.
Fixes#4605
A std::set is used instead of std::vector, where the elements are
ordered by member index first. Decorations for fields are now looked up
by going over the range of decorations for the member only, instead of
the whole set.
In an ANGLE test that generates a struct with 4096 members, validation
goes down from ~140ms to ~90ms. On debug builds, the difference is more
pronounced, going down from ~2.5s to ~600ms.
This change adds a folding rule which transforms x * y - a and a - x * y
into FMA(x, y, -a) and FMA(-x, y, a), respectively.
While the SPIR-V instruction count remains the same, target instruction
sets typically feature FMA instruction variants that can negate an
operand. Also this transformation may unlock further optimizations which
eliminate the negation.
(Google bug: b/226145988)
* Add move folding for composite instructions
Fold chains of insert into construct
If a chain of OpCompositeInsert instruction write to every element of a
composite object, then we can replace it with an OpCompositeConstruct.
Fold a construct fed by extracts to a single extract
We already fold an OpCompositeConstruct when it is simlpy reconstructing
an object that was decomposed by a series of OpCompositeExtract
instructions. However, we do not do that if that object is an element
of a larger object.
I have updated the rule, so that if the original object is a an element
of a larger object, then the OpCompositeConstruct is replaced with a
single OpCompositeExtract from the larger object.
Fixes#4371.
* Handle 64-bit integers in local access chain convert
The local access chain convert pass does on run on module that have
64-bit integers, even if they have nothing to to with access chains.
This is very limiting because other passes rely on the access chains
being removed. So this commit will add this functionality to the pass.
spirv validation require OpFunctionCall with memory object, usually this
is non issue as all the functions are inlined.
This pass deal with some case for
DontInline function. accesschain input operand would be replaced new
created variable
- Add assembler/disassembler support
- Add validator support
Signed-off-by: Kevin Petit <kevin.petit@arm.com>
Change-Id: Iffcedd5d5e636a0e128a5906ffe634dd85727de1
This reverts commit 671f6e633f.
PR #4783 was reverted because it caused OpenCL CTS failures for clvk.
The was in clspv, which was not adding the no contract decoration when
it was required. This has been fixed in
https://github.com/google/clspv/pull/845. We can now reapply #4783.
* linker: Recalculate interface variables
By resolving extern symbols Entry Points might access variables they
hadn't declared before.
* test/linker: add test to verify linked spir-vs importing functions validate
Without the fix Validate will complain about:
"ERROR: 9: Interface variable id <5> is used by entry point 'bar' id <1>, but is not listed as an interface\n %5 = OpVariable %_ptr_Input_v3uint Input\n"
Adding Fma instruction can speed up the code. This was requested by
swiftshader, so they do not have to do this analysis themselves. It can
also help reduce the code size, and the work the ICD compilers have to
do.
In the installed file
/usr/lib64/cmake/SPIRV-Tools/SPIRV-ToolsTarget.cmake, occurences of
librt in the INTERFACE_LINK_LIBRARIES property are incorrect. The
property contains the absolute path to librt. In most situations, this
produces no problem. But when building in a sysroot, which is commonly
done when cross-compiling, the absolute path breaks dependent projects.
For example, when building spirv-tools using the Chrome OS SDK, and
targeting the board 'volteer', where the build sysroot is
'/build/volteer', the file includes this line
INTERFACE_LINK_LIBRARIES "/build/volteer/usr/lib64/librt.so"
when it should instead say
INTERFACE_LINK_LIBRARIES "rt"
The CMake documentation agrees [1]:
Note that it is not advisable to populate the
INTERFACE_LINK_LIBRARIES of a target with absolute paths to
dependencies. That would hard-code into installed packages the
library file paths for dependencies as found on the machine the
package was made on.
[1] https://cmake.org/cmake/help/latest/prop_tgt/INTERFACE_LINK_LIBRARIES.html
* Fix gen_build_version on Windows
The paths used in the gen_build_version script are causing some failures
in downstream builds. Switch to using the location of the CHANGES file
rather than the ".." parent directory workaround.
* Update other build files
The code accidentally expected OpTypeFunction operand count to match.
This is fixed so that OpTypeFunction instructions with different operand
counts are considered not matching.
Currently, the diff tool matches types bottom up, so on every
instruction it expects to know if its operands are already matched or
not. With cyclical references, it cannot know that. Type matching
would need significant rework to be able to support such a use case; for
example, it may need to maintain a set of plausable matches between type
pointers that are forward-referenced, and potentially back track when
later the types turn out to be incompatible.
In this change, OpTypeForwardPointer is supported in the more common and
trivial case. Firstly, forwarded type pointers are only matched if they
have they have the same storage class and point to the same type opcode:
- In the presence of debug info, matching is done only if the names are
unique in both src and dst.
- In the absence of debug info, matching is done only if there is only
one possible matching.
Fixes: #4754
spread-volatile-semantics pass spreads Volatile semantics for builtin
variables used by certain execution models based on
VUID-StandaloneSpirv-VulkanMemoryModel-04678 and
VUID-StandaloneSpirv-VulkanMemoryModel-04679 (See "Standalone SPIR-V
Validation" section of Vulkan spec "Appendix A: Vulkan Environment for
SPIR-V"). Therefore, shaders without execution model (e.g., used only
for linkage) are not the target of the pass. This commit lets the pass
just return SuccessWithoutChange in that case.
In preparation for supporting OpTypeForwardPointer, which adds more
usages like this. This change refactors common code used to group
instructions and match the groups.
Clang added -Wunqualified-std-cast-call in
https://reviews.llvm.org/D119670, which warns on unqualified std::move
and std::forward calls. This change qualifies these calls to allow the
project to build on HEAD Clang -Werror.
If the body of the module does not have any ids change, compact ids will
not change the id bound. This can cause problems because the id bound
could be much higher than the largest id in that is used. It should be
reset any time it is not the larger id used + 1.
Fixes#4604
When folding a vector shuffle feeding a vector shuffle, we do not
propagate an 0xFFFFFFFF, which has a special meaning, correctly. We
adjust the value making it lose it meaning as an undefined value.
Fixes#4581
CCP does not want to fold an instruction unless it folds to a constant.
There is an asser to check for this. The question if a spec constant
counts as a constant. The constant folder considers a spec constant a
constand, but CCP does not. I've fixed the assert in CCP to match what
the folder does. It should not require any new changes to CCP.
Swift shader needs a way to inline all functions, even those marked as
DontInline. See https://github.com/KhronosGroup/SPIRV-Tools/pull/4471.
This implements the suggestion I made in the PR. We add a pass that
will remove the DontInline function control, so that the inlining passes
will inline them.
SwiftShader will still have to modify their code to add this pass before
the other passes are run.
The function `BuildInvalideAnalyses` will be rebuilt for every analysis that
has been requested, but it is not necessary. It also can cause problems
because if the CFG needs to be rebuilt, so do the dominator trees.
This change will make the functionality match the description of the
function.
* Optimize DefUseManager allocations
Saves around 30-35% of compilation time.
For inst->use_ids, use a pool linked list instead of allocating vectors for every instruction. For inst->uses, use a "PooledLinkedList"' -- a linked list that has shared storage for all nodes. Neither re-use nodes, instead we do a bulk compaction operation when too much memory is being wasted (tuneable).
Includes separate PooledLinkedList templated datastructure, a very special case construct, but split out to make the code a little easier to understand.
Incrementally compute the hash instead of collecting words
Avoids allocating temporary space in a std::vector and std::u32string, and making three passes over all the hashed data.
Switch to using std::vector to prevent processing duplicates instead of std::unordered_set: avoids an allocation/deletion every call to ComputeHashValue, and ends up faster due to much better cache behaviour and smaller constant-factor when searching the (generally very small) list.
In my test case, made Type::HashValue go from 7.5% of compilation time to .5%
Previously, array sizes were presumed to be OpConstant, which is not
necessarily true. This change ensures OpSpecConstant array sizes as
matched exactly, instead of taken as OpConstant and matched by value.
* Reimplement LCS used by spirv-diff
Two improvements are made to the LCS algorithm:
- The LCS algorithm is reimplemented to use a std::stack instead of
being recursive. This prevents stack overflow in the LCSTest.Large
test.
- The LCS algorithm uses an NxM table. Previously, entries of this
table were {size_t, bool, bool}, which is now packed in 32 bits. The
first entry can assume a maximum value of min(N, M), which
realistically for SPIR-V diff will not be larger than 1 billion
instructions. This reduces memory usage of LCS by 75%.
This partially reverts 845f3efb8a and
enables LCS tests.
* Stabilize the output of spirv-diff
std::map is used instead of std::unordered_map to ensure the output of
spirv-diff is identical everywhere.
This partially reverts 845f3efb8a and
enables spirv-diff tests.
Scalar replacement generates a null when there value for a member will
not be used. The null is used to make sure things are
deterministic in case there is an error.
However, some type cannot be null, so we will change that to use undef.
To keep the code simpler we will always use the undef.
Fixes#3996
spirv-diff is a new tool that produces diff-style output comparing two
SPIR-V modules. The instructions between the src and dst modules are
matched as best as the tool can, and output is produced (in src
id-space) that shows which instructions are removed in src, added in dst
or modified between them. The order of instructions are not retained.
Matching instructions between two SPIR-V modules is not trivial, and
thus a number of heuristics are applied in this tool. In particular,
without debug information, it's hard to match functions as they can be
reordered. As such, this tool is primarily useful to produce the diff
of two SPIR-V modules derived from the same source.
This tool can be useful in a number of scenarios:
- Compare the SPIR-V before and after modifying a shader
- Compare the SPIR-V produced from a shader before and after compiler
codegen changes.
- Compare the SPIR-V produced from a shader before and after some
transformation or optimization.
- Compare the SPIR-V produced from a shader with different compilers.
The handling of the RayQueryKHR type is not complete in the type
manager. The tests were not picking this up. I've added a test to make
sure that the `GenerateAllTypes` function actually does generate all of
the types. Once it is added there other tests should pick up on the
other parts that were missing.
Add a pass to spread Volatile semantics to variables with SMIDNV,
WarpIDNV, SubgroupSize, SubgroupLocalInvocationId, SubgroupEqMask,
SubgroupGeMask, SubgroupGtMask, SubgroupLeMask, or SubgroupLtMask BuiltIn
decorations or OpLoad for them when the shader model is the ray
generation, closest hit, miss, intersection, or callable shaders. This
pass can be used for VUID-StandaloneSpirv-VulkanMemoryModel-04678 and
VUID-StandaloneSpirv-VulkanMemoryModel-04679 (See "Standalone SPIR-V
Validation" section of Vulkan spec "Appendix A: Vulkan Environment for
SPIR-V").
Handle variables used by multiple entry points:
1. Update error check to make it working regardless of the order of
entry points.
2. For a variable, if it is used by two entry points E1 and E2 and
it needs the Volatile semantics for E1 while it does not for E2
- If VulkanMemoryModel capability is enabled, which means we have to
set memory operation of load instructions for the variable, we
update load instructions in E1, but do not update the ones in E2.
- If VulkanMemoryModel capability is disabled, which means we have
to add Volatile decoration for the variable, we report an error
because E1 needs to add Volatile decoration for the variable while
E2 does not.
For the simplicity of the implementation, we assume that all functions
other than entry point functions are inlined.
C++20 automatically adds reversed versions of operator overloads for
consideration; in this particular instance this results in infinite
recursion, which has now been pointed out elsewhere as a known issue
when migrating to C++20. Here we just disable one of the overloads in
C++20 mode and let the auto-reversing take care of it for us.
* linker: Address conversion error introduced in earlier rework
Also rework the code slightly to first compute the new ID bound and
validate it, and only then, offset all existing IDs.
This makes those two steps clearer, and avoids potentially overflowing
the IDs within a module (even if that would have been caught later on).
Fixes: c3849565 ("Linker improvements (#4679)")
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4684
* test/linker: Disable IdsLimit tests for now
They are taking too long to run and results in the CI jobs timing out.
* test/linker: Code factorisation and small tweaks
* linker: Do not fail when going over limits
The limits are minima and implementations or APIs might support higher
limits, so just warn the user about it. And only check for the limits
right before emitting the binary, as limits might change earlier when
removing duplicate instructions, function prototypes, etc.
The only check performed right before merging, is making sure the ID
bound will not overflow the 32 bits following the merge.
Also, use the defines for the limits instead of hard-coding them.
* linker: Require a memory model in each input module
The existing code could run into weird situation. For example, if the
first module had no memory model, it would not emit any memory model
(sort of reasonable) and would accept without complains all possible mix
from later modules as it would not verify them.
* linker: Replace hex version with SPV_SPIRV_VERSION_WORD
* linker: Error out when linking together different versions
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4135
* tools/linker: Do not write to disk if linking failed
Also, do not consider warnings as errors.
* tools/linker: Fix formatting in help message
* tools/linker: Further clarify the use of --target-env
Also update the text for the default version to reflect the change made
in 7d768812 ("Basic support for SPIR-V 1.6 (#4663)").
* PassManager: Print errors occurring during disassembly
Otherwise one could be greeted by the following text when running
spirv-opt withe the `--print-all` flag:
; IR before pass wrap-opkill
; IR before pass eliminate-dead-branches
; IR before pass merge-return
With this commit, one will instead get:
error: line 143: Invalid opcode: 400
warning: line 0: Disassembly failed before pass wrap-opkill
error: line 143: Invalid opcode: 400
warning: line 0: Disassembly failed before pass eliminate-dead-branches
error: line 143: Invalid opcode: 400
warning: line 0: Disassembly failed before pass merge-return
* PassManager: Use the right target environment when disassembling
Disassembly would fail if features from a newer version of SPIR-V than
1.2 were used.
The comment in `Array::GetExtraHashWords` is misleading because getting
the hash words is split up between the generic `Type::GetHashWords` and
the type specific `Type::GetExtraHashWords`. While `IsSameImpl` is
self-contained. Removing the comment since it is misleading and no
comment is really needed.
Fixes#3248
Ensures that instruction's opcode is set to something default when
parsing the module with --preserve-numeric-ids enabled. This avoids
uninitialized accesses and knock-on buffer overflows.
Fixes#4672.
For a shader input/output interface variable of structure type:
If the structure has BuiltIn members, then the structure type
must be Block decorated.
Otherwise, the variable, or the struct members must have locations,
but nothing can have both a location be a BuiltIn.
Implements validation needed to reject the example in
https://github.com/KhronosGroup/SPIRV-Registry/issues/134
* Basic support for SPIR-V 1.6
* Update SPIRV-Headers deps
* Add new environment enum for SPIR-V 1.6
* Make default environment 1.6 for most tools
* Update tests
* Disallow conditional branch with duplicate labels
* Disallow Dim=Buffer with sampled images
* Do not require the non-semantic extension after SPIR-V 1.5
The pass to remove the nonsemantic information and instructions
is used for drivers or tools that may not support them. Debug
information was only partially handle, which is causing a
problem. We need to either fully remove debug information or
not remove it all. Since I can see it being useful to keep the
debug information even when the nonsemantic instructions are
removed, I propose we do not remove debug info.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4269
In https://github.com/KhronosGroup/SPIRV-Tools/pull/3110, the strip reflect
pass was changed to also remove all explicitly nonsemantic instructions. This
makes it so that the name of the pass no longer reflects what the pass actually
does. This change renames the pass so that it reflects what the pass actaully does.
The upcoming spirv-diff tool also outputs disassembly, although in a
per-instruction basis. This change refactors the disassembler code to
support such a use case.
The change in
commit 4ac8e5e541
Author: Greg Fischer <greg@lunarg.com>
Date: Wed Sep 15 12:38:34 2021 -0600
Add preserve_interface mode to aggressive_dead_code_elim (#4520)
Broke the C++ ABI for spirv-tools shared libraries on Linux, for not a great reason.
Restore the previous ABI.
* Fix endianness of string literals
To get correct and consistent encoding and decoding of string literals
on big-endian platforms, use spvtools::utils::MakeString and MakeVector
(or wrapper functions) consistently for handling string literals.
- add variant of MakeVector that encodes a string literal into an
existing vector of words
- add variants of MakeString
- add a wrapper spvDecodeLiteralStringOperand in source/
- fix wrapper Operand::AsString to use MakeString (source/opt)
- remove Operand::AsCString as broken and unused
- add a variant of GetOperandAs for string literals (source/val)
... and apply those wrappers throughout the code.
Fixes #149
* Extend round trip test for StringLiterals to flip word order
In the encoding/decoding roundtrip tests for string literals, include
a case that flips byte order in words after encoding and then checks for
successful decoding. That is, on a little-endian host flip to big-endian
byte order and then decode, and vice versa.
* BinaryParseTest.InstructionWithStringOperand: also flip byte order
Test binary parsing of string operands both with the host's and with the
reversed byte order.
Ensures that when an attempt to read a floating-point value from an
input stream fails, the recieving variable for the read does not end up
uninitialised.
Fixes OSS-Fuzz:40432
Fixes#4644
Currently if an ID overflow occurs, spirv-opt (and other users of
IRContext) emits a warning and starts returning 0 when fresh ids are
requested. This tends to lead to crashes - such as null pointer
exceptions. When these arise during fuzzing they lead to auto-reported
bugs.
This change uses an ifdef guard to instead gracefully exit as soon as an
ID overflow occurs when the build is a fuzzing build.
Related issue: #4539.
This prevents CCP from making constant -> constant transitions when
evaluating instruction values. In this case, FClamp is evaluated twice.
On the first evaluation, if computes FClamp(0.5, 0.5, -1) which returns
-1. On the second evaluation, it computes FClamp(0.5, 0.5, VARYING)
which returns 0.5.
Both fold() computations are correct given the semantics of FClamp() but
this causes a lateral transition in the constant lattice which was not
being considered VARYING by CCP.
Along with OpDecorate, also clone the OpDecorateString instructions for
variables created in the descriptor scalar replacement pass.
Fixesmicrosoft/DirectXShaderCompiler#3705
* https://github.com/KhronosGroup/Vulkan-Docs/issues/666 clearly
specified that interfaces do not require an input if there is an
associated output
* ADCE can now remove unused input variables (though they are kept if
the preserve interfaces option is used)
Fixes#4469
* Checks that decorations only usable with structure members are not
used by OpDecorate or OpDecorateId
* Checks that decorations not allowed on structure members are not used
with OpMemberDecorate
* Checks decoration targets for most core decorations
* Performs some Vulkan specific validation on deorations
* Add wasm build
* Run wasm ci on push
* Add copyright notice to wasm files
* [wasm] Update Emscripten
* [wasm] Change global lambda to regular function
* [wasm] Show detected core count during build
* [wasm] Set JS version from CHANGES, GITHUB_RUN_ID
Also remove custom docker emscripten build with brotli, as not used
* [wasm] Change github actions to use npm-publish
* [wasm] Us docker-compose up for CI
* [wasm] pass GITHUB_RUN_ID to docker
* [wasm] Change GITHUB_RUN_ID to GITHUB_RUN_NUMBER
* [wasm] Fix GITHUB_RUN_NUMBER in docker-compose.yml
If the ids overflow when creating an integer constant in the ir_builder, there will be a nullptr dereference. This is happening from inside merge return.
We need to propagate the error up, and make sure it is handled appropriately.
In #3404 a logical && was replaced with a bitwise & to ensure that
both side-effecting arguments were evaluated. However, this leads to
warnings from some compilers (leading to the OSS-Fuzz build breaking,
in particular).
This change reworks the relevant code so that both arguments to the
logical && are evaluated into temporaries.
