The assembler assigns ID numbers sequentially, so it's confusing
to have a %1 in the source assembly when it isn't the first mentioned
ID. Rewrite the ID names to avoid this situation in a few cases.
From the SPIR-V Spec 2.16.1:
A function declaration (an OpFunction with no basic blocks), must have
a Linkage Attributes Decoration with the Import Linkage Type.
A function definition (an OpFunction with basic blocks) cannot be
decorated with the Import Linkage Type.
The limit for the number of struct members is parameterized using
command line options.
Add --max-struct-depth command line option.
Add --max-switch-branches command line option.
Add --max-function-args command line option.
Add --max-control-flow-nesting-depth option.
Add --max-access-chain-indexes option.
If a merge block is reachable, then it must be *strictly* dominated
by its header. Until now we've allowed the header and the merge
block to be the same.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/551
Also: Use dominates and postdominates methods on BasicBlock to
improve readability.
It is best to check the error messages of unit tests that fail
validation. This will ensure that a validation failure is due to what we
expect and not due to some secondary reason.
Updating SPIR-V Validator unit tests with error message checks.
When applied to a structure-type member, all members of that structure
type must also be decorated with BuiltIn. (No allowed mixing of built-in
variables and non-built-in variables within a single structure.)
When applied to a structure-type member, that structure type cannot be
contained as a member of another structure type.
There is at most one object per Storage Class that can contain a
structure type containing members decorated with BuiltIn, consumed per
entry-point.
It is acceptable for OpAccessChain, OpInBoundsAccessChain,
OpPtrAccessChain, OpInBoundsPtrAccessChain, OpCompositeInsert, and
OpCompositeExtract to not take any indexes as arguments. In such cases,
no indexing will be done on the Base pointer/composite.
Added a new file where all the decoration validation can be performed.
In this change the SPIRV Spec Section 2.16.1 is implemented:
"It is illegal to initialize an imported variable. This means
that a module-scope OpVariable with initialization value cannot be
marked with the Import Linkage Type."
Also added unit tests.
* Added the decoration class as well as the code that registers the
decorations for each <id> and also decorations for struct members.
* Added unit tests for decorations in ValidationState as well as
decoration id tests.
According to the SPIRV Spec (2.16.1):
* There is at least one OpEntryPoint instruction, unless the Linkage
capability is being used.
* No function can be targeted by both an OpEntryPoint instruction and an
OpFunctionCall instruction.
Also updated unit tests to includ OpEntryPoint.
We are adding a new API which can be called to run the SPIR-V validator,
and retrieve the ValidationState_t object. This is very useful for
unit testing.
I have also added basic unit tests that demonstrate usage of this flow
and ease of use to verify correctness.
The validity of each command is checked based on the descripton in
SPIR-V Spec Section 3.32.12 (Composite Instructions).
Also checked that the number of indexes passed to these commands does
not exceed the limit described in 2.17 (Universal Limits).
Also added unit tests for each one.
entry_block_to_construct_ maps an entry block to its construct. The key
in this map (the entry block) is not unique, and therefore the entry for
the continue construct gets overwritten when the selection construct is
discovered.
Since a given block may be the entry block of different types of
constructs, the (basic_block, construct_type) pair should be able to
uniquely identify the construct.
Adds test:
- In this test, a basic block is the entry block of a continue construct
as well as the entry block of a selection construct.
It can be shown that this unit test would crash without the fix in this
PR and passes with the fix in this PR.
Validation for OpPtrAccessChain is similar to OpAccessChain with the
following difference: OpPtrAccessChain takes an extra argument (word 4)
which is the Element <id> argument.
Validation for OpInBoundsPtrAccessChain is also similar to OpPtrAccessChain.
Also added tests for all access chain instructions:
Modified the existing parameterized tests to accommodate OpPtrAccessChain and
OpInBoundsPtrAccessChain.
Also fixed a typo in previous commits.
Using parameterized unit tests to avoid duplicate code that runs the
tests of OpAccessChain and OpInBoundsAccessChain.
This is also a steppingstone to adding tests for OpPtrAccessChain and
OpInBoundsPtrAccessChain.
According to Section 2.17 (Universal Limits) of the SPIR-V Spec, the
control flow nesting depth may not be larger than 1023.
This is checked only when we are required to have structured
control flow. Otherwise it's not clear how to compute control
flow nesting depth.
- Parse CHANGES file with Universal Python line endings in case
the source tree was checked out with Windows line endings.
- Use our own clone of strnlen_s which might not be available
everywhere.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/508
According to sectin 2.17 in SPIR-V Spec, the structure nesting depth may
not be larger than 255. This is interpreted as structures nested in
structures. The code does not look into arrays or follow pointers to see
if it reaches a structure downstream.
Use memoization to avoid exponential runtime.
The validation code for OpAccessChain was missing OpTypeRuntimeArray as
a possible type that can be indexed into.
This was caught by running the validator on VKCTS.
Also adding unit tests for it.
* Result Type must be an OpTypePointer. Its Type operand must be the
type reached by walking the Base’s type hierarchy down to the last
provided index in Indexes, and its Storage Class operand must be the
same as the Storage Class of Base.
* Base must be a pointer, pointing to the base of a composite object.
* Indexes walk the type hierarchy to the desired depth, potentially down
to scalar granularity. The first index in Indexes will select the
top-level member/element/component/element of the base composite. All
composite constituents use zero-based numbering, as described by their
OpType... instruction. The second index will apply similarly to that
result, and so on. Once any non-composite type is reached, there must
be no remaining (unused) indexes. Each of the Indexes must:
- be a scalar integer type,
- be an OpConstant when indexing into a structure.
* Check for the case where no indexes are passed to OpAccessChain.
Minor improvements based on code review.
According to the Universal Limits section of the SPIR-V Spec (2.17), the
number of global variables may not exceed 65,535 and the number of local
variables may not exceed 524,287.
Also added unit tests for each one.
According to the SPIR-V spec (section 2.17: Universal Limits), the
OpTypeFunction instruction may not take more than 255 arguments for the
function. Also added unit tests for it.
The number of (literal, label) pairs passed to OpSwitch may not exceed
16,383. Added code to validate this and added unit tests for it.
Also fixed a typo in another validor error message.
This is described in Section 2.17 of the SPIR-V Spec.
* Updated existing unit test 'SemanticsIdIsAnIdNotALiteral' to pass by
manipulating the ID bound in its binary header.
* Fixed boundary check in the code.
* Added unit test to check the case that the largest ID is equal to the
ID bound.
This change implements the validation for usages of OpSampledImage
instruction as described in the Data Rules section of the Universal
Validation Rules of the SPIR-V Spec.
SpecConstantComposite may specialize to a vector, matrix, array, or
struct. In each case, the number of components and type of components
that are being specialized to must match the expected result type.
Removed use of macros in these tests.
Now using the spvValidateBase class. Using CompileSuccessfully(), and
ValidateInstructions() to compile to binary and run the validator. Also
using getDiagnosticString() to check the proper error message string.
All the heavy lifting is done in ValidateBase class.
According to the Data Rules section of 2.16.1. Universal Validation
Rules of the SPIR-V Spec:
Forward reference operands in an OpTypeStruct
* must be later declared with OpTypePointer
* the type pointed to must be an OpTypeStruct
* had an earlier OpTypeForwardPointer forward reference to the same <id>
These rules are under "Data Rules" in 2.16.1 (Universal Validation
Rules) part of the SPIR-V 1.1 Specification document:
* Scalar floating-point types can be parameterized only as 32 bit, plus
any additional sizes enabled by capabilities.
* Scalar integer types can be parameterized only as 32 bit, plus any
additional sizes enabled by capabilities.
* Vector types can only be parameterized with numerical types or the
OpTypeBool type.
* Matrix types can only be parameterized with floating-point types.
* Matrix types can only be parameterized as having only 2, 3, or 4
columns.
* Specialization constants (see Specialization) are limited to integers,
Booleans, floating-point numbers, and vectors of these.
Number of components in a vector can be 2 or 3 or 4. If Vector16
capability is used, 8 and 16 components are also allowed.
Also added unit tests for vector data rule.
* Allows OpTypeForwardPointer to reference IDs not yet declared in
the module
* Allows OpTypeStruct to reference IDs not yet declared in
the module
Possible Issue: OpTypeStruct should only allow forward references
if the ID is a pointer that is referenced by a forward pointer. Need
Type support in Validator which is currently a work in progress.
There is no difference between the previous IgnoreMessage() function
and a null std::function, from functionality's perspective.
The user can set nullptr as the MessageConsumer, so need to guard
against nullptr before calling the consumer anyway. It's better
we use it internally so that it may expose problems by us instead
of the user.
Default-constructed Pass/PassManager will have a MessageConsumer
which ignores all messages. SetMessageConsumer() should be called
to supply a meaningful MessageConsumer.
Requires use of SPIRV-Headers that has support
for SPV_KHR_shader_ballot.
Adds assembler, disassembler, binary parser support.
Adds general support for allowing an operand to be
only enabled by a set of extensions.
TODO: Validator support for extension checking.
* Use PIMPL idiom in the C++ interface.
* Clean up interface for assembling and disassembling.
* Add validation into C++ interface.
* Add more tests for the C++ interface.
Add the following macros for logging purpose:
* SPIRV_ASSERT
* SPIRV_DEBUG
* SPIRV_UNIMPLEMENTED
* SPIRV_UNREACHABLE
The last two is always turned on, while the first two can only
be turned on in debug build.
Every time an event happens in the library that the user should be
aware of, the callback will be invoked.
The existing diagnostic mechanism is hijacked internally by a
callback that creates an diagnostic object each time an event
happens.
Defer removal of a Phi's result id from the undefined-forward-reference
set until after you've scanned the arguments. The reordering is only
significant for Phi.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/415
ParseNumber(): Returns false if the given string is a nullptr.
ParseAndEncodeXXXX(): Returns kInvalidText and populate error message:
"The given text is a nullptr", if the givne string is a nullptr.
The pass instance is constructed with a map from spec id (uint32_t) to
default values in string format. The default value strings will be
parsed to numbers according to the target spec constant type.
If the Spec Id decoration is found to be applied on multiple different
target ids, that decoration instruction (OpDecorate or OpGroupDecorate)
will be skipped. But other decoration instrucitons may still be
processed.
Pull out the number parsing logic from
AssemblyContext::binaryEncodeNumericLiteral() to utilities.
The new utility function: `ParseAndEncodeNumber()` now accepts:
* number text to parse
* number type
* a emit function, which is a function which will be called with each
parsed uint32 word.
* a pointer to std::string to be overwritten with error messages.
(pass nullptr if expect no error message)
and returns:
* an enum result type to indicate the status
Type/Structs moved to utility:
* template<typename T> class ClampToZeroIfUnsignedType
New type:
* enum EncodeNumberStatus: success or error code
* NumberType: hold the number type information for the number to be parsed.
* several helper functions are also added for NumberType.
Functions moved to utility:
* Helpers:
* template<typename T> checkRangeAndIfHexThenSignExtend() -> CheckRangeAndIfHex....()
* Interfaces:
* template<typename T> parseNumber() -> ParseNumber()
* binaryEncodeIntegerLiteral() -> ParseAndEncodeIntegerNumber()
* binaryEncodeFloatingPointLiteral() -> ParseAndEncodeFloatingPointNumber()
* binaryEncodeNumericLiteral() -> ParseAndEncodeNumber()
Tests added/moved to test/ParseNumber.cpp, including tests for:
* ParseNumber(): This is moved from TextToBinary.cpp to ParseNumber.cpp
* ParseAndEncodeIntegerNumber(): New added
* ParseAndEncodeFloatingPointNumber(): New added
* ParseAndEncodeNumber(): New added
Note that the error messages are kept almost the same as before, but
they may be inappropriate for an utility function. Those will be fixed
in another CL.
De-duplicate constants and unifies the uses of constants for a SPIR-V
module. If two constants are defined exactly the same, only one of them
will be kept and all the uses of the removed constant will be redirected
to the kept one.
This pass handles normal constants (defined with
OpConstant{|True|False|Composite}), some spec constants (those defined
with OpSpecConstant{Op|Composite}) and null constants (defined with
OpConstantNull).
There are several cases not handled by this pass:
1) If there are decorations for the result id of a constant defining
instruction, that instruction will not be processed. This means the
instruction won't be used to replace other instructions and other
instructions won't be used to replace it either.
2) This pass does not unify null constants (defined with
OpConstantNull instruction) with their equivalent zero-valued normal
constants (defined with OpConstant{|False|Composite} with zero as the
operand values or component values).
Also removed the default argument value of `skip_nop` for function
`SinglePassRunAndCheck()` and `SinglePassRunAndDisassemble()`. This is
required to support variadic arguments.
Use libspirv::CapabilitySet instead of a 64-bit mask.
Remove dead function spvOpcodeRequiresCapability and its tests.
The JSON grammar parser is simplified since it just writes the
list of capabilities as a braced list, and takes advantage of
the CapabilitySet intializer-list constructor.
For the spec constants defined by OpSpecConstantOp and
OpSpecContantComposite, if all of their operands are constants with
determined values (normal constants whose values are fixed), calculate
the correct values of the spec constants and re-define them as normal
constants.
In short, this pass replaces all the spec constants defined by
OpSpecContantOp and OpSpecConstantComposite with normal constants when
possible. So far not all valid operations of OpSpecConstantOp are
supported, we have several constriction here:
1) Only 32-bit integer and boolean (both scalar and vector) are
supported for any arithmetic operations. Integers in other width (like
64-bit) are not supported.
2) OpSConvert, OpFConvert, OpQuantizeToF16, and all the
operations under Kernel capability, are not supported.
3) OpCompositeInsert is not supported.
Note that this pass does not unify normal constants. This means it is
possible to have new generatd constants defining the same values.
This lets us write smaller test cases with the IrLoader, avoiding
boilerplate for function begin/end, and basic block begin/end.
Also ForEachInst is more forgiving of cases where a basic block
doesn't have a label, and when a function doesn't have a defining
or end instruction.
Also:
- Add const forms of ForEachInst
- Rewrite Module::ToBinary in terms of ForEachInst
- Add Instruction::ToBinaryWithoutAttachedDebugInsts
- Delete the ToBinary method on Function, BasicBlock, and Instruction
since it can now be implemented with ForEachInst in a less confusing
way, e.g. without recursion.
- Preserve debug line instructions on OpFunctionEnd (and store that
instruction as a unique-pointer, for regularity).
* Fix the behavior when analyzing an individual instruction:
* exisiting instruction:
Clear the original records and re-analyze it as a new instruction.
* new instruction with exisiting result id:
Clear the original records of the exisiting result id. This means
the records of the analyzed result-id-defining instruction will be
overwritten by the record of the new instruction with the same
result id.
* new instruction with new result id or without result id:
Just update the internal records to incorperate the new
instruction.
* Add tests for analyzing individual instruction w/o an exisiting module.
* Refactor ClearInst() implementation
* Remove ClearDef() function.
* Fixed a bug in DefUseManager::ReplaceAllUsesWith() that OpName
instruction may trigger the assertion incorrectly.
* update the blurbs for EraseUseRecordsOfOperandIds()
By deriving from std::iterator, iterator_traits will be properly
set up for our custom iterator type, thus we can use algorithms
from STL with our custom iterators.
Previously we use vectors of objects and move semantics to handle
ownership. That approach has the flaw that inserting an object into
the middle of a vector, which may trigger a vector reallocation,
can invalidate some addresses taken from instructions.
Now the in-memory representation internally uses vector of unique
pointers to handle ownership. Since objects are explicitly heap-
allocated now, pointers to them won't be invalidated by vector
resizing anymore.
The NEW behavior is to not dereference variables or interpret keywords
that have been quoted or bracketed.
For more information, see
https://cmake.org/cmake/help/v3.1/policy/CMP0054.html.
This is to suppress a warning when using CMake 3.1.3+.
- Find unreachable continue targets. Look for back edges
with a DFS traversal separate from the dominance traversals,
where we count the OpLoopMerge from the header to the continue
target as an edge in the graph.
- It's ok for a loop to have multiple back edges, provided
they are all from the same block, and we call that the latch block.
This may require a clarification/fix in the SPIR-V spec.
- Compute postdominance correctly for infinite loop:
Bias *predecessor* traversal root finding so that you use
a later block in the original list. This ensures that
for certain simple infinite loops in the CFG where neither
block branches to a node without successors, that we'll
compute the loop header as dominating the latch block, and the
latch block as postdominating the loop header.
Fixes dominance calculation when there is a forward arc from an
unreachable block A to a reachable block B. Before this fix, we would
say that B is not dominated by the graph entry node, and instead say
that the immediate dominator of B is the psuedo-entry node of the
augmented CFG.
The fix:
- Dominance is defined in terms of a traversal from the entry block
of the CFG. So the forward DFS should start from the function
entry block, not the pseudo-entry-block.
- When following edges backward during dominance calculations, only go to
nodes that are actually reachable in the forward traversal.
Important: the sense of reachability flips around when computing
post-dominance.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/297
AssemblyBuilder contains boilplates.
Adds OpName instructions for all added defining instructions.
Adds OpDecorate SpecId for all spec constants added with OpSpecConstant,
OpSpecConstantTrue and OpSpecConstantFalse instructions.
The def-use dominance checker doesn't have enough info to know
that a particular use is in an OpPhi, so skip tracking those uses
for now. Add a TODO to do a proper OpPhi variable-argument check
in the future.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/286
Ensure the dominance calculation visits all nodes in the CFG.
The successor list of the pseudo-entry node is augmented with
a single node in each cycle that otherwise would not be visited.
Similarly, the predecssors list of the pseduo-exit node is augmented
with the a single node in each cycle that otherwise would not
be visited.
Pulls DepthFirstSearch out so it's accessible outside of the dominator
calculation.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/279
Add a pass to freeze spec constants to their default values. This pass does
not fold the frozen spec constants and does not handle SpecConstantOp
instructions and SpecConstantComposite instructions.
* Creates an ID class which manages definition and use of IDs
* Moved tracking code from validate.cpp to validate_id.cpp
* Rename and combine SsaPass and ProcessIds into IdPass
* Remove module dependency in Function
* The `Input` StorageClass doesn't require the `Shader` capability
anymore.
* The `Sampled1D` and `SampledBuffer` capabilities don't require
the `Shader` capability anymore. So they do not indirectly
depend on the `Matrix` capability. So are the `Image1D` and
`ImageBuffer` capabilities, which depend on `Sampled1D` and
`SampledBuffer`.
A new GLSL grammar file is uploaded for SPIR-V 1.1, but it's the
same as the existing one for SPIR-V 1.0.
Now tracking commit 3814effb879ab5a98a7b9288a4b4c7849d2bc8ac in
SPIRV-Headers.
For DependencyInfinite and DependencyLength, test
that they don't require a capability to be turned on.
Also, that they are assembled, binary parsed, and disassembled
correctly.
Works around issue 248 by weakening the test:
https://github.com/KhronosGroup/SPIRV-Tools/issues/248
The validator should try to track (32-bit) constant values, and then
for capability checks on IDs, check the referenced value, not the
raw ID number.
For dominance calculations we use an "augmented" CFG
where we always add a pseudo-entry node that is the predecessor
in the augmented CFG to any nodes that have no predecessors in the
regular CFG. Similarly, we add a pseudo-exit node that is the
predecessor in the augmented CFG that is a successor to any
node that has no successors in the regular CFG.
Pseudo entry and exit blocks live in the Function object.
Fixes a subtle problem where we were implicitly creating
the block_details for the pseudo-exit node since it didn't
appear in the idoms map, and yet we referenced it. In such a case the
contents of the block details could be garbage, or zero-initialized.
That sometimes caused incorrect calculation of immediate dominators
and post-dominators. For example, on a debug build where the details
could be zero-initialized, the dominator of an unreachable block would
be given as the pseudo-exit node. Bizarre.
Also, enforce the rule that you must have an OpFunctionEnd to close off
the last function.
The operands following the extended instruction literal
number are determined by the extended instruction itself.
So drop the zero-or-more IdRef pattern at the end of OpExtInst.
It's arguable whether this should actually be a grammar fix. I've
chosen to patch this in SPIRV-Tools instead of in the grammar file.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/233
Also fix two test cases for OpenCL extended instructions. These
errors of supplying too many operands are now detected.
* ValidationState_t and idUsage now store the addressing model and memory model of the SPIR-V module (this is necessary for certain instructions that need different checks depending on if the logical or physical addressing model is used)
* removed SpvOpPtrAccessChain and SpvOpInBoundsPtrAccessChain from spvOpcodeIsPointer again as these are disallowed in logical addressing mode and only allowed in physical addressing mode (which doesn't use/need spvOpcodeIsPointer in the first place)
* added SpvOpImageTexelPointer and SpvOpCopyObject to spvOpcodeIsPointer
* OpLoad/OpStore now only check if the used pointer operand originated from a valid pointer producing opcode in logical addressing mode (as per 2.16.1)
* moved bitcast pointer tests to the kernel / physical addressing model part (+cleanup)
* renamed spvOpcodeIsPointer to spvOpcodeReturnsLogicalPointer to clarify this function is only meant to be used with the logical addressing model
Refactor the ValidateCapability test fixture.
Explain the meaning of test parameters. Factor out methods for
convenience and readability. DRY v1.0 and v1.1 tests.
Add a high level version number for SPIRV-Tools, beginning
with v2016.0-dev. The README describes the format of the
version number.
The high level version number is extracted from the CHANGES
file. That works around:
- stale-bait for when we don't add tags to the repository
- our inability to add tags to the repository
Option --version causes spirv-as, spirv-dis, and spirv-val to
show the high level version number.
Add spvSoftwareVersionString to return the C-string for
the high level version number.
Add spvSoftwareVersionDetailsString() so that clients can get
more information if they want to.
Also allows us to clean up the uses in the tool executables files,
so now only one file includes build-version.inc.
Move the update-build-version logic to the only
CMakeLists file that needs it.
The update build version script takes a new argument
to name the output file.
Introduced in v1.1, SubgroupDispatch adds the following:
- two new execution modes
- one new capability
- two new opcodes
Extend ValidateBase methods to take a spv_target_env. Replace the
context_ member with ScopedContext inside the said methods. Give
ScopedContext wider visibility by moving it outside
TextToBinaryTestBase.
Add test for named-barrier instructions and capability.
Add spv_target_env as an optional argument to CompileSuccessfully() and
CompileFailure(). Currently defaults to UNIVERSAL_1_0, though that
could change in the future.
Make spv_context a local variable in test methods instead of a
TextToBinaryTestBase member. Introduce ScopedContext to make temp
contexts easier.
For fulfilling this purpose, the |opcode| field in the
|spv_parsed_instruction_t| struct is changed to of type uint16_t.
Also add functions to query the information of a given SPIR-V
target environment.
This patch uses a Python script to parse the JSON grammar file to
generate the opcode table and operand kind tables.
Now we don't need to do the post-processing (from OperandClass
to spv_operand_type_t) and copying of the opcode info table is
not required anymore!
Previously, the grammar allowed many execution modes for a single
OpExecutionMode instruction.
Removes the variable- and optional- execution mode operand type
enum values.
Issue found by antiagainst@
Recognize SpvOpInBoundsPtrAccessChain and SpvOpPtrAccessChain as opcodes
returning a pointer.
* spvOpcodeIsPointer: recognize SpvOpInBoundsPtrAccessChain and SpvOpPtrAccessChain as opcodes returning a pointer
* isValid<SpvOpEntryPoint>: don't check kernel function signatures (these don't have to be 'void main(void)')
* added tests for kernel OpEntryPoint, OpInBoundsPtrAccessChain and OpPtrAccessChain, as well as facilities to actually test kernel/OpenCL SPIR-V
* fixed pow and pown specification (both should take 2 parameters), spec bug reported at https://www.khronos.org/bugzilla/show_bug.cgi?id=1469
* use ASSERT_TRUE instead of ASSERT_EQ
* added pow and pown test (pow(val, 2.0f) and pown(val, 3))
Revert " * fixed pow and pown specification (both should take 2 parameters), spec bug reported at https://www.khronos.org/bugzilla/show_bug.cgi?id=1469"
This reverts commit c3d5a87e73.
Revert " * added pow and pown test (pow(val, 2.0f) and pown(val, 3))"
This reverts commit 7624aec720.
Users always want to run all the checks. The spv_validate_options_t
mechanism, which provides little benefits to users, complicates the
internal implementation and also makes the tests exercise different
paths as users do.
Right now the tests are more like integration tests instead of
unit tests, which should be our next refactoring aim.
Now we have public headers arranged as follows:
$SPIRV_TOOLS_ROOT/include/spirv-tools/libspirv.h
$SPIRV_TOOLS_ROOT/include/spirv/spirv.h
$SPIRV_TOOLS_ROOT/include/spirv/GLSL.std.450.h
$SPIRV_TOOLS_ROOT/include/spirv/OpenCL.std.h
A project should use -I$SPIRV_TOOLS_ROOT/include
and then #include "spirv-tools/libspirv.h"
The headers from the SPIR-V Registry can be accessed as "spirv/spirv."
for example.
The install target should also install the headers from the SPIR-V
Registry. The libspirv.h header is broken otherwise.
The SPIRV-Tools library depends on the headers from the SPIR-V Registry.
The util/bitutils.h and util/hex_float.h are pulled into the internal
source tree. Those are not part of the public API to SPIRV-Tools.
Also tweak the image operands test with OpImageFetch so the input assembly
more closely matches the expected output. But don't make it a round
trip test since we want to check the assembler's ability to read the
image operand mask with out-of-order mask names.
The OpImageSparseRead assembly test is also a round trip disassembler
test.
- The SPIR-V spec generator has changed how it represents optional
operands. Now it tracks a separate boolean flag indicating optionality.
However, SPIRV-Tools still wants to represent both operand class
and optionality in the same enums space (SPV_OPERAND_TYPE_*).
So there's extra work in the patch.
- In the spec generator, OperandImage is now OperandImageOperands.
This affects enum translation in opcode.cpp.
- In the spec generator, image operands are explicitly followed by
Id, and VariableIds. However, SPIRV-Tools uses the bits set
in the image operand bitmask to control the number and meaning
of the Ids that follow. So in writing the opcode.inc syntax
table, drop all operands after OperandImageOperands.
- Some enums are now more explicitly represented in the generated
opcode.inc:
- AccessQualifier (e.g. on OpTypeImage), in both required and
optional flavours.
- MemoryAccess (e.g. on loads and stores)
- Add SPV_OPERAND_TYPE_OPTIONAL_ACCESS_QUALIFIER
- Add tests for the optional AccessQualifier operand on OpTypeImage.
- Update the AccessQualifier test for OpTypeImage so it's a round
trip test through the disassembler as well.
- For 32- and 64-bit floats, overflow is a parse error
This works around a difference between Xcode's istringstream
and other platforms. Xcode's runtime library will happlily
"round up" overflow values to infinity. We want to make it fail.
- When parsing a float fails due to bad syntax, follow C++11
behaviour for operator>> and set the value to zero.
- When parsing a 32-bit or 64-bit float overflows, follow C++11
behaviour for operator>> and set the value to the nearest
normal value: either max or lowest finite value for the type.
- Add FloatProxy<T>::max() and ::lowest()
- Make 16-bit overflow behaviour more consistent: we always get a
16-bit infinity of the right sign, whether the original string
is a normal value for 32-bit or an overflow value for 32-bit.
That matches our earlier intent.
Added TODO's to make 16-bit overflow always an error, just like
for 32-bit and 64-bit.
- Simplify normal parsing of Float16 values by delegating to
normal parsing of 32-bit floats.
Also checks some hex literal cases.
This addresses part of
https://github.com/KhronosGroup/SPIRV-Tools/issues/45
by removing the parseNumber case for "-0" on unsigned
integers. We don't care about that platform difference
at the level of std::istringstream, since we reject it
at a higher parsing level.
Also
- Add type_id to spv_id_info_t.
- Use spv_id_info_t::type_id instead of words[1].
Triggered some asserts on tests, where the code incorrectly assumed
words[1] had a type. Remove the asserts and handle gracefully.
- Add tests for OpStore of a label, a void, and a function.
Remove redundant validations of OpConstant and OpConstantComposite.
Binary parser already performs these checks, so the validations can
never be triggered.
Enable bad-constant tests.
Replace two other, imperfect mechanisms for use-def tracking.
Use ValidationState_t::entry_points to track entry points.
Concentrate undefined-ID diagnostics in a single place.
Move validate_types.h content into validate.h due to increased
inter-dependency.
Track uses of all IDs: TYPE_ID, SCOPE_ID, ...
Also update some blurbs.
Fix entry-point accumulation and move it outside ProcessIds().
Remove validate_types.h from CMakeLists.txt.
Blurb for spvIsIdType.
Remove redundant diagnostics for undefined IDs.
Join "can not" and reformat.
This adds function and block layout checks to the validator. Very
basic CFG code has been added to make sure labels and branches
are correctly ordered.
Also:
* MemoryModel and Variable instruction checks/tests
* Use spvCheckReturn instead of CHECK_RESULT
* Fix invalid SSA tests
* Created libspirv::spvResultToString in diagnostic.h
* Documented various functions and classes
* Fixed error messages
* Fixed using declaration for FunctionDecl enum class
Added additional compilation flags to gcc and clang builds.
Adds -Wall -Wextra -Wno-long-long -Wshadow -Wundef -Wconversion
-WNo-sign-conversion and -Wno-missing-field-initializers
where appropriate.
Does not add -Wundef to tests, because GTEST tests undefined
macros all over the place.
This adds half-precision constants to spirv-tools.
16-bit floats are always disassembled into hex-float format,
but can be assembled from floating point or hex-float inputs.
If we later add a source/ as an -I include directory,
then avoid confusing other headers that want to include the
standard "endian.h" from /usr/include.
Also rename source/endian.cpp to source/spirv_endian.cpp
* Validates module level instructions for logical layout
conformance
* Does not validate:
1. Function logical layout
2. Minor cases with OpVariable
3. Missing MemoryModel instruction in module
4. Order of function definition and function declaration
* 782 unit tests for logical layout
Addressed feedback
Most uses of an ID must occur after the definition
of the ID. Forward references are allowed for
things like OpName, OpDecorate, and various cases
of control-flow instructions such as OpBranch, OpPhi,
and OpFunctionCall.
TODO: Use CFG analysis for SSA checks. In particular,
an ID defined inside a function body is only usable inside
that function body. Also, use dominator info to catch
some failing cases.
Also:
* Validator test cases use (standard) assignment form.
* Update style to more closely follow the Google C++ style guide
* Remove color-diagnostics flag.
This is enabled by default on terminals with color. Prints
hidden ASCII for terminals that can't handle color(Emacs)
* Pass functors to SSAPass to check if the
operand can be forward referenced based on its index value
* Return SPV_ERROR_INVALID_ID for ID related errors
spvBinaryParse returned SPV_ERROR_INVALID_BINARY for all types of
errors. Since spvBinaryParse does some ID validation, this was
returning inappropriate error codes for some tests.
* Common fixture for validation tests.
It only runs certian validation passes.
* Add a SPV_VALIDATE_SSA_BIT for testing purposes
* Fixtures now return error codes
* Add OpName support in diag message and unit tests
* Binary parsing can fail with invalid ID or invalid binary error code
Tests include:
* OpDecorate
* OpName
* OpMemberName
* OpBranchConditional
* OpSelectionMerge
* OpMemberDecorate
* OpGroupDecorate
* OpDeviceEnqueue
* Enable several tests failing in ID validation.
This is a grammar fix. The Decoration operand of OpDecorate (and
OpMemberDecorate) determines the remaining operands. Don't just
allow any number of literal numbers as operands.
(The OperandVariableLiterals operand class as the last member
of the OpDecorate and OpMemberDecorate entries in in opcode.inc is
an artifact of how the spec generates the opcode descriptions. It's
not suitable for parsing those instructions.)
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/34
Covers the case where the string literal is the
last operand.
TODO: Case where one more operand is expected after
the string literal. Occurs only for the LinkageAttributes
decoration. To test that, we need to update the grammar
as well.
Add unit tests for all diagnostics issued by spvBinaryParse.
Handle image format operands in the binary parser and the
disassembler.
Document that the callback function pointers can be null,
in which case they are ignored.
Detect exhaustion of input when parsing an operand,
to avoid buffer overruns on some invalid input cases.
Fix the description strings for some operand types.
Make the diagnostic messages for those operand types
consistent between the assembler and binary parser.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/29
Add a non-zero spv_result_t value SPV_REQUESTED_TERMINATION
which should be used to signal an ok result, but signals
early termination for a process, such as binary parsing.
Tests include:
- correct contents sent to header and instruction callbacks
- non-zero status from a callback should terminate parsing,
but the parser should not generate its own diagnostic.
TODO: Check diagnostics generated by the parser itself.
- Removed dead configuration in CMakeLists.txt.
- Used target_compile_options() instead of CMAKE_{C|CXX}_FLAGS.
- Turned on warnings on tests.
- Fixed various warnings for comparing signed with unsigned values.
- Removed dead code exposed by compiler warnings.
Bits 24-31: 0
Bits 16-23: SPIR-V major number (1)
Bits 8-15: SPIR-V minor number (0)
Bits 0-7: SPIR-V minor number (2)
The assembler will construct the word appropriately,
and the disassemble will print it in major.minor.revision form.
The high 16-bits are a registered generator tool.
These are registered at
https://www.khronos.org/registry/spir-v/api/spir-v.xml
The low 16-bits are tool-specific. It might be a version number,
for example, but is not constrained by the spec or by the registration
process.
The disassembler prints the tool name when we know it.
If we don't, print "Unknown" and then the numeric tool number
in parentheses.
In all cases, the disassembler prints lower 16-bit number on the
same line but after the tool name.
Also add newly registered generators:
6: Khronos LLVM/SPIR-V Translator
7: Khronos SPIR-V Tools Assembler
Decoration Stream depends on it.
GeometryStreams depends on Geometry capability.
Spot check dependence of OpEmitStreamVertex on GeometryStreams.
(Opcode dependencies on capabilities are automatically generated from
opcode.inc)
Previously the opcode table is declared as an global array and we
have spvOpcodeTableInitialize() modifying it. That can result in
race condition. Now spvOpcodeTabelGet() copies the whole underlying
array.
Updated readme.
Note: The header advertises itself as Rev 1, but contains
many (all?) the updates intended for Rev 2. We might need
to update one more time before SPIR-V 1.0 Rev2 is published.
Regenerated syntax tables for 1.0.
Changed names:
InputTriangles -> Triangles
InputQuads -> Quads
InputIsolines -> Isolines
WorkgroupLocal -> Workgroup
WorkgroupGlobal -> CrossWorkgroup
PrivateGlobal -> Private
(Dim) InputTarget -> SubpassData
WorkgroupLocalMemoryMask -> WorkgroupMemoryMask
WorkgroupGlobalMemoryMask -> CrossWorkgroupMemoryMask
AsyncGroupCopy -> GroupAsyncCopy
WaitGroupEvents -> GroupWaitEvents
Remove:
IndependentForwardProgress capability
Smooth decoration
FragColor BuiltIn
WorkgroupLinearId in favour of LocalInvocationId
ImageSRGBWrite capability
Special OpenCL image instructions
Add:
image channel data type UnormInt101010_2
AcquireReleaseMask
InputTargetIndex updates:
InputTargetIndex -> InputAttachmentIndex
InputAttachmentIndex depends on InputAttachment capability,
and it takes a literal number argument.
Capability StorageImageExtendedFormats updates:
Enum value changed from 26 to 49. (Changes position in tables).
Replaces AdvancedImageFormat capability.
OpenCL source language -> OpenCL_C, OpenCL_CPP
Replaced uint64_t with size_t in the places that make sense and
added spv_const_binary{,_t} to allow the interface to accept non
modifiable spirv where appropriate.
The bit pattern for a hex float is preserved through
assembly and disassembly.
You can use a hex float to express Inf and any kind of NaN
in a portable way.
Zero and normal floating point values are printed with enough
enough digits to reproduce all the bits exactly.
Other float values (subnormal, infinity, and NaN) are printed
as hex floats.
Fix a binary parse bug: Count partially filled words in a
typed literal number operand.
TODO: Assembler support for hex numbers, and therefore reading
infinities and NaNs.
- Concrete operand types are never optional.
Split them to make this so, e.g. add SPV_OPERAND_TYPE_IMAGE
since there was SPV_OPERAND_TYPE_OPTIONAL_IMAGE.
Similarly for SPV_OPERAND_TYPE_MEMORY_ACCESS.
This entails duplicating two operand table entries.
- The above, plus some rearranging of enums, allows us to define
first and last optional operand types, and first and last
variable operand types.
This lets us simplify the code for spvOperandIsOptional, and
spvOperandIsVariable.
- Replace SPV_OPERAND_TYPE_MULTIWORD_LITERAL_NUMBER with the
more accurately named SPV_OPERAND_TYPE_TYPED_LITERAL_NUMBER.
Its special characteristic is that the type of the literal
number is determined by some previous operand in the instruction.
This is used for literals in OpSwitch, OpConstant, and OpSpecConstant.
This lets us refactor operand parsing cases in the assembler.
- Remove the special required-thing-in-optional-tuple in favour of
the corresponding concrete operand type:
SPV_OPERAND_TYPE_ID_IN_OPTIONAL_TUPLE
--> SPV_OPERAND_TYPE_ID
SPV_OPERAND_TYPE_INTEGER_LITERAL_IN_OPTIONAL_TUPLE
--> SPV_OPERAND_TYPE_INTEGER_LITERAL
- Constrain spvOpeandTypeStr to only have to work for non-variable
operand types. Add a test for this.
The binary parser has a C API, described in binary.h.
Eventually we will make it public in libspirv.h.
The API is event-driven in the sense that a callback is called
when a valid header is parsed, and for each parsed instruction.
Classify some operand types as "concrete". The binary parser uses
only concrete operand types to describe parsed instructions.
The old disassembler APIs are moved into disassemble.cpp
TODO: Add unit tests for spvBinaryParse.
Note that we are more strict than Google style for one aspect:
pointer/reference indicators are adjacent to their types, not
their variables.
find . -name "*.h" -exec clang-format -i {} \;
find . -name "*.cpp" -exec clang-format -i {} \;
It is valid for float values to be modified on copy if they are NaN,
so long as they remain the correct NaN. What this means is that
we can not rely on the float data-type for storing float values
if we want to retain bit patterns.
Added FloatProxy which stores data in an unsigned integer, and updated
the HexFloat template to deal with FloatProxy values instead.
This is an attempt to fix the unit tests on DeveloperStudio 2013.
Currently, the size of the earth_africa string is reported as 2
on Windows. But I think that may be 2 16-bit characters.
Versions 1.2, 2.0, and 2.1 all use the same
extended instruction list.
Updated the source code patch for the SPIR-V doc generator,
so it can both generate the core syntax table, and also the
OpenCL extended instructions table.
Tested the Math and Common functions.
TODO: test the remaining entries.
Except for OpConstant and OpSpecConstant, all other literal number
operands are indeed unsigned integers. So,
* Rename all *LITERAL_NUMBER* operand types to *LITERAL_INTEGER*.
* Expect unsigned integers for *LITERAL_INTEGER* operands.
* Keep MULITPLE_WORD_LITERAL untouched since it is only used by
OpConstant and OpSpecConstant.
And we want to provide the capability to specify floating-point
numbers after !<integer> in the alternate parsing mode. So,
OPTIONAL_LITERAL_NUMBER is reserved for OPTIONAL_CIV.
The DiagnosticStream will not emit the accumulated message
text if the error is SPV_FAILED_MATCH.
Change various interfaces to accept the intended error
code instead of a boolean "is_optional". This allows
us to avoid repeating the following type of logic deep
inside helper methods:
if (is_optional) return SPV_FAILED_MATCH;
return diagnostic() << " message text ";
Affects OpConstant, and OpSwitch.
Adds constant libspirv::kUnknownType for readability.
Adds tests for hexadecimal number parsing.
Updates syntax.md to describe hex parsing, including
sign extension.
Use this to shorten error return code in the assembler.
For example, change this:
if (error = something()) {
diagnostic() << " Bad integer literal " << value;
return SPV_ERROR_INVALID_VALUE;
}
to this:
if (error = something())
return diagnostic() << " Bad integer literal " << value;
Also shorten code due to the fact that binaryEncodeU32 and
binaryCodeU64 can't fail (short of failure to expand a std::vector).
We need to know how to generate correct SPIRV for cases like
OpConstant %int64 42 since the current parser will encode the 42 as a
32-bit value incorrectly.
This change is the first of a pair. This one tracks types, and makes
sure that OpConstant and OpSpecConstant are only ever called with
Integer or Float types, and OpSwitch is only called with integer
generating values.
Move the definition of spv_instruction_t to an internal
header file, since it now depends on C++ and is not
used by the external interface.
Use a std::vector<uint32_t> in spv_instruction_t
instead of a fixed size array.
Fixes dependencies among capabilities. (The table should store
the mask of capabilites, not the capability enum.)
Remove the old spot check test for capabilities of enums.
Currently it uses a union, a write to one field, and read from
another of a different type.
Moves the endianness-determining macro close to the variable
it references.
Adds a form of spvtest::MakeInstruction which takes two vectors
of operands. That leads to simpler test code.
We can clean up some other test code, in another CL.
Implement some outstanding feedback from
Ic29c5a4a8178a62a5a1acad13d02f19cc1307097:
- use "token" instead of "word" when referring to assembly text
- specify how the numbers are parsed
Add a test for negative numbers.
Many instructions added and a few changed structure.
Workarounds:
- Some operands can be enabled by either one of two
capabilities. The spv_operand_desc_t does not handle that
now. For now just select the first one.
Fixes to tests:
- OpLoopMerge now takes a mandatory continue target.
- OpTypePipe drops the type argument. Pipes are opaque.
- OpLine no longer takes a target ID argument.
The ID validator was fixed the OpLine and OpTypePipe
changes. Those were the only ID validator tests affected.
The patch to the spec doc generator was updated so it handles
the two-capability case, even if in an hacky way.
Also removed un-necessary heap-allocation of spv_named_id_table.
This removed the necessity to expose a function to create/destroy it
and simplified the interface.
Make it a class, since it has non-trivial behaviour for converting
the enumerated value to a uint32_t value. (Comply with style guide.)
Merge EnumCaseWithOperands into EnumCase.
Fixed an issue where some of the tests were testing
the wrong word with the wrong operation. (| != ||).
Coalesced the many versions of EnumCase into one.
Added a get_value() to EnumCase to convert to a uint32_t.
Replaces ASSERT_TRUE(pointer), with ASSERT_NE(nullptr, pointer),
so that we do not do implicit pointer->bool conversion.
Removed const from some test structs since gtest needs to be
able to swap them.
Properly support a memory access mask with a combination
of bits, including the Aligned bit. When the Aligned bit is
set, the parser should expect an alignment value literal operand.
For example, support combining mask enums with "|",
such as "NotNaN|AllowRecip" for the fast math mode.
This is supported for mask values that don't modify the
expected operand pattern:
- fast math mode
- function control
- loop control
- selection control
TODO: disassembler support to print them as mask expressions.
This is in preparation for coming tests that will also
use the templated EnumCase instead of making their
own structs.
Also reformat AccessQualifier test.
Add a new operand type SPV_OPERAND_TYPE_MULTIWORD_LITERAL_NUMBER
to represent those operands that can expand into multiple words.
Now only OpConstant and OpSpecConstant have such kind of operand.
Added a new enum for supported assembly syntax formats:
Canonical Assembly Format (CAF) and Assignment Assembly Format (AAF).
Updated assembler interface functions to support choice of assembly
syntax format.
The undefined pointer might get through to the call
to spvTextDestroy if the test fails on an earlier step.
In that case I just want to see the test failure, and
not a test failure followed by undefined behaviour.
Add text_fixture::TextToBinaryTestBase::CompiledInstructions,
to more easily just examine the generated instructions by skipping
over the header.
Add spvtest::MakeInstruction utility function to easily generate
a vector containing an opcode and its operands.
Previous the api used spv_text_t and spv_binary_t for both input
and output, but depending on the usage, you either MUST
call spvBinaryDestroy or you MUST NOT call spvBinaryDestroy on the
pointer.
Always try to destroy the binary during common methods
of test fixtures. This is safe if no other code in the test
attempted to destroy the binary.
Take advantage of the fact spvBinaryDestroy is a no-op on a nullptr,
by eliminating the null pointer check in the caller.
The assembler and disassembler now use a dynamically adjusted
sequence of expected operand types. (Internally, it is a deque,
for readability.) Both parsers repeatedly pull an expected operand
type from the left of this pattern list, and try to match the next
input token against it.
The expected pattern is adjusted during the parse to accommodate:
- an extended instruction's expected operands, depending on the
extended instruction's index.
- when an operand itself has operands
- to handle sequences of zero or more operands, or pairs of
operands. These are expanded lazily during the parse.
Adds spv::OperandClass from the SPIR-V specification generator.
Modifies spv_operand_desc_t:
- adds hasResult, hasType, and operandClass array to the opcode
description type.
- "wordCount" is replaced with "numTypes", which counts the number
of entries in operandTypes. And each of those describes a
*logical* operand, including the type id for the instruction,
and the result id for the instruction. A logical operand could be
variable-width, such as a literal string.
Adds opcode.inc, an automatically-generated table of operation
descriptions, with one line to describe each core instruction.
Externally, we have modified the SPIR-V spec doc generator to
emit this file.
(We have hacked this copy to use the old semantics for OpLine.)
Inside the assembler, parsing an operand may fail with new
error code SPV_FAIL_MATCH. For an optional operand, this is not
fatal, but should trigger backtracking at a higher level.
The spvTextIsStartOfNewInst checks the case of the third letter
of what might be an opcode. So now, "OpenCL" does not look like
an opcode name.
In assembly, the EntryPoint name field is mandatory, but can be
an empty string.
Adjust tests for changes to:
- OpSampedImage
- OpTypeSampler
Start using GMock: modify CMakeLists, fix googletest URL in readme.
Add useful utilities to the TestFixture class. Also make it conform to
go/gunit recommendations about setup/teardown.
Use double quotes ("). They can be interspersed with
other whitespace characters, just like shell quoting.
A backslash (\) always escapes the next character.
The end of the stream always terminates the word.
Add AutoText struct to unit test utilities, to easily
make spv_text_t values and reference them as spv_text values.
- a single hyphen is a string, not a number.
- a string with more than one period is a string, not a number
- check for string overflow
Add some unit tests
Since now we can distinguish between def and use according to
the variable's location, there is no need to keep two variable
prefixes.
Also reformat tests to use the value generating instruction
format ("<result-id> = <opcode> <operand>..").
Fix the bug that TextAdvance() forgot to skip whitespace at the
beginning of the next line after a comment line.
Fix the bug that TextAdvanceLine() increase line number after going
over a character.
For enum Capability and enum Op, not all newly added enumerants are
registered into capabilityInfoEntries and opcodeTableEntries yet.
That will come in following commits.