Commit Graph

61 Commits

Author SHA1 Message Date
dong-ja
706dc27a62
Add new target for spirv-lint (#4446)
This PR adds a new executable spirv-lint with a simple "Hello, world!"
program, along with its associated library and a dummy unit test.

For now, only adds to CMake and Bazel; other build systems will be added
in a future PR.

Issue: #3196
2021-08-06 14:03:59 -04:00
Alastair Donaldson
3ab6fb9c03
Add CMake rules for libFuzzer targets (#4445)
Adds CMake rules for the libFuzzer targets, with a view to running
them on OSS-Fuzz.

Fixes #4448.
2021-08-05 08:11:21 +00:00
Ben Clayton
7403dfafd8
CMake: Add SPIRV_TOOLS_BUILD_STATIC flag (#3910)
If enabled the following targets will be created:

* `${SPIRV_TOOLS}-static` - `STATIC` library. Has full public symbol visibility.
* `${SPIRV_TOOLS}-shared` - `SHARED` library. Has default-hidden symbol visibility.
* `${SPIRV_TOOLS}`        - will alias to one of above, based on BUILD_SHARED_LIBS.

If disabled the following targets will be created:

* `${SPIRV_TOOLS}`        - either `STATIC` or `SHARED` based on the new `SPIRV_TOOLS_LIBRARY_TYPE` flag. Has full public symbol visibility.
* `${SPIRV_TOOLS}-shared` - `SHARED` library. Has default-hidden symbol visibility.

Defaults to `ON`, matching existing build behavior.

This flag can be used by package maintainers to ensure that all libraries are built as shared objects.
2020-10-29 09:25:26 -04:00
Ben Clayton
6aed7ffbc7
CMake: Enable building with BUILD_SHARED_LIBS=1 (#3490)
Rename the `${SPIRV_TOOLS}` target to `${SPIRV_TOOLS}-static` and alias `${SPIRV_TOOLS}` to either `${SPIRV_TOOLS}-static` or `${SPIRV_TOOLS}-shared` depending on `BUILD_SHARED_LIBS`.

Re-point all internal uses of `${SPIRV_TOOLS}` to `${SPIRV_TOOLS}-static`.

`${SPIRV_TOOLS}-static` is explicitly renamed to just `${SPIRV_TOOLS}` to ensure the name does not change from current behavior.

Build the `SPIRV-Tools-*` libraries as static, as this is what they always were.

Force the external targets `gmock` and `effcee` to be built statically. These either do not support being built as shared libraries, or require special flags.

Issue: #3482
2020-07-27 13:29:07 -04:00
David Neto
64f36ea529
Support OpenCL.DebugInfo.100 extended instruction set (#3080)
* Clone opencl.debuginfo.100 grammar from debuginfo grammar

Update version number to 200 revision 2

* Apply content from OpenCL.DebugInfo.100 extension text

* Rename grammar file

* Support OpenCL.DebugInfo.100 extended instructions

Add support for prefixing operand type names, to disambiguate
them between different instruction sets.

* Add tests for OpenCL.DebugInfo.100

* Support lookup of OpenCL.DebugInfo.100 extinst

* Add tests for enum values

* Recognize 2017-2019 as copyright date range

* Android.mk: support OpenCL.DebugInfo.100 extended instruction set

Also, stop generating core instruction tables for non-unified1 versions
of the grammar.

* Imported entity operand type is concrete

* Bazel: Suppoort OpenCL.DebugInfo.100

* BUILD.gn: Support OpenCL.DebugInfo.100
2019-12-19 17:16:26 -05:00
David Neto
e70b009b0f
Add support for SPV_KHR_non_semantic_info (#3110)
Add support for SPV_KHR_non_semantic_info

This entails a couple of changes:

- Allowing unknown OpExtInstImport that begin with the prefix `NonSemantic.`
- Allowing OpExtInst that reference any of those sets to contain unknown
  ext inst instruction numbers, and assume the format is always a series of IDs
  as guaranteed by the extension.
- Allowing those OpExtInst to appear in the types/variables/constants section.
- Not stripping OpString in the --strip-debug pass, since it may be referenced
  by these non-semantic OpExtInsts.
- Stripping them instead in the --strip-reflect pass.

* Add adjacency validation of non-semantic OpExtInst

- We validate and test that OpExtInst cannot appear before or between
  OpPhi instructions, or before/between OpFunctionParameter
  instructions.

* Change non-semantic extinst type to single value

* Add helper function spvExtInstIsNonSemantic() which will check if the extinst
  set is non-semantic or not, either the unknown generic value or any future
  recognised non-semantic set.

* Add test of a complex non-semantic extinst

* Use DefUseManager in StripDebugInfoPass to strip some OpStrings

* Any OpString used by a non-semantic instruction cannot be stripped, all others
  can so we search for uses to see if each string can be removed.
* We only do this if the non-semantic debug info extension is enabled, otherwise
  all strings can be trivially removed.

* Silence -Winconsistent-missing-override in protobufs
2019-12-18 18:10:29 -05:00
Alastair Donaldson
1b71e45338
Add "split block" transformation. (#2633)
With this pass, the fuzzer can split blocks in the input module.  This
is mainly useful in order to give other (future) transformations more
opportunities to apply.
2019-05-29 16:42:46 +01:00
dan sinclair
42abaa099a Remove MarkV and Stats code. (#2576)
* Remove MarkV and Stats code.

This Cl removes the MarkV and Stats code from SPIRV-Tools. This code was
unused and currently un-maintained.
2019-05-24 15:43:59 -04:00
David Neto
63f57d95d6
Support SPIR-V 1.4 (#2550)
* SPIR-V 1.4 headers, add SPV_ENV_UNIVERSAL_1_4

* Support --target-env spv1.4 in help for command line tools

* Support asm/dis of UniformId decoration

* Validate UniformId decoration

* Fix version check on instructions and operands

Also register decorations used with OpDecorateId

* Extension lists can differ between enums that match

Example: SubgroupMaskEq vs SubgroupMaskEqKHR

* Validate scope value for Uniform decoration, for SPIR-V 1.4

* More unioning of exts

* Preserve grammar order within an enum value

* 1.4: Validate OpSelect over composites

* Tools default to 1.4

* Add asm/dis test for OpCopyLogical

* 1.4: asm/dis tests for PtrEqual, PtrNotEqual, PtrDiff

* Basic asm/Dis test for OpCopyMemory

* Test asm/dis OpCopyMemory with 2-memory access

Add asm/dis tests for OpCopyMemorySized

Requires grammar update to add second optional memory access operand
to OpCopyMemory and OpCopyMemorySized

* Validate one or two memory accesses on OpCopyMemory*

* Check av/vis on CopyMemory source and target memory access

This is a proposed rule. See
https://gitlab.khronos.org/spirv/SPIR-V/issues/413

* Validate operation for OpSpecConstantOp

* Validate NonWritable decoration

Also permit NonWritable on members of UBO and SSBO.

* SPIR-V 1.4: NonWrtiable can decorate Function and Private vars

* Update optimizer CLI tests for SPIR-V 1.4

* Testing tools: Give expected SPIR-V version in message

* SPIR-V 1.4 validation for entry point interfaces

* Allow only unique interfaces
* Allow all global variables
* Check that all statically used global variables are listed
* new tests

* Add validation fixture CompileFailure

* Add 1.4 validation for pointer comparisons

* New tests

* Validate with image operands SignExtend, ZeroExtend

Since we don't actually know the image texel format, we can't fully
validate.  We need more context.

But we can make sure we allow the new image operands in known-good
cases.

* Validate OpCopyLogical

* Recursively checks subtypes
* new tests

* Add SPIR-V 1.4 tests for NoSignedWrap, NoUnsignedWrap

* Allow scalar conditions in 1.4 with OpSelect

* Allows scalar conditions with vector operands
* new tests

* Validate uniform id scope as an execution scope

* Validate the values of memory and execution scopes are valid scope
values
* new test

* Remove SPIR-V 1.4 Vulkan 1.0 environment

* SPIR-V 1.4 requires Vulkan 1.1

* FIX: include string for spvLog

* FIX: validate nonwritable

* FIX: test case suite for member decorate string

* FIX: test case for hlsl functionality1

* Validation test fixture: ease debugging

* Use binary version for SPIR-V 1.4 specific features

* Switch checks based on the SPIR-V version from the target environment
to instead use the version from the binary
* Moved header parsing into the ValidationState_t constructor (where
version based features are set)
* Added new versions of tests that assemble a 1.3 binary and validate a
1.4 environment

* Fix test for update to SPIR-V 1.4 headers

* Fix formatting

* Ext inst lookup: Add Vulkan 1.1 env with SPIR-V 1.4

* Update spirv-val help

* Operand version checks should use module version

Use the module version instead of the target environment version.

* Fix comment about two-access form of OpCopyMemory
2019-05-07 12:27:18 -04:00
Alastair Donaldson
f3acb955c2 Initial commit for spirv-reduce. (#2056)
Creates a new tool that can be used to reduce failing testcases, similar to creduce.
2018-11-21 14:03:09 -05:00
Jeff Bolz
c06a35b902 Rename PCH macro to spvtools_pch to avoid conflicts with other projects. Also add pch to test/opt. (#2034) 2018-11-07 09:15:04 -05:00
Jeff Bolz
60fac96c6b Enable precompiled headers for spirv-tools(-shared) and some unit tests (#2026) 2018-11-06 09:26:23 -05:00
dan sinclair
9e6f5134d1
Reduce number of test targets (#2024)
This CL takes the various opt unit tests and makes a single executable
instead of one per test. This reduces the number of build targets by
~125 when building with ninja.
2018-11-01 10:19:37 -04:00
Diego Novillo
03000a3a38 Add testing framework for tools.
This forks the testing harness from https://github.com/google/shaderc
to allow testing CLI tools.

New features needed for SPIRV-Tools include:

1- A new PlaceHolder subclass for spirv shaders.  This place holder
   calls spirv-as to convert assembly input into SPIRV bytecode. This is
   required for most tools in SPIRV-Tools.

2- A minimal testing file for testing basic functionality of spirv-opt.

Add tests for all flags in spirv-opt.

1. Adds tests to check that known flags match the names that each pass
   advertises.
2. Adds tests to check that -O, -Os and --legalize-hlsl schedule the
   expected passes.
3. Adds more functionality to Expect classes to support regular
   expression matching on stderr.
4. Add checks for integer arguments to optimization flags.
5. Fixes #1817 by modifying the parsing of integer arguments in
   flags that take them.
6. Fixes -Oconfig file parsing (#1778). It reads every line of the file
   into a string and then parses that string by tokenizing every group of
   characters between whitespaces (using the standard cin reading
   operator).  This mimics shell command-line parsing, but it does not
   support quoting (and I'm not planning to).
2018-08-17 15:03:14 -04:00
dan sinclair
5fc011b453
Move bit_stream, move_to_front and huffman_codec. (#1833)
bit_stream, move_to_front and huffman_codec are only used by
source/tools. Move into that directory to make the usage clearer.
2018-08-14 09:52:05 -04:00
dan sinclair
e3ea909ebe
Simplify MoveToFront (#1806)
This CL removes the templating from the MoveToFront code as all non-test
code uses uint32_t as the variable.
2018-08-07 09:10:25 -04:00
Jaebaek Seo
3b594e1630 Add --time-report to spirv-opt
This patch adds a new option --time-report to spirv-opt.  For each pass
executed by spirv-opt, the flag prints resource utilization for the pass
(CPU time, wall time, RSS and page faults)

This fixes issue #1378
2018-03-20 21:30:06 -04:00
Lei Zhang
1ef6b19260 Migrate to use unified grammar tables
Previously we keep a separate static grammar table for opcodes/
operands per SPIR-V version. This commit changes that to use a
single unified static grammar table for opcodes/operands.

This essentially changes how grammar facts are queried against
a certain target environment. There are only limited filtering
according to the desired target environment; a symbol is
considered as available as long as:

1. The target environment satisfies the minimal requirement of
   the symbol; or
2. There is at least one extension enabling this symbol.

Note that the second rule assumes the extension enabling the
symbol is indeed requested in the SPIR-V code; checking that
should be the validator's work.

Also fixed a few grammar related issues:
* Rounding mode capability requirements are moved to client APIs.
* Reserved symbols not available in any extension is no longer
  recognized by assembler.
2018-03-17 15:25:26 -04:00
Józef Kucia
3013897556 Build SPIRV-Tools as shared library
Add pkg-config file for shared libraries

Properly build SPIRV-Tools DLL

Test C interface with shared library

Set PATH to shared library file for c_interface_shared test

Otherwise, the test won't find SPIRV-Tools-shared.dll.

Do not use private functions when testing with shared library

Make all symbols hidden by default for shared library target
2018-02-07 10:43:32 -05:00
David Neto
59de6100b5 Add asm, dis support for DebugInfo extended instruction set
Add grammar file for DebugInfo extended instruction set
- Each new operand enum kind in extinst.debuginfo.grammar.json maps
  to a new value in spv_operand_type_t.
- Add new concrete enum operand types for DebugInfo

Generate a C header for the DebugInfo extended instruction set

Add table lookup of DebugInfo extended instrutions

Handle the debug info operand types in binary parser,
disassembler, and assembler.

Add DebugInfo round trip tests for assembler, disassembler

Android.mk: Support DebugInfo extended instruction set

The extinst.debuginfo.grammar.json file is currently part of
SPIRV-Tools source.

It contributes operand type enums, so it has to be processed
along with the core grammar files.

We also generate a C header DebugInfo.h.

Add necessary grammar file processing to Android.mk.
2017-12-22 09:39:36 -05:00
David Neto
7ba59ac8dc Force gtest to expose ::testing::Combine
Work around faulty logic in googletest, where ::testing::Combine
is accidentally disabled for VS 2017.
See https://github.com/google/googletest/issues/1352
2017-12-12 08:58:51 -05:00
Lei Zhang
b93c066b0b CMake: allow both SPIRV-Headers and spirv-headers
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/1057
2017-12-05 14:41:31 -05:00
David Neto
76555bd4ba Tests: Add optional dependency on Effcee stateful matcher
Add Effcee as an optional dependency for use in tests.  In future it will
be a required dependency.

Effcee is a stateful pattern matcher that has much of the functionality
of LLVM's FileCheck, except in library form.  Effcee makes it much easier
to write tests for optimization passes.

Demonstrate its use in a test for the strength-reduction pass.

Update README.md with example commands of how to get sources.

Update Appveyor and Travis-CI build rules.

Also: Include test libraries if not SPIRV_SKIP_TESTS
- SPIRV_SKIP_TESTS is implied by SPIRV_SKIP_EXECUTABLES
2017-11-13 11:34:21 -05:00
Steven Perron
720beb161a Generic intrusive linked list class.
This commit is the initial implementation of the intrusive linked list
class.  It includes the implementation in the header files, and unit
test.

The iterators are circular: incrementing end() gives begin() and
decrementing begin() gives end().  Also made it valid to
decrement end().

Expliticly defines move constructor and move assignment
- Visual Studio 2013 does not implicitly generate the move constructor or
  move assignments.  So they need to be explicit, otherwise it will try to
  use the copy constructor, which we explicitly deleted.
- Can't use "= default" either.
  Seems like VS2013 does not support explicitly using the default move
  constructors and move assignments, so I wrote them out.
2017-10-12 12:40:18 -04:00
Pierre Moreau
86627f7b3f Implement Linker (module combiner)
Add extra iterators for ir::Module's sections
Add extra getters to ir::Function
Add a const version of BasicBlock::GetLabelInst()

Use the max of all inputs' version as version

Split debug in debug1 and debug2
- Debug1 instructions have to be placed before debug2 instructions.

Error out if different addressing or memory models are found

Exit early if no binaries were given

Error out if entry points are redeclared

Implement copy ctors for Function and BasicBlock
- Visual Studio ends up generating copy constructors that call deleted
  functions while compiling the linker code, while GCC and clang do not.
  So explicitly write those functions to avoid Visual Studio messing up.

Move removing duplicate capabilities to its own pass

Add functions running on all IDs present in an instruction

Remove duplicate SpvOpExtInstImport

Give default options value for link functions

Remove linkage capability if not making a library

Check types before allowing to link

Detect if two types/variables/functions have different decorations

Remove decorations of imported variables/functions and their types

Add a DecorationManager

Add a method for removing all decorations of id

Add methods for removing operands from instructions

Error out if one of the modules has a non-zero schema

Update README.md to talk about the linker

Do not freak out if an imported built-in variable has no export
2017-10-06 18:33:53 -04:00
Andrey Tuganov
73e8dac5b9 Added compression tool tools/spirv-markv. Work in progress.
Command line application is located at tools/spirv-markv
API at include/spirv-tools/markv.h

At the moment only very basic compression is implemented, mostly varint.
Scope of supported SPIR-V opcodes is also limited.

Using a simple move-to-front implementation instead of encoding mapped
ids.

Work in progress:
- Does not cover all of SPIR-V
- Does not promise compatibility of compression/decompression across
  different versions of the code.
2017-06-30 12:22:48 -04:00
Andrey Tuganov
8d3882a408 Added log(n) move-to-front implementation
The implementation is based on AVL and order statistic tree.

It accepts all kinds of values and the implementation
doesn't expect the behaviour to be consistent with id coding.

Intended by SPIR-V compression algorithms.
2017-06-29 16:16:18 -04:00
Andrey Tuganov
40a2829611 Added Huffman codec to utils
Attached ids to Huffman nodes for deterministic internal node
comparison.
2017-06-29 14:51:01 -04:00
Andrey Tuganov
3eb716cae4 Added bit stream utils
Also added generalized zigzag coding.
- Due to signed integers being mostly non-negative, improved zigzag coding
  to favor positive values.
2017-05-12 17:08:33 -04:00
Andrey Tuganov
b173d1c3cd Added option --preserve-numeric-ids to tools/spirv-as
Added options to tools::Assemble
2017-04-13 14:13:33 -04:00
Andrey Tuganov
4f216402ba Added statistical analysis tool (tool/stats)
Currently analyzes percentages of the following:
- version (how many 1.0 or 1.1)
- generator
- extension
- capability
- opcode prevalence
2017-04-10 17:15:04 -04:00
Lei Zhang
5c3c054c1f Group targets into folders
This plays nicer with IDEs.
2017-04-06 09:44:33 -04:00
Andrey Tuganov
c804c125c5 Autogenererating enum-string mappings from grammar
Autogenerating the following code:
- extension enum
- extension-to-string
- string-to-extension
- capability-to-string

Capability mapping table will not compile if incomplete.
TODO: Use "spirv-latest-version.h" instead of 1.1.

Added function to generate capability tables for tests.
2017-03-16 15:29:17 -04:00
Andrey Tuganov
b011633171 Validator checks if operands require extension
If required extension is not found, returning new error
SPV_ERROR_MISSING_EXTENSION.
2017-03-14 13:39:43 -04:00
Andrey Tuganov
1fb8c37b57 Refactored enum_set
- removed forgotten file enum_set.cpp
- added IsEmpty and HasAnyOf
- hidden unsafe functions Add(uint32_t), Contains(uint32_t)
- added new tests
2017-03-10 13:38:32 -05:00
David Neto
37422e9dba Support building under CYGWIN
- Parse CHANGES file with Universal Python line endings in case
  the source tree was checked out with Windows line endings.
- Use our own clone of strnlen_s which might not be available
  everywhere.

Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/508
2016-12-21 10:34:23 -05:00
Lei Zhang
4f57e140bf Renaming test files to comply with Google style guide.
Also posfixing test files with `_test' to make it more clear.
2016-11-07 14:41:51 -05:00
Lei Zhang
bfd7cd6d6e Add utility macros for logging messages.
Add the following macros for logging purpose:
* SPIRV_ASSERT
* SPIRV_DEBUG
* SPIRV_UNIMPLEMENTED
* SPIRV_UNREACHABLE

The last two is always turned on, while the first two can only
be turned on in debug build.
2016-09-16 09:56:34 -04:00
Lei Zhang
755f97f534 Add a callback mechanism for communicating messages to callers.
Every time an event happens in the library that the user should be
aware of, the callback will be invoked.

The existing diagnostic mechanism is hijacked internally by a
callback that creates an diagnostic object each time an event
happens.
2016-09-15 12:35:48 -04:00
qining
1773b95737 Pull out the number parsing logic
Pull out the number parsing logic from
AssemblyContext::binaryEncodeNumericLiteral() to utilities.

The new utility function: `ParseAndEncodeNumber()` now accepts:
  * number text to parse
  * number type
  * a emit function, which is a function which will be called with each
    parsed uint32 word.
  * a pointer to std::string to be overwritten with error messages.
    (pass nullptr if expect no error message)
and returns:
  * an enum result type to indicate the status

Type/Structs moved to utility:
  * template<typename T> class ClampToZeroIfUnsignedType

New type:
* enum EncodeNumberStatus: success or error code
* NumberType: hold the number type information for the number to be parsed.
 * several helper functions are also added for NumberType.

Functions moved to utility:
  * Helpers:
    * template<typename T> checkRangeAndIfHexThenSignExtend() -> CheckRangeAndIfHex....()
  * Interfaces:
    * template<typename T> parseNumber() -> ParseNumber()
    * binaryEncodeIntegerLiteral() -> ParseAndEncodeIntegerNumber()
    * binaryEncodeFloatingPointLiteral() -> ParseAndEncodeFloatingPointNumber()
    * binaryEncodeNumericLiteral() -> ParseAndEncodeNumber()

Tests added/moved to test/ParseNumber.cpp, including tests for:
* ParseNumber(): This is moved from TextToBinary.cpp to ParseNumber.cpp
* ParseAndEncodeIntegerNumber(): New added
* ParseAndEncodeFloatingPointNumber(): New added
* ParseAndEncodeNumber(): New added

Note that the error messages are kept almost the same as before, but
they may be inappropriate for an utility function. Those will be fixed
in another CL.
2016-09-08 18:42:54 -04:00
Lei Zhang
a35919c008 Merge diagnostic tests and single them out in a separate binary. 2016-09-08 17:26:53 -04:00
Lei Zhang
219f0cc13c Create separate binaries for validation tests. 2016-09-07 17:43:15 -04:00
Lei Zhang
ef600df16e Prefix test binaries with test_, rename UnitSPIRV, update README. 2016-09-07 17:26:18 -04:00
David Neto
9fc8658ef3 Relicense SPIRV-Tools under Apache 2.0
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/383

Finalize v2016.4
2016-09-02 10:00:29 -04:00
David Neto
47f2a1414b Add libspirv::CapabilitySet
It's optimized for the common case, where capabilities have value
at most 63.
2016-08-30 15:41:39 -04:00
David Neto
2607bbf30b Add Instruction cbegin and cend to access operands 2016-08-25 15:24:32 -04:00
Lei Zhang
190b0d3162 Use NEW behavior for policy CMP0054.
The NEW behavior is to not dereference variables or interpret keywords
that have been quoted or bracketed.

For more information, see
https://cmake.org/cmake/help/v3.1/policy/CMP0054.html.

This is to suppress a warning when using CMake 3.1.3+.
2016-08-09 16:09:59 -04:00
David Neto
c296701964 Add target envs for OpenCL, OpenGL
Covers: OpenCL 2.1, OpenCL 2.2
Covers: OpenGL 4.0, OpenGL 4.1, OpenGL 4.2, OpenGL 4.3, OpenGL 4.5.
The OpenGL environments assume the use of GL_ARB_gl_spirv, of course.
2016-08-05 18:19:30 -04:00
Lei Zhang
71fac5a29c Check configuration and gmock existence when adding unit tests. 2016-08-05 14:38:18 -04:00
David Neto
0bdcc23f7e Disassembling: map IDs to friendly names.
Add a FriendlyNameMapper to deduce friendly names for IDs based on
OpName, type structure, etc.
2016-07-19 16:28:17 -04:00