Commit Graph

68 Commits

Author SHA1 Message Date
David Neto
64f2750e5d
Update copyright checker for 2019 (#2256) 2018-12-28 17:27:53 -05:00
dan sinclair
aa53271b9c
Script to fixup fuzzer results (#1792) 2018-08-02 20:28:40 -04:00
dan sinclair
3dad1cda11
Change libspirv to spvtools namespace (#1678)
This CL changes all of the libspirv namespace code to spvtools to match
the rest of the code base.
2018-07-07 09:38:00 -04:00
Greg Roth
878b3b400b check_symbol_exports on Python3, Mac
subprocess.Popen returns byte data by default. Python2 was happy
to try to execute string operations on such data and hope for the
best, but python3 is more persnickety. Luckily, there's a simple
way to indicate to the Popen class that text data is wanted that
benefits the script. Just specifying universal_newlines will cause
the returned data to be text and also convert any system-specific
newlines to '\n' which the script relies on anyway.

Enabled on Mac as an incidental change after confirming that the
script works there just as well as it does on Linux.

It probably works on FreeBSD too, but I retired my BSD system years
ago. So I have no way to check.

Don't run it on Windows.
- It didn't work after all. It was just detecting non-posix and
  returning success.
2018-06-27 18:08:18 -04:00
David Neto
082b8b08f1 More automatic extension support
Update grammar table generation:
- Get extensions from instructions, not just operand-kinds
- Don't explicitly list extensions that come from the SPIR-V core
  grammar or from a KHR extended instruction set grammar.

This makes it easier to support new extensions since the recommended
extension strategy is to add instructions to the core grammar file.

Also, test the validator has trivial support for passing through
the extensions SPV_NV_shader_subgroup_partitioned and
SPV_EXT_descriptor_indexing.
2018-04-05 18:36:08 -04:00
Jeff Bolz
efb41541c3 Add SPV_NV_shader_subgroup_partitioned to EXTENSIONS_FROM_SPIRV_REGISTRY 2018-04-03 12:56:38 -04:00
Lei Zhang
ddbaf32460 Use standard SPIR-V version scheme for version requirement
Previously we use symbols in spv_target_env as the minimum version
requirements for features. That makes version check implicitly
relies on the order of entries in the spv_target_env enum, which
also contains client APIs. Instead, we should use the standard
scheme for constructing SPIR-V version; and by doing that we can
also map client API entries to universial SPIR-V versions.
2018-03-29 12:06:54 -04:00
Lei Zhang
1ef6b19260 Migrate to use unified grammar tables
Previously we keep a separate static grammar table for opcodes/
operands per SPIR-V version. This commit changes that to use a
single unified static grammar table for opcodes/operands.

This essentially changes how grammar facts are queried against
a certain target environment. There are only limited filtering
according to the desired target environment; a symbol is
considered as available as long as:

1. The target environment satisfies the minimal requirement of
   the symbol; or
2. There is at least one extension enabling this symbol.

Note that the second rule assumes the extension enabling the
symbol is indeed requested in the SPIR-V code; checking that
should be the validator's work.

Also fixed a few grammar related issues:
* Rounding mode capability requirements are moved to client APIs.
* Reserved symbols not available in any extension is no longer
  recognized by assembler.
2018-03-17 15:25:26 -04:00
Rex Xu
314cfa29b2 Add missing SPV extension strings 2018-03-08 21:54:00 +08:00
David Neto
5f69f75126 Support SPV_GOOGLE_decorate_string and SPV_GOOGLE_hlsl_functionality1
This commit add assembling, disassembling, and basic validation for two
Google extensions to better support HLSL translation.
2018-03-05 13:34:13 -05:00
Lei Zhang
f3a10470d3
Avoid using static unordered_map (#1304)
unordered_map is not POD. Using it as static may cause problems
when operator new() and operator delete() is customized.

Also changed some function signatures to use const char* instead
of std::string, which will give caller the flexibility to avoid
creating a std::string.
2018-02-15 10:19:15 -05:00
David Neto
886859159e Fix generation of Vim syntax file 2018-02-09 17:47:51 -05:00
Alan Baker
2e93e806e4 Initial implementation of if conversion
* Handles simple cases only
* Identifies phis in blocks with two predecessors and attempts to
convert the phi to an select
 * does not perform code motion currently so the converted values must
 dominate the join point (e.g. can't be defined in the branches)
 * limited for now to two predecessors, but can be extended to handle
 more cases
* Adding if conversion to -O and -Os
2018-01-25 09:42:00 -08:00
Alan Baker
672494da13 Adding ostream operators for IR structures
* Added for Instruction, BasicBlock, Function and Module
* Uses new disassembly functionality that can disassemble individual
instructions
 * For debug use only (no caching is done)
 * Each output converts module to binary, parses and outputs an
 individual instruction
* Added a test for whole module output
* Disabling Microsoft checked iterator warnings
* Updated check_copyright.py to accept 2018
2018-01-12 11:19:58 -05:00
Pierre Moreau
120ddffb41 Ignore clang-format-diff.py from copyrights check
Having that script locally, for example to run check_code_format.sh
before sending a pull request, would cause the copyright notices check
to fail.
2018-01-05 10:19:10 -05:00
Leo Izen
b185a3d808 utils/generate_grammar_tables.py: use spaces for indentation
One part of this file had indented using tabs. This causes newer
versions of python to complain. This changes the indentation to
consistently use four spaces throughout the file, to be consistent
with the rest of the file and the rest of the project.
2017-12-29 08:59:40 -05:00
David Neto
59de6100b5 Add asm, dis support for DebugInfo extended instruction set
Add grammar file for DebugInfo extended instruction set
- Each new operand enum kind in extinst.debuginfo.grammar.json maps
  to a new value in spv_operand_type_t.
- Add new concrete enum operand types for DebugInfo

Generate a C header for the DebugInfo extended instruction set

Add table lookup of DebugInfo extended instrutions

Handle the debug info operand types in binary parser,
disassembler, and assembler.

Add DebugInfo round trip tests for assembler, disassembler

Android.mk: Support DebugInfo extended instruction set

The extinst.debuginfo.grammar.json file is currently part of
SPIRV-Tools source.

It contributes operand type enums, so it has to be processed
along with the core grammar files.

We also generate a C header DebugInfo.h.

Add necessary grammar file processing to Android.mk.
2017-12-22 09:39:36 -05:00
Lei Zhang
235ef34571 Travis CI: add clang-format check
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/1029
2017-12-05 08:02:33 -05:00
David Neto
d9129f00a5 Test for pollution of the global namespace
Works on Linux only for now.  That's a good start.

Move ValidateBinaryUsingContextAndValidationState into anonymous
namespace in source/validate.cpp.
2017-11-23 21:27:21 -05:00
Steven Perron
b1b494835d Fix copyright check when the build directory is in the source directory.
The IDE I am using places the cmake build directory inside the source
directory.  When that happens the copyright check fails because the
build process creates a file CMakeCXXCompilerId.cpp.  That file does
not have the copyright information, so the test fails.

I want this file to be ignored, so I added its directory, CompilerIdCXX,
to the list of directories to be ignored.
2017-10-31 11:57:43 -04:00
Jesus Carabano
f063f91d24 Use std::lower_bound for opcode lookup
Use std::lower_bound for opcode-to-string

Stable sort the generated instruction table.
2017-10-28 18:34:01 -04:00
Lei Zhang
063dbea0f1 Turn all function static non-POD variables into global POD variables
Function static non-POD data causes problems with DLL lifetime.
This pull request turns all static info tables into strict POD
tables. Specifically, the capabilities/extensions field of
opcode/operand/extended-instruction table are turned into two
fields, one for the count and the other a pointer to an array of
capabilities/extensions. CapabilitySet/EnumSet are not used in
the static table anymore, but they are still used for checking
inclusion by constructing on the fly, which should be cheap for
the majority cases.

Also moves all these tables into the global namespace to avoid
C++11 function static thread-safe initialization overhead.
2017-10-25 15:44:19 -04:00
Pierre Moreau
86627f7b3f Implement Linker (module combiner)
Add extra iterators for ir::Module's sections
Add extra getters to ir::Function
Add a const version of BasicBlock::GetLabelInst()

Use the max of all inputs' version as version

Split debug in debug1 and debug2
- Debug1 instructions have to be placed before debug2 instructions.

Error out if different addressing or memory models are found

Exit early if no binaries were given

Error out if entry points are redeclared

Implement copy ctors for Function and BasicBlock
- Visual Studio ends up generating copy constructors that call deleted
  functions while compiling the linker code, while GCC and clang do not.
  So explicitly write those functions to avoid Visual Studio messing up.

Move removing duplicate capabilities to its own pass

Add functions running on all IDs present in an instruction

Remove duplicate SpvOpExtInstImport

Give default options value for link functions

Remove linkage capability if not making a library

Check types before allowing to link

Detect if two types/variables/functions have different decorations

Remove decorations of imported variables/functions and their types

Add a DecorationManager

Add a method for removing all decorations of id

Add methods for removing operands from instructions

Error out if one of the modules has a non-zero schema

Update README.md to talk about the linker

Do not freak out if an imported built-in variable has no export
2017-10-06 18:33:53 -04:00
Lei Zhang
16981f87fe Avoid using global static variables
Previously we have several grammar tables defined as global static
variables and these grammar table entries contains non-POD struct
fields (CapabilitySet/ExtensionSet). The initialization of these
non-POD struct fields may require calling operator new. If used
as a library and the caller defines its own operator new, things
can screw up.

This pull request changes all global static variables into
function static variables, which is lazy evaluated in a thread
safe way as guaranteed by C++11.
2017-09-26 10:59:15 -04:00
David Neto
a91cecfefc Recognize SPV_AMD_shader_fragment_mask 2017-09-14 10:37:18 -04:00
David Neto
0d3b8329a4 Make enums for all currently published extensions
Use the list from the SPIR-V registry page.  Also, capture it as
a string so it's much easier to update via copy-paste.

The validator will accept modules that declare these known
extensions.  However, we might not know about new tokens or
instructions declared in them.  For that we need grammar updates
applied to SPIRV-Headers.
2017-09-02 15:10:52 -04:00
Rex Xu
5fbbadca4e Add support for SPV AMD extensions 2017-06-21 15:08:07 -04:00
Andrey Tuganov
20fe946666 Added extension SPV_VALIDATOR_ignore_type_decl_unique
If extension is present in the module, validator will ignore the "type
declaration must be unique" rule.
2017-05-25 11:36:48 -04:00
David Neto
bf68c81426 Support SPV_KHR_storage_buffer_storage_class
Includes interaction with SPV_KHR_variable_pointers

TODO: Update capabilitly dependency tests for new capabilities.
2017-04-07 09:54:38 -04:00
David Neto
7fe8a57a5b Support SPV_AMD_gcn_shader
Supported in assembler, disassembler, and binary parser.

The validator does not check SPV_AMD_gcn_shader validation rules
beyond parsing the extension.

Adds generic support for generating instruction tables for vendor
extensions.

Adds generic support for extensions the validator should recognize
(but not check) but which aren't derived from the SPIR-V core
grammar file.

Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/594
2017-03-23 16:32:35 -04:00
David Neto
654feb32f2 Do nothing if trying to make the current dir 2017-03-20 11:23:35 -04:00
Andrey Tuganov
c804c125c5 Autogenererating enum-string mappings from grammar
Autogenerating the following code:
- extension enum
- extension-to-string
- string-to-extension
- capability-to-string

Capability mapping table will not compile if incomplete.
TODO: Use "spirv-latest-version.h" instead of 1.1.

Added function to generate capability tables for tests.
2017-03-16 15:29:17 -04:00
Andrey Tuganov
12f5509734 Fix extension grammar parsing
Extensions were not parsed correctly due to a typo in the Python script.
2017-03-09 15:32:42 -05:00
David Neto
699943d297 Say why we allow override of build timestamp
It's for reproducible builds.

Also, fix Python formatting.
2017-01-17 15:42:02 -05:00
Bernhard M. Wiedemann
29b37fb310 make build reproducible
see https://reproducible-builds.org/ for why this is good
2017-01-17 19:36:39 +01:00
Ehsan Nasiri
fcf7df069d Adding decoration class and tests.
* Added the decoration class as well as the code that registers the
decorations for each <id> and also decorations for struct members.

* Added unit tests for decorations in ValidationState as well as
decoration id tests.
2017-01-16 16:52:08 -05:00
David Neto
558ac1e5aa Allow trailing whitespace on version lines in CHANGES 2017-01-06 13:18:50 -05:00
David Neto
37422e9dba Support building under CYGWIN
- Parse CHANGES file with Universal Python line endings in case
  the source tree was checked out with Windows line endings.
- Use our own clone of strnlen_s which might not be available
  everywhere.

Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/508
2016-12-21 10:34:23 -05:00
David Neto
f8cc397573 Generate spvasm.vim
Generate a vim syntax file for SPIR-V assembly.
Copy the resulting spvasm.vim into your $HOME/.vim/syntax directory
to get syntax highlighting in Vim.

Also, suggest that the grammar file include information
about what opcodes can be used in OpSpecConstantOp.
2016-12-20 11:36:41 -05:00
David Neto
5a0b5ca4a7 Get vendor tool info from the SPIR-V registry file
Update old tests to use officially registered vendor names;
this affected "Codeplay".
2016-12-09 14:01:43 -05:00
David Neto
9382035a22 Add basic support for SPV_KHR_shader_ballot
Requires use of SPIRV-Headers that has support
for SPV_KHR_shader_ballot.

Adds assembler, disassembler, binary parser support.

Adds general support for allowing an operand to be
only enabled by a set of extensions.

TODO: Validator support for extension checking.
2016-09-16 16:38:36 -04:00
steve-lunarg
6accefc3a4 add spirv-dis/spirv-as support for emacs when loading .spv binaries 2016-09-11 09:55:15 -06:00
David Neto
9fc8658ef3 Relicense SPIRV-Tools under Apache 2.0
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/383

Finalize v2016.4
2016-09-02 10:00:29 -04:00
David Neto
6f35405a97 Add check_copyright to check or update copyright.
It will also replace an MIT or Khronos free use license with
the Apache license notice.
2016-09-02 10:00:29 -04:00
David Neto
0e001b70e4 Fix Google copyright lines 2016-09-02 10:00:29 -04:00
David Neto
273920c554 Handle capabilities larger than value 63
Use libspirv::CapabilitySet instead of a 64-bit mask.

Remove dead function spvOpcodeRequiresCapability and its tests.

The JSON grammar parser is simplified since it just writes the
list of capabilities as a braced list, and takes advantage of
the CapabilitySet intializer-list constructor.
2016-08-30 15:41:39 -04:00
David Neto
033b7d00f0 Avoid race on mkdir
Should fix https://github.com/KhronosGroup/SPIRV-Tools/issues/340
2016-08-15 11:14:34 -04:00
Lei Zhang
0d8ddd0b75 Create directory first if not existing and always refresh. 2016-08-05 10:23:41 -04:00
Lei Zhang
c6465fb220 Avoid redefining builtin dir. 2016-08-05 10:23:41 -04:00
David Neto
1a18739650 Fix ExtInst parsing: no IdRef* at end
The operands following the extended instruction literal
number are determined by the extended instruction itself.
So drop the zero-or-more IdRef pattern at the end of OpExtInst.

It's arguable whether this should actually be a grammar fix.  I've
chosen to patch this in SPIRV-Tools instead of in the grammar file.

Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/233

Also fix two test cases for OpenCL extended instructions.  These
errors of supplying too many operands are now detected.
2016-06-15 10:00:06 -04:00