Commit Graph

113 Commits

Author SHA1 Message Date
Alastair Donaldson
6578899781
spirv-fuzz: Manage available instructions efficiently (#4177)
Introduces a data structure for efficient management of available
instructions in the fuzzer.
2021-03-20 18:51:18 +00:00
Corentin Wallez
4a59fd4763
Fix -Wextra-semi-stmt -Wsuggest-destructor-override -Wdeprecated-copy-dtor (#4164)
* Fix -Wextra-semi-stmt
* Fix -Wsuggest-destructor-override
* Fix -Wdeprecated-copy-dtor
2021-03-09 13:16:43 +00:00
Ben Clayton
d28186db93
Fixes for the vscode language server extension (#4150)
Fix URL to the `OpenCL.DebugInfo.100` grammar file, which moved from `SPIRV-Tools` to `SPIRV-Headers`.

Add a `go.mod` and `go.sum` file.
Go has now fully migrated to using modules, and relative imports no longer work.
Fixup all the relative imports to use a fully qualified path based on the module name.

Fix the `.exe` executable extension on windows.

Update the generated grammar files.

Update the `README.md` with more information.

Disable generation of the log file. Seems to end up in the workspaceDir on windows, not the executable path
2021-02-26 18:07:00 +00:00
Ben Clayton
8383bd5d6f
Migrate all Kokoro build scripts over to use the docker VM image (#4114)
* Work around GCC-9 warning treated as error

```
../source/opt/instruction.h:101:23: error: '*((void*)& operand +32)' may be used uninitialized in this function [-Werror=maybe-uninitialized]
  101 |     uint64_t result = uint64_t(words[0]);
```

* Migrate all Kokoro build scripts over to use the docker VM image

Required updating the NDK SDK and build scripts, as well as the check_copyright for handling 2021.
2021-01-18 13:36:26 -05:00
Ryan Harrison
09d1fea589
Force using Python 3 git-sync-deps (#4067)
This scripts fails to run in Python 2. For systems that have both
Python 2 and 3 installed, depending on your PATH the existing code may
or may not run. Explictly using Python 3 to avoid this.
2020-12-10 13:39:37 -05:00
David Neto
cd590fa334
Update MeshShadingNV dependencies (and land Ray tracing updates) (#4028)
* Update to final ray tracing extensions

Drop Provisional from ray tracing enums
    sed -ie 's/RayQueryProvisionalKHR/RayQueryKHR/g' **/*
    sed -ie 's/RayTracingProvisionalKHR/RayTracingKHR/g' **/*
Add terminator support for SpvOpIgnoreIntersectionKHR and SpvOpTerminateRayKHR
Update deps for SPIRV-Headers

* Update capability dependencies for MeshShadingNV

Accommodate https://github.com/KhronosGroup/SPIRV-Headers/pull/180

MeshShadingNV: enables PrimitiveId, Layer, and ViewportIndex

Co-authored-by: Daniel Koch <dkoch@nvidia.com>
2020-11-23 12:23:54 -05:00
Alastair Donaldson
ed9863e46e
Favour 'integrity' over 'coherence' as a replacement for 'sanity'. (#3619) 2020-09-10 09:52:21 -04:00
Alastair Donaldson
f9b088fe0d
Avoid use of 'sanity' and 'sanity check' in the code base (#3585)
In line with:

  https://source.android.com/setup/contribute/respectful-code

this change uses the terms 'coherence' and 'coherence check' where
'sanity' and 'sanity check' were previously used.
2020-07-28 23:55:02 -04:00
Ryan Harrison
969f028647
Change DEPS rolling script to point at external/ (#3584) 2020-07-23 16:40:05 -04:00
Stefano Milizia
5f8cdd8b45
Implement transformation to record synonymous constants. (#3494)
Adds a fact-only transformation that records that two constants in the module are synonymous.
2020-07-10 14:02:14 +01:00
Ryan Harrison
36b5bb701d
Add support for different default/trunks in roll-deps (#3442) 2020-06-22 13:24:43 -04:00
dan sinclair
52a5f074e9
Update access control lists. (#3433)
This CL updates the access control lists used in SPIRV-Tools to the more
descriptive allow/deny naming.
2020-06-15 13:20:40 -04:00
Ben Clayton
23d68608b0
vscode: Handle '|' chains on BitEnum / ValueEnum (#3309)
* vscode: Regenerate schema with new opcodes.

* vscode: Handle '|' chains on BitEnum / ValueEnum

These were highlighting as errors.
2020-04-27 16:14:48 -04:00
Aaron Franke
7f341ffee6
Make file formatting comply with POSIX standards (#3242)
Add newlines to two files
2020-03-23 10:20:00 -04:00
Daniel Koch
5a97e3a391
Add support for KHR_ray_{query,tracing} extensions (#3235)
Update validator for SPV_KHR_ray_tracing.

* Added handling for new enum types
* Add SpvScopeShaderCallKHR as a valid scope
* update spirv-headers

Co-authored-by: alelenv <alele@nvidia.com>
Co-authored-by: Torosdagli <ntorosda@amd.com>
Co-authored-by: Tobias Hector <tobias.hector@amd.com>
Co-authored-by: Steven Perron <stevenperron@google.com>
2020-03-17 15:30:19 -04:00
Vasyl Teliman
533af49812
spirv-fuzz: Add fuzzer pass to permute function parameters (#3212)
Fixes #3194.
2020-03-08 14:27:05 +00:00
Alastair Donaldson
66a682b6a8
spirv-fuzz: Add swap commutable operands transformation (#3205)
In this PR, the classes that represent the swap commutable operands
transformation and the fuzzer pass were implemented.

Fixes #3205.
2020-03-05 08:18:39 +00:00
David Neto
e1688b60ca
Avoid use of Python distutils.dir_util (#3203)
Required for compatibility with (some?) Python3 installations.
2020-02-28 16:50:06 -05:00
Geoff Lang
fb6e3e48d5
Combine extinst-name and extinst-output-base into one arg. (#3200)
* Combine the extinst-name and extinst-output-base into one arg.

Some build systems such as Android blueprints require that the inputs
and outputs of generator scripts are all provided as arguments.  These
two arguments to generate_language_headers.py are combined to form the
output path in the script.  This change simply lets the user provide the
whole output path as an argument.

* Fix typo in build_defs.bzl and update Android.mk
2020-02-25 00:46:52 -05:00
Ben Clayton
79f8caf915
vscode: Add missing fields to schema.Opcode (#3169)
These were declared in the `Opcode` struct type, but were never assigned.
They're not actually used by the language server, but I reused this go schema package for a local experiment and found they were missing.
2020-02-18 06:01:56 -08:00
Ben Clayton
9e52bc0d0c
utils/vscode: LSP - Support OpExtInst (#3140)
This adds language server support for the `GLSL.std.450`, `OpenCL.std` and `OpenCL.DebugInfo.100` extension instructions.
2020-02-04 11:21:10 -05:00
Alastair Donaldson
fdd0c87765
Update script that checks copyright years. (#3173) 2020-02-04 08:57:47 -05:00
David Neto
fb2e819819
Fix GN build for OpenCL.DebugInfo.100 update (#3134)
The utils/generate_grammar_tables.py vendor-operand-kind-prefix
can now take a value of "...nil..." to signify the empty string.
That's needed because GN doesn't quote empty string arguments
correctly down into its action commands.

Fixes #3120
2020-01-08 17:27:38 -05:00
Ben Clayton
c86dc0bcdb Fix bad parameter names in error message (#3129) 2020-01-08 09:54:45 -05:00
David Neto
64f36ea529
Support OpenCL.DebugInfo.100 extended instruction set (#3080)
* Clone opencl.debuginfo.100 grammar from debuginfo grammar

Update version number to 200 revision 2

* Apply content from OpenCL.DebugInfo.100 extension text

* Rename grammar file

* Support OpenCL.DebugInfo.100 extended instructions

Add support for prefixing operand type names, to disambiguate
them between different instruction sets.

* Add tests for OpenCL.DebugInfo.100

* Support lookup of OpenCL.DebugInfo.100 extinst

* Add tests for enum values

* Recognize 2017-2019 as copyright date range

* Android.mk: support OpenCL.DebugInfo.100 extended instruction set

Also, stop generating core instruction tables for non-unified1 versions
of the grammar.

* Imported entity operand type is concrete

* Bazel: Suppoort OpenCL.DebugInfo.100

* BUILD.gn: Support OpenCL.DebugInfo.100
2019-12-19 17:16:26 -05:00
David Neto
e70b009b0f
Add support for SPV_KHR_non_semantic_info (#3110)
Add support for SPV_KHR_non_semantic_info

This entails a couple of changes:

- Allowing unknown OpExtInstImport that begin with the prefix `NonSemantic.`
- Allowing OpExtInst that reference any of those sets to contain unknown
  ext inst instruction numbers, and assume the format is always a series of IDs
  as guaranteed by the extension.
- Allowing those OpExtInst to appear in the types/variables/constants section.
- Not stripping OpString in the --strip-debug pass, since it may be referenced
  by these non-semantic OpExtInsts.
- Stripping them instead in the --strip-reflect pass.

* Add adjacency validation of non-semantic OpExtInst

- We validate and test that OpExtInst cannot appear before or between
  OpPhi instructions, or before/between OpFunctionParameter
  instructions.

* Change non-semantic extinst type to single value

* Add helper function spvExtInstIsNonSemantic() which will check if the extinst
  set is non-semantic or not, either the unknown generic value or any future
  recognised non-semantic set.

* Add test of a complex non-semantic extinst

* Use DefUseManager in StripDebugInfoPass to strip some OpStrings

* Any OpString used by a non-semantic instruction cannot be stripped, all others
  can so we search for uses to see if each string can be removed.
* We only do this if the non-semantic debug info extension is enabled, otherwise
  all strings can be trivially removed.

* Silence -Winconsistent-missing-override in protobufs
2019-12-18 18:10:29 -05:00
Ben Clayton
f31f26f739 utils/vscode: Add install.bat (#3071)
Installs the language server on windows.
Also add info to / correct `README.md`.
2019-11-26 16:23:57 -05:00
Ben Clayton
cdd6829a3e utils/vscode: Add SPIR-V language server support
Supports:
* Syntax highlighting.
* Red squiggles for parse errors.
* Basic sanity checking of operand types.
* Jump to definition of id.
* Find all uses of id.
* Rename id.
* Hover definition of id.
* Format file.
2019-11-13 13:48:11 -05:00
Ben Clayton
b96c9a057e utils/vscode: Add go language server packages.
The utils/vscode/src/lsp directory contains code forked from https://github.com/golang/tools/tree/master/internal/lsp.

This code has been modified to remove additional, unneeded features and dependencies.

Submitted on behalf of a third-party: The Go Authors
2019-11-13 13:48:11 -05:00
David Neto
3e4abc9ac3
Try to fix go linter: exported constant comment (#3026)
The linter wants the comment on an exported constant to
begin with the name of the constant.
2019-11-08 14:12:39 -05:00
David Neto
8a35bf030b Python 3: remove deprecated U qualifier on 'open' (#3019)
The U qualifier (universal line endings) has no effect
in Python3. Remove its use to avoid a Python warning.
2019-11-07 08:39:09 -05:00
David Neto
d9fbf02348
Fix Go lint issues: comment exported symbols (#3014) 2019-11-05 17:41:10 -05:00
Chris Lamb
178c4ddea5 Correct "environment" typo (#3007) 2019-11-04 21:01:27 -05:00
Ben Clayton
7e2cba6a52 utils/vscode: Change assembly file ext to .spvasm (#2995)
Instead of `.spirv`.
`.spvasm` is the official extension for these textual files.
2019-10-29 11:16:17 -04:00
Ben Clayton
42f8852381 utils: Add a vscode extension for SPIR-V disassembly files (#2987)
This is a very simple Visual Studio code extension that adds syntax highlighting to .spirv disassembly files.
2019-10-28 10:06:14 -07:00
Chris Lamb
e8c3f9b0b4 Ensure timestamp does not vary with timezone. (#2982) 2019-10-18 17:08:16 -04:00
Raun Krisch
99793fa67d Adding valilidation checks for OpEntryPoint duplicate names and execution mode (#2862) 2019-09-16 19:13:30 -04:00
alan-baker
5a48c0da15 SPIRV-Tools support for SPIR-V 1.5 (#2865)
* Ensure same enum values have consistent extension lists

* val: fix checking of capabilities

The operand for an OpCapability should only be
checked for the extension or core version.
The InstructionPass registers a capability, and all its implied
sub-capabilities before actually checking the operand to an
OpCapability.

* Add basic support for SPIR-V 1.5

- Adds SPV_ENV_UNIVERSAL_1_5
- Command line tools default to spv1.5 environment
- SPIR-V 1.5 incorporates several extensions.  Now the disassembler
  prefers outputing the non-EXT or non-KHR names.  This requires
  updates to many tests, to make strings match again.
- Command line tests: Expect SPIR-V 1.5 by default

* Test validation of SPIR-V 1.5 incorporated extensions

Starting with 1.5, incorporated features no longer require
the associated OpExtension instruction.
2019-09-13 14:59:02 -04:00
Ryan Harrison
65f49dfc39
Remove unneeded future imports (#2739)
Also, adds explicitly setting python executable in the NDK build script, rewrites 
some Python2-isms to 3isms, and formats some code.

Fixes #2738
2019-07-24 15:29:38 -04:00
Ryan Harrison
69b9459925 Add infrastructure for maintaining and using DEPS (#2684)
This adds in a script for updating DEPS and another for pulling in the
specified versions in the DEPS file.

DEPS entries that are not part of the documented build processs are
removed. Using the documented cmake process with them made for a very
sad state of one's checkout. If this removal breaks workflows we can
revist this, since the state is saved in git.

DEPS are rolled forward, so that this is taking use to a known good
state. These look like scary big rolls, but in reality the bots and
most people are running near HEAD for external/, so this integration
should already have been tested.

Summary of DEPS roll:
+ Rolling 3 dependencies
+ Roll external/effcee/ 04b624799..b83b58d17 (14 commits)
+ Roll external/googletest/ 98a0d007d..2f42d769a (576 commits)
+ Roll external/re2/ 6cf8ccd82..848dfb7e1 (90 commits)
+ Created with:
+   roll-dep external/effcee external/googletest external/re2 external/spirv-headers

Fixes #2665
2019-06-20 14:57:28 -04:00
Alastair Donaldson
fe9f870130
Add library for spirv-fuzz (#2618)
Adds a library for spirv-fuzz, consisting of a Fuzzer class that will
transform a module with respect to (a) facts about the module provided
via a FactManager class, and (b) a source of random numbers and
parameters to control the transformation process provided via a
FuzzerContext class.  Transformations will be applied via classes that
implement a FuzzerPass interface, and both facts and transformations
will be represented via protobuf messages.  Currently there are no
concrete facts, transformations nor fuzzer passes; these will follow.
2019-05-27 14:34:55 +01:00
Kévin Petit
4c73ebc46f Enable the base branch to be overridden in the code format check script (#2607)
Default to master.

Signed-off-by: Kevin Petit <kevin.petit@arm.com>
2019-05-24 14:33:43 -04:00
alan-baker
45fb696668 Use last version (#2578)
* Use grammar last version

Fixes #2560

* Parse last version and use it in checks

* Update grammar header generation

* Fix NonWritable tests

* Fix check and add specific tests
2019-05-10 11:02:01 -04:00
Raul Tambre
2b65a71d45 Fix use of 'is' operator for comparison (#2547)
The 'is' operator is not meant to be used for comparisons. It currently working is an implementation detail of CPython.
CPython 3.8 has added a SyntaxWarning for this.
2019-05-07 14:19:32 -04:00
David Neto
63f57d95d6
Support SPIR-V 1.4 (#2550)
* SPIR-V 1.4 headers, add SPV_ENV_UNIVERSAL_1_4

* Support --target-env spv1.4 in help for command line tools

* Support asm/dis of UniformId decoration

* Validate UniformId decoration

* Fix version check on instructions and operands

Also register decorations used with OpDecorateId

* Extension lists can differ between enums that match

Example: SubgroupMaskEq vs SubgroupMaskEqKHR

* Validate scope value for Uniform decoration, for SPIR-V 1.4

* More unioning of exts

* Preserve grammar order within an enum value

* 1.4: Validate OpSelect over composites

* Tools default to 1.4

* Add asm/dis test for OpCopyLogical

* 1.4: asm/dis tests for PtrEqual, PtrNotEqual, PtrDiff

* Basic asm/Dis test for OpCopyMemory

* Test asm/dis OpCopyMemory with 2-memory access

Add asm/dis tests for OpCopyMemorySized

Requires grammar update to add second optional memory access operand
to OpCopyMemory and OpCopyMemorySized

* Validate one or two memory accesses on OpCopyMemory*

* Check av/vis on CopyMemory source and target memory access

This is a proposed rule. See
https://gitlab.khronos.org/spirv/SPIR-V/issues/413

* Validate operation for OpSpecConstantOp

* Validate NonWritable decoration

Also permit NonWritable on members of UBO and SSBO.

* SPIR-V 1.4: NonWrtiable can decorate Function and Private vars

* Update optimizer CLI tests for SPIR-V 1.4

* Testing tools: Give expected SPIR-V version in message

* SPIR-V 1.4 validation for entry point interfaces

* Allow only unique interfaces
* Allow all global variables
* Check that all statically used global variables are listed
* new tests

* Add validation fixture CompileFailure

* Add 1.4 validation for pointer comparisons

* New tests

* Validate with image operands SignExtend, ZeroExtend

Since we don't actually know the image texel format, we can't fully
validate.  We need more context.

But we can make sure we allow the new image operands in known-good
cases.

* Validate OpCopyLogical

* Recursively checks subtypes
* new tests

* Add SPIR-V 1.4 tests for NoSignedWrap, NoUnsignedWrap

* Allow scalar conditions in 1.4 with OpSelect

* Allows scalar conditions with vector operands
* new tests

* Validate uniform id scope as an execution scope

* Validate the values of memory and execution scopes are valid scope
values
* new test

* Remove SPIR-V 1.4 Vulkan 1.0 environment

* SPIR-V 1.4 requires Vulkan 1.1

* FIX: include string for spvLog

* FIX: validate nonwritable

* FIX: test case suite for member decorate string

* FIX: test case for hlsl functionality1

* Validation test fixture: ease debugging

* Use binary version for SPIR-V 1.4 specific features

* Switch checks based on the SPIR-V version from the target environment
to instead use the version from the binary
* Moved header parsing into the ValidationState_t constructor (where
version based features are set)
* Added new versions of tests that assemble a 1.3 binary and validate a
1.4 environment

* Fix test for update to SPIR-V 1.4 headers

* Fix formatting

* Ext inst lookup: Add Vulkan 1.1 env with SPIR-V 1.4

* Update spirv-val help

* Operand version checks should use module version

Use the module version instead of the target environment version.

* Fix comment about two-access form of OpCopyMemory
2019-05-07 12:27:18 -04:00
David Neto
64f2750e5d
Update copyright checker for 2019 (#2256) 2018-12-28 17:27:53 -05:00
dan sinclair
aa53271b9c
Script to fixup fuzzer results (#1792) 2018-08-02 20:28:40 -04:00
dan sinclair
3dad1cda11
Change libspirv to spvtools namespace (#1678)
This CL changes all of the libspirv namespace code to spvtools to match
the rest of the code base.
2018-07-07 09:38:00 -04:00
Greg Roth
878b3b400b check_symbol_exports on Python3, Mac
subprocess.Popen returns byte data by default. Python2 was happy
to try to execute string operations on such data and hope for the
best, but python3 is more persnickety. Luckily, there's a simple
way to indicate to the Popen class that text data is wanted that
benefits the script. Just specifying universal_newlines will cause
the returned data to be text and also convert any system-specific
newlines to '\n' which the script relies on anyway.

Enabled on Mac as an incidental change after confirming that the
script works there just as well as it does on Linux.

It probably works on FreeBSD too, but I retired my BSD system years
ago. So I have no way to check.

Don't run it on Windows.
- It didn't work after all. It was just detecting non-posix and
  returning success.
2018-06-27 18:08:18 -04:00
David Neto
082b8b08f1 More automatic extension support
Update grammar table generation:
- Get extensions from instructions, not just operand-kinds
- Don't explicitly list extensions that come from the SPIR-V core
  grammar or from a KHR extended instruction set grammar.

This makes it easier to support new extensions since the recommended
extension strategy is to add instructions to the core grammar file.

Also, test the validator has trivial support for passing through
the extensions SPV_NV_shader_subgroup_partitioned and
SPV_EXT_descriptor_indexing.
2018-04-05 18:36:08 -04:00