* Fixed Bazel Windows build
Before this commit, the bazel build setup would not work on windows.
This is due to the fact, that genrule tries to use bash, which fails.
One fix would be to use bazel-skylib's run_binary.
This however does not work (easily) since genrules is more complex.
To (temporarily) fix the windows build, I added the `cmd_bat` property to every genrule.
This seems more like a hack, because it basically repeat commands,
but for now it at least builds on windows.
* Removed BAZEL_SH from bazel presubmit build script
Thanks to @s-perron for pointing out, that the presubmit script uses the msys64 bash shell for the bazel build.
Since adding the `cmd_bat` argument removes the dependency on bash, this is no longer needed.
Swift shader needs a way to inline all functions, even those marked as
DontInline. See https://github.com/KhronosGroup/SPIRV-Tools/pull/4471.
This implements the suggestion I made in the PR. We add a pass that
will remove the DontInline function control, so that the inlining passes
will inline them.
SwiftShader will still have to modify their code to add this pass before
the other passes are run.
The function `BuildInvalideAnalyses` will be rebuilt for every analysis that
has been requested, but it is not necessary. It also can cause problems
because if the CFG needs to be rebuilt, so do the dominator trees.
This change will make the functionality match the description of the
function.
* Use bazel 5 for Linux bazel builds.
* Download bazel 5.0.0 for macos and windows bazel builds.
* Modify the readme to mention bazel 5.0.0 as the version to use.
* Optimize DefUseManager allocations
Saves around 30-35% of compilation time.
For inst->use_ids, use a pool linked list instead of allocating vectors for every instruction. For inst->uses, use a "PooledLinkedList"' -- a linked list that has shared storage for all nodes. Neither re-use nodes, instead we do a bulk compaction operation when too much memory is being wasted (tuneable).
Includes separate PooledLinkedList templated datastructure, a very special case construct, but split out to make the code a little easier to understand.
Incrementally compute the hash instead of collecting words
Avoids allocating temporary space in a std::vector and std::u32string, and making three passes over all the hashed data.
Switch to using std::vector to prevent processing duplicates instead of std::unordered_set: avoids an allocation/deletion every call to ComputeHashValue, and ends up faster due to much better cache behaviour and smaller constant-factor when searching the (generally very small) list.
In my test case, made Type::HashValue go from 7.5% of compilation time to .5%
In newer versions of protobuf the Status building code has been made
internal, so that embedders cannot build their own instances like is
being done here.
Changing this code to just use the .ok() method on the status object,
since if the status is OK or not is what is actually being tested.
This will make it easier in the future to update external/protobuf.
The copyright check has hardcoded the years that can appear in the
copyright check. This means is has to be constantly updated as new
years or range of years are needed.
We try to avoid this, at least for a while. We allow years up to 2027.
The ranges for the years allow any combination of from 2014 to 2027.
* Update test for parsing memory access masks
Needed to support SPV_INTEL_memory_access_aliasing extension
There is a negative test that checks unused mask bits.
Some of those bits are now sued by the new Intel extension.
* Update deps for new SPIRV-Headers
Previously, array sizes were presumed to be OpConstant, which is not
necessarily true. This change ensures OpSpecConstant array sizes as
matched exactly, instead of taken as OpConstant and matched by value.
* Reimplement LCS used by spirv-diff
Two improvements are made to the LCS algorithm:
- The LCS algorithm is reimplemented to use a std::stack instead of
being recursive. This prevents stack overflow in the LCSTest.Large
test.
- The LCS algorithm uses an NxM table. Previously, entries of this
table were {size_t, bool, bool}, which is now packed in 32 bits. The
first entry can assume a maximum value of min(N, M), which
realistically for SPIR-V diff will not be larger than 1 billion
instructions. This reduces memory usage of LCS by 75%.
This partially reverts 845f3efb8a and
enables LCS tests.
* Stabilize the output of spirv-diff
std::map is used instead of std::unordered_map to ensure the output of
spirv-diff is identical everywhere.
This partially reverts 845f3efb8a and
enables spirv-diff tests.
Scalar replacement generates a null when there value for a member will
not be used. The null is used to make sure things are
deterministic in case there is an error.
However, some type cannot be null, so we will change that to use undef.
To keep the code simpler we will always use the undef.
Fixes#3996
spirv-diff is a new tool that produces diff-style output comparing two
SPIR-V modules. The instructions between the src and dst modules are
matched as best as the tool can, and output is produced (in src
id-space) that shows which instructions are removed in src, added in dst
or modified between them. The order of instructions are not retained.
Matching instructions between two SPIR-V modules is not trivial, and
thus a number of heuristics are applied in this tool. In particular,
without debug information, it's hard to match functions as they can be
reordered. As such, this tool is primarily useful to produce the diff
of two SPIR-V modules derived from the same source.
This tool can be useful in a number of scenarios:
- Compare the SPIR-V before and after modifying a shader
- Compare the SPIR-V produced from a shader before and after compiler
codegen changes.
- Compare the SPIR-V produced from a shader before and after some
transformation or optimization.
- Compare the SPIR-V produced from a shader with different compilers.
The handling of the RayQueryKHR type is not complete in the type
manager. The tests were not picking this up. I've added a test to make
sure that the `GenerateAllTypes` function actually does generate all of
the types. Once it is added there other tests should pick up on the
other parts that were missing.
* Roll external/googletest/ bf0701daa..f45d5865e (47 commits)
bf0701daa9...f45d5865ed
$ git log bf0701daa..f45d5865e --date=short --no-merges --format='%ad %ae %s'
2022-01-24 absl-team Make recreate_environments_when_repeating=false the default.
2022-01-19 absl-team Factor out AssertionResult into dedicated gtest-assertion-result header + implementation files to prevent cyclic includes between gtest.h and gtest_pred_impl.h
2022-01-18 absl-team Consistently apply IWYU pragmas across googletest and googlemock headers
2022-01-14 dinor Fix reference to source file in gmock FAQ
2022-01-14 dmauro Update test Docker image to GCC 11.2, Clang 14 (prerelease), CMake 3.22.1, and Bazel 4.2.2
2022-01-13 ayush854032 FIX#2174 -- remove `DEBUG_POSTFIX`
2022-01-12 absl-team Fix run-on sentence
2022-01-06 hgsilverman Fix a typo in comments.
2021-12-22 hgsilverman Use normal for loop instead of accumulate.
2021-12-17 hgsilverman Improve code readablity.
2021-12-17 hgsilverman Apply requested changes to preserve old behavior.
2021-12-03 hgsilverman Reimplement MatchesFilter with new interfaces.
2021-11-23 hgsilverman Apply requested changes.
2021-10-17 hgsilverman Get rid of redundant filter matching code
2021-10-17 hgsilverman Add comments describing the behavior of filters
2021-10-17 hgsilverman Process filter string once instead of per test
2022-01-05 bsilver16384 Remove another GTEST_DISALLOW_ASSIGN_ that crept in
2022-01-04 4789010+ramkumar-kr Update example to reflect the tests below
2022-01-01 ayush854032 FIX#3719 -- Fix `clang` conversion warnings
2021-12-27 absl-team Include the param names in the generated description of the MATCHER_P matchers.
2021-12-22 absl-team Clarify "package" means "Bazel package", and promote `testonly=True` rather than `testing` sub-directory.
2021-12-22 dmauro Makes the Python imports consistently use full paths from the repository root, unifying the behavior between Bazel and CMake
2021-12-21 dmauro Makes TestForDeathTest.CRTDebugDeath only run when _DEBUG is defined
2021-12-21 philip.j.m link to regex on qnx in cmake
2021-12-15 dinor Release tests for UnitTestOptions::MatchesFilter
2021-12-07 absl-team Add NOLINT to address modernize-use-trailing-return-type in TEST_F uses
2021-12-04 ayush854032 Allow to specify a random seed independently of whether shuffling is requested
2021-12-03 absl-team Support std::reference_wapper as a smart pointer type in gmock
2021-12-02 absl-team Fix typo in GoogleMock Cookbook.
2021-11-29 absl-team Work around Android KitKat tzset bug
2021-11-25 ludger.paehler Fix typo in the queue test.
2021-11-16 denis.samoylov Suppress unused-value warning
2021-11-16 93249401+assafpr fixed branch from master to main in readme.md
2021-11-16 93249401+assafpr Update README.md
2021-11-16 krystian.kuzniarek fix a typo
2021-11-15 cramertj Clarify "Times() after X" error message
2021-11-15 dmauro Update URLs in Bazel Quickstart
2021-11-12 absl-team Fix typo in gmock-matchers_test.cc: CanDescribeNegationOfExpectingOneElment => CanDescribeNegationOfExpectingOneElement
2021-11-10 absl-team Googletest export
2021-11-09 dmauro Googletest export
2021-11-10 memmenlauer googletest/src/gtest-port.cc: Added GetLastError() on Windows for CreateThread()
2021-11-09 absl-team Googletest export
2021-11-08 absl-team Googletest export
2021-11-08 absl-team Googletest export
2021-11-04 absl-team Googletest export
2021-11-05 raul.tambre Replace use of deprecated std::iterator
2021-10-20 h.zeller Start a simple github actions CI
Created with:
roll-dep external/googletest
* Roll external/re2/ 4244cd1cb..611baecbc (42 commits)
4244cd1cb4...611baecbce
$ git log 4244cd1cb..611baecbc --date=short --no-merges --format='%ad %ae %s'
2022-01-24 junyer Update to the latest version of FuzzedDataProvider.
2022-01-19 junyer Replace the build job with the build-appleclang job.
2022-01-18 junyer Address a `-Wunused-but-set-variable' warning from Clang 13.x.
2022-01-18 junyer Fix the build job on macos-latest.
2022-01-17 junyer Remove GCC 4.x and 5.x from the CI configuration.
2022-01-16 stefano Don't specify the `-std` flag in Makefile or re2.pc.
2022-01-12 junyer Use Python 3 shebangs for make_unicode_*.py.
2021-12-24 andrei.bencsik Fix CMake packaging for installation.
2021-11-18 junyer Remove a redundant map access.
2021-11-02 junyer Produce static-testinstall and shared-testinstall binaries.
2021-10-27 junyer Precompute bit_state_text_max_size during compilation.
2021-10-27 junyer Fix another latent bug in Regexp::Walker<T>::Reset().
2021-10-25 junyer Update Unicode data to 14.0.0.
2021-10-25 junyer Make the fuzzer check for large subexpressions.
2021-10-23 junyer Address a `-Wshadow' warning.
2021-10-19 junyer Don't assume that iterators can be compared.
2021-10-18 junyer Switch back to native C++ rules.
2021-09-02 junyer Crudely limit the use of 'k', 'K', 's' and 'S' when fuzzing.
2021-09-02 junyer Inspect substrings with a Walker<> instead of Prefilter.
2021-08-31 junyer thread_local support in MinGW seems to be buggy.
2021-08-26 junyer Use SRWLOCK_INIT instead of InitializeSRWLock().
2021-08-23 junyer Add Clang 13 to the build matrix.
2021-08-13 junyer Tweak the OnePass versus DFA cutoff.
2021-08-09 junyer Improve the D wrapper details.
2021-08-09 junyer Mention the D wrapper.
2021-08-03 junyer Fix breakage when RE2_USE_ICU is defined.
2021-08-03 junyer Permit Unicode characters beyond ASCII in capture names.
2021-07-24 junyer Further reduce the maximum repeat count when fuzzing.
2021-07-08 junyer Add a SECURITY.md file.
2021-07-07 junyer Note that case-insensitive prefixes are already lowercase.
2021-07-06 junyer Tidy up BuildShiftDFA() a bit more.
2021-07-06 junyer Flip the hot loops to do-while.
2021-07-06 junyer Optimise the hot loop some more for Clang.
2021-07-06 junyer Work around a code generation issue seen with Clang.
2021-07-05 junyer Fix a couple of style nits.
2021-07-04 junyer Fix a bug that affects 32-bit platforms.
2021-07-03 junyer Implement case-insensitive prefix acceleration.
2021-07-02 junyer Add Search_Easy2_Cached* benchmarks.
2021-06-30 junyer Migration from master to main.
2021-06-23 junyer Revert "Yikes. int64_t{1}<<31 doesn't actually make sense."
2021-06-23 junyer Fix GetCachedProg() for OnePass.
2021-06-23 junyer Yikes. int64_t{1}<<31 doesn't actually make sense.
Created with:
roll-dep external/re2
* Roll external/spirv-headers/ eddd4dfc9..b42ba6d92 (2 commits)
eddd4dfc93...b42ba6d92f
$ git log eddd4dfc9..b42ba6d92 --date=short --no-merges --format='%ad %ae %s'
2022-01-17 michael.kinsner Allocate additional loop control bit for upcoming Intel extension
2021-11-16 dragonjoker59 Add ShaderWriter as SPIR-V generation tool.
Created with:
roll-dep external/spirv-headers
Two deps changes branch names, and roll_deps.sh must be updated.
We also don't want the script to fail just because one of the deps is
already up-to-date. Set `+e` before rolling the deps.
Add a pass to spread Volatile semantics to variables with SMIDNV,
WarpIDNV, SubgroupSize, SubgroupLocalInvocationId, SubgroupEqMask,
SubgroupGeMask, SubgroupGtMask, SubgroupLeMask, or SubgroupLtMask BuiltIn
decorations or OpLoad for them when the shader model is the ray
generation, closest hit, miss, intersection, or callable shaders. This
pass can be used for VUID-StandaloneSpirv-VulkanMemoryModel-04678 and
VUID-StandaloneSpirv-VulkanMemoryModel-04679 (See "Standalone SPIR-V
Validation" section of Vulkan spec "Appendix A: Vulkan Environment for
SPIR-V").
Handle variables used by multiple entry points:
1. Update error check to make it working regardless of the order of
entry points.
2. For a variable, if it is used by two entry points E1 and E2 and
it needs the Volatile semantics for E1 while it does not for E2
- If VulkanMemoryModel capability is enabled, which means we have to
set memory operation of load instructions for the variable, we
update load instructions in E1, but do not update the ones in E2.
- If VulkanMemoryModel capability is disabled, which means we have
to add Volatile decoration for the variable, we report an error
because E1 needs to add Volatile decoration for the variable while
E2 does not.
For the simplicity of the implementation, we assume that all functions
other than entry point functions are inlined.
C++20 automatically adds reversed versions of operator overloads for
consideration; in this particular instance this results in infinite
recursion, which has now been pointed out elsewhere as a known issue
when migrating to C++20. Here we just disable one of the overloads in
C++20 mode and let the auto-reversing take care of it for us.
* linker: Address conversion error introduced in earlier rework
Also rework the code slightly to first compute the new ID bound and
validate it, and only then, offset all existing IDs.
This makes those two steps clearer, and avoids potentially overflowing
the IDs within a module (even if that would have been caught later on).
Fixes: c3849565 ("Linker improvements (#4679)")
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4684
* test/linker: Disable IdsLimit tests for now
They are taking too long to run and results in the CI jobs timing out.
* test/linker: Code factorisation and small tweaks
* linker: Do not fail when going over limits
The limits are minima and implementations or APIs might support higher
limits, so just warn the user about it. And only check for the limits
right before emitting the binary, as limits might change earlier when
removing duplicate instructions, function prototypes, etc.
The only check performed right before merging, is making sure the ID
bound will not overflow the 32 bits following the merge.
Also, use the defines for the limits instead of hard-coding them.
* linker: Require a memory model in each input module
The existing code could run into weird situation. For example, if the
first module had no memory model, it would not emit any memory model
(sort of reasonable) and would accept without complains all possible mix
from later modules as it would not verify them.
* linker: Replace hex version with SPV_SPIRV_VERSION_WORD
* linker: Error out when linking together different versions
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4135
* tools/linker: Do not write to disk if linking failed
Also, do not consider warnings as errors.
* tools/linker: Fix formatting in help message
* tools/linker: Further clarify the use of --target-env
Also update the text for the default version to reflect the change made
in 7d768812 ("Basic support for SPIR-V 1.6 (#4663)").
* PassManager: Print errors occurring during disassembly
Otherwise one could be greeted by the following text when running
spirv-opt withe the `--print-all` flag:
; IR before pass wrap-opkill
; IR before pass eliminate-dead-branches
; IR before pass merge-return
With this commit, one will instead get:
error: line 143: Invalid opcode: 400
warning: line 0: Disassembly failed before pass wrap-opkill
error: line 143: Invalid opcode: 400
warning: line 0: Disassembly failed before pass eliminate-dead-branches
error: line 143: Invalid opcode: 400
warning: line 0: Disassembly failed before pass merge-return
* PassManager: Use the right target environment when disassembling
Disassembly would fail if features from a newer version of SPIR-V than
1.2 were used.
The comment in `Array::GetExtraHashWords` is misleading because getting
the hash words is split up between the generic `Type::GetHashWords` and
the type specific `Type::GetExtraHashWords`. While `IsSameImpl` is
self-contained. Removing the comment since it is misleading and no
comment is really needed.
Fixes#3248
The test harness for the opt fuzzer was failing to consider that the
input might use a very large id bound, despite no id approaching this
bound actually being used.
This change modifies the test harness to use the module's id bound,
rather than looking through the module for large ids.
Fixes: oss-fuzz:42386
Ensures that instruction's opcode is set to something default when
parsing the module with --preserve-numeric-ids enabled. This avoids
uninitialized accesses and knock-on buffer overflows.
Fixes#4672.
For a shader input/output interface variable of structure type:
If the structure has BuiltIn members, then the structure type
must be Block decorated.
Otherwise, the variable, or the struct members must have locations,
but nothing can have both a location be a BuiltIn.
Implements validation needed to reject the example in
https://github.com/KhronosGroup/SPIRV-Registry/issues/134
* Basic support for SPIR-V 1.6
* Update SPIRV-Headers deps
* Add new environment enum for SPIR-V 1.6
* Make default environment 1.6 for most tools
* Update tests
* Disallow conditional branch with duplicate labels
* Disallow Dim=Buffer with sampled images
* Do not require the non-semantic extension after SPIR-V 1.5
The pass to remove the nonsemantic information and instructions
is used for drivers or tools that may not support them. Debug
information was only partially handle, which is causing a
problem. We need to either fully remove debug information or
not remove it all. Since I can see it being useful to keep the
debug information even when the nonsemantic instructions are
removed, I propose we do not remove debug info.
Fixes https://github.com/KhronosGroup/SPIRV-Tools/issues/4269
