AuroraMiddleware/SPIRV-Cross
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
dee35bf3ce
SPIRV-Cross/reference/shaders-msl-no-opt/comp/subgroups.nocompat.invalid.vk.msl22.ios.comp

283 lines
9.2 KiB
Plaintext
Raw Normal View History

MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
#pragma clang diagnostic ignored "-Wmissing-prototypes"
#include <metal_stdlib>
#include <simd/simd.h>
using namespace metal;
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
struct SSBO
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
float FragColor;
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
};
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
constant uint3 gl_WorkGroupSize [[maybe_unused]] = uint3(1u);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
template<typename T>
inline T spvSubgroupBroadcast(T value, ushort lane)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_broadcast(value, lane);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<>
inline bool spvSubgroupBroadcast(bool value, ushort lane)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return !!quad_broadcast((ushort)value, lane);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<uint N>
inline vec<bool, N> spvSubgroupBroadcast(vec<bool, N> value, ushort lane)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return (vec<bool, N>)quad_broadcast((vec<ushort, N>)value, lane);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<typename T>
inline T spvSubgroupBroadcastFirst(T value)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_broadcast_first(value);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<>
inline bool spvSubgroupBroadcastFirst(bool value)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return !!quad_broadcast_first((ushort)value);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<uint N>
inline vec<bool, N> spvSubgroupBroadcastFirst(vec<bool, N> value)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return (vec<bool, N>)quad_broadcast_first((vec<ushort, N>)value);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
inline uint4 spvSubgroupBallot(bool value)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return uint4((quad_vote::vote_t)quad_ballot(value), 0, 0, 0);
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
}
inline bool spvSubgroupBallotBitExtract(uint4 ballot, uint bit)
{
return !!extract_bits(ballot[bit / 32], bit % 32, 1);
}
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
inline uint spvSubgroupBallotFindLSB(uint4 ballot, uint gl_SubgroupSize)
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
uint4 mask = uint4(extract_bits(0xFFFFFFFF, 0, gl_SubgroupSize), uint3(0));
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
ballot &= mask;
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
return select(ctz(ballot.x), select(32 + ctz(ballot.y), select(64 + ctz(ballot.z), select(96 + ctz(ballot.w), uint(-1), ballot.w == 0), ballot.z == 0), ballot.y == 0), ballot.x == 0);
}
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
inline uint spvSubgroupBallotFindMSB(uint4 ballot, uint gl_SubgroupSize)
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
uint4 mask = uint4(extract_bits(0xFFFFFFFF, 0, gl_SubgroupSize), uint3(0));
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
ballot &= mask;
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
return select(128 - (clz(ballot.w) + 1), select(96 - (clz(ballot.z) + 1), select(64 - (clz(ballot.y) + 1), select(32 - (clz(ballot.x) + 1), uint(-1), ballot.x == 0), ballot.y == 0), ballot.z == 0), ballot.w == 0);
}
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
inline uint spvPopCount4(uint4 ballot)
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
{
return popcount(ballot.x) + popcount(ballot.y) + popcount(ballot.z) + popcount(ballot.w);
}
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
inline uint spvSubgroupBallotBitCount(uint4 ballot, uint gl_SubgroupSize)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
uint4 mask = uint4(extract_bits(0xFFFFFFFF, 0, gl_SubgroupSize), uint3(0));
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
return spvPopCount4(ballot & mask);
}
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
inline uint spvSubgroupBallotInclusiveBitCount(uint4 ballot, uint gl_SubgroupInvocationID)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
uint4 mask = uint4(extract_bits(0xFFFFFFFF, 0, gl_SubgroupInvocationID + 1), uint3(0));
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
return spvPopCount4(ballot & mask);
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
}
inline uint spvSubgroupBallotExclusiveBitCount(uint4 ballot, uint gl_SubgroupInvocationID)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
uint4 mask = uint4(extract_bits(0xFFFFFFFF, 0, gl_SubgroupInvocationID), uint2(0));
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
return spvPopCount4(ballot & mask);
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
}
template<typename T>
inline bool spvSubgroupAllEqual(T value)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_all(all(value == quad_broadcast_first(value)));
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
}
template<>
inline bool spvSubgroupAllEqual(bool value)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_all(value) || !quad_any(value);
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
}
MSL: Support vectors with OpGroupNonUniformAllEqual. This was not tested here in SPIRV-Cross. Predictably, it broke when I tried it in the CTS.
2020-10-21 04:04:43 +00:00
template<uint N>
inline bool spvSubgroupAllEqual(vec<bool, N> value)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_all(all(value == (vec<bool, N>)quad_broadcast_first((vec<ushort, N>)value)));
MSL: Support vectors with OpGroupNonUniformAllEqual. This was not tested here in SPIRV-Cross. Predictably, it broke when I tried it in the CTS.
2020-10-21 04:04:43 +00:00
}
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
template<typename T>
inline T spvSubgroupShuffle(T value, ushort lane)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_shuffle(value, lane);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<>
inline bool spvSubgroupShuffle(bool value, ushort lane)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return !!quad_shuffle((ushort)value, lane);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<uint N>
inline vec<bool, N> spvSubgroupShuffle(vec<bool, N> value, ushort lane)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return (vec<bool, N>)quad_shuffle((vec<ushort, N>)value, lane);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<typename T>
inline T spvSubgroupShuffleXor(T value, ushort mask)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_shuffle_xor(value, mask);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<>
inline bool spvSubgroupShuffleXor(bool value, ushort mask)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return !!quad_shuffle_xor((ushort)value, mask);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<uint N>
inline vec<bool, N> spvSubgroupShuffleXor(vec<bool, N> value, ushort mask)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return (vec<bool, N>)quad_shuffle_xor((vec<ushort, N>)value, mask);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<typename T>
inline T spvSubgroupShuffleUp(T value, ushort delta)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_shuffle_up(value, delta);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<>
inline bool spvSubgroupShuffleUp(bool value, ushort delta)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return !!quad_shuffle_up((ushort)value, delta);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<uint N>
inline vec<bool, N> spvSubgroupShuffleUp(vec<bool, N> value, ushort delta)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return (vec<bool, N>)quad_shuffle_up((vec<ushort, N>)value, delta);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<typename T>
inline T spvSubgroupShuffleDown(T value, ushort delta)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return quad_shuffle_down(value, delta);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<>
inline bool spvSubgroupShuffleDown(bool value, ushort delta)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return !!quad_shuffle_down((ushort)value, delta);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<uint N>
inline vec<bool, N> spvSubgroupShuffleDown(vec<bool, N> value, ushort delta)
{
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
return (vec<bool, N>)quad_shuffle_down((vec<ushort, N>)value, delta);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
}
template<typename T>
inline T spvQuadBroadcast(T value, uint lane)
{
return quad_broadcast(value, lane);
}
template<>
inline bool spvQuadBroadcast(bool value, uint lane)
{
return !!quad_broadcast((ushort)value, lane);
}
template<uint N>
inline vec<bool, N> spvQuadBroadcast(vec<bool, N> value, uint lane)
{
return (vec<bool, N>)quad_broadcast((vec<ushort, N>)value, lane);
}
template<typename T>
inline T spvQuadSwap(T value, uint dir)
{
return quad_shuffle_xor(value, dir + 1);
}
template<>
inline bool spvQuadSwap(bool value, uint dir)
{
return !!quad_shuffle_xor((ushort)value, dir + 1);
}
template<uint N>
inline vec<bool, N> spvQuadSwap(vec<bool, N> value, uint dir)
{
return (vec<bool, N>)quad_shuffle_xor((vec<ushort, N>)value, dir + 1);
}
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
kernel void main0(device SSBO& _9 [[buffer(0)]], uint gl_NumSubgroups [[quadgroups_per_threadgroup]], uint gl_SubgroupID [[quadgroup_index_in_threadgroup]], uint gl_SubgroupSize [[thread_execution_width]], uint gl_SubgroupInvocationID [[thread_index_in_quadgroup]])
{
uint4 gl_SubgroupEqMask = uint4(1 << gl_SubgroupInvocationID, uint3(0));
uint4 gl_SubgroupGeMask = uint4(insert_bits(0u, 0xFFFFFFFF, gl_SubgroupInvocationID, gl_SubgroupSize - gl_SubgroupInvocationID), uint3(0));
uint4 gl_SubgroupGtMask = uint4(insert_bits(0u, 0xFFFFFFFF, gl_SubgroupInvocationID + 1, gl_SubgroupSize - gl_SubgroupInvocationID - 1), uint3(0));
uint4 gl_SubgroupLeMask = uint4(extract_bits(0xFFFFFFFF, 0, gl_SubgroupInvocationID + 1), uint3(0));
uint4 gl_SubgroupLtMask = uint4(extract_bits(0xFFFFFFFF, 0, gl_SubgroupInvocationID), uint3(0));
_9.FragColor = float(gl_NumSubgroups);
_9.FragColor = float(gl_SubgroupID);
_9.FragColor = float(gl_SubgroupSize);
_9.FragColor = float(gl_SubgroupInvocationID);
simdgroup_barrier(mem_flags::mem_device | mem_flags::mem_threadgroup | mem_flags::mem_texture);
simdgroup_barrier(mem_flags::mem_device | mem_flags::mem_threadgroup | mem_flags::mem_texture);
simdgroup_barrier(mem_flags::mem_device);
simdgroup_barrier(mem_flags::mem_threadgroup);
simdgroup_barrier(mem_flags::mem_texture);
bool _39 = quad_is_first();
bool elected = _39;
_9.FragColor = float4(gl_SubgroupEqMask).x;
_9.FragColor = float4(gl_SubgroupGeMask).x;
_9.FragColor = float4(gl_SubgroupGtMask).x;
_9.FragColor = float4(gl_SubgroupLeMask).x;
_9.FragColor = float4(gl_SubgroupLtMask).x;
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
float4 broadcasted = spvSubgroupBroadcast(float4(10.0), 8u);
bool2 broadcasted_bool = spvSubgroupBroadcast(bool2(true), 8u);
float3 first = spvSubgroupBroadcastFirst(float3(20.0));
bool4 first_bool = spvSubgroupBroadcastFirst(bool4(false));
Only run spirv-opt if the spirv is valid. This CL updates the test runner to only run spirv-opt if the generated SPIR-V is valid. If validation is skipped it's possible to hit aborts and other memory errors in the optimizer as it assumes the SPIR-V is valid.
2019-11-05 15:59:41 +00:00
uint4 ballot_value = spvSubgroupBallot(true);
bool inverse_ballot_value = spvSubgroupBallotBitExtract(ballot_value, gl_SubgroupInvocationID);
bool bit_extracted = spvSubgroupBallotBitExtract(uint4(10u), 8u);
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
uint bit_count = spvSubgroupBallotBitCount(ballot_value, gl_SubgroupSize);
Only run spirv-opt if the spirv is valid. This CL updates the test runner to only run spirv-opt if the generated SPIR-V is valid. If validation is skipped it's possible to hit aborts and other memory errors in the optimizer as it assumes the SPIR-V is valid.
2019-11-05 15:59:41 +00:00
uint inclusive_bit_count = spvSubgroupBallotInclusiveBitCount(ballot_value, gl_SubgroupInvocationID);
uint exclusive_bit_count = spvSubgroupBallotExclusiveBitCount(ballot_value, gl_SubgroupInvocationID);
MSL: Mask ballots passed to Ballot bit ops. Only the least *n* bits are significant, where *n* is the subgroup size. The Vulkan CTS actually checks this. The `FindLSB` tests weren't actually failing, but I masked that anyway, in case there's some corner case the CTS is missing.
2020-10-21 04:59:30 +00:00
uint lsb = spvSubgroupBallotFindLSB(ballot_value, gl_SubgroupSize);
uint msb = spvSubgroupBallotFindMSB(ballot_value, gl_SubgroupSize);
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
uint shuffled = spvSubgroupShuffle(10u, 8u);
bool shuffled_bool = spvSubgroupShuffle(true, 9u);
uint shuffled_xor = spvSubgroupShuffleXor(30u, 8u);
bool shuffled_xor_bool = spvSubgroupShuffleXor(false, 9u);
uint shuffled_up = spvSubgroupShuffleUp(20u, 4u);
bool shuffled_up_bool = spvSubgroupShuffleUp(true, 4u);
uint shuffled_down = spvSubgroupShuffleDown(20u, 4u);
bool shuffled_down_bool = spvSubgroupShuffleDown(false, 4u);
MSL: Expand subgroup support. Add support for declaring a fixed subgroup size. Metal, like Vulkan with `VK_EXT_subgroup_size_control`, allows the thread execution width to vary depending on factors such as register usage. Unfortunately, this breaks several tests that depend on the subgroup size being what the device says it is. So we'll fix the subgroup size at the size the device declares. The extra invocations in the subgroup will appear to be inactive. Because of this, the ballot mask builtins are now ANDed with the active subgroup mask. Add support for emulating a subgroup of size 1. This is intended to be used by Vulkan Portability implementations (e.g. MoltenVK) when the hardware/software combo provides insufficient support for subgroups. Luckily for us, Vulkan 1.1 only requires that the subgroup size be at least 1. Add support for quadgroup and SIMD-group functions which were added to iOS in Metal 2.2 and 2.3. This will allow clients to take advantage of expanded quadgroup and SIMD-group support in recent Metal versions and on recent Apple GPUs (families 6 and 7). Gut emulation of subgroup builtins in fragment shaders. It turns out codegen for the SIMD-group functions in fragment wasn't implemented for AMD on Mojave; it's a safe bet that it wasn't implemented for the other drivers either. Subgroup support in fragment shaders now requires Metal 2.2.
2020-11-19 05:16:46 +00:00
bool has_all = quad_all(true);
bool has_any = quad_any(true);
Only run spirv-opt if the spirv is valid. This CL updates the test runner to only run spirv-opt if the generated SPIR-V is valid. If validation is skipped it's possible to hit aborts and other memory errors in the optimizer as it assumes the SPIR-V is valid.
2019-11-05 15:59:41 +00:00
bool has_equal = spvSubgroupAllEqual(0);
has_equal = spvSubgroupAllEqual(true);
MSL: Support vectors with OpGroupNonUniformAllEqual. This was not tested here in SPIRV-Cross. Predictably, it broke when I tried it in the CTS.
2020-10-21 04:04:43 +00:00
has_equal = spvSubgroupAllEqual(float3(0.0, 1.0, 2.0));
has_equal = spvSubgroupAllEqual(bool4(true, true, false, true));
MSL: Cast broadcast booleans to ushort. Metal doesn't support broadcasting or shuffling boolean values, but we can work around that by casting it to `ushort`, then casting it back to `bool`. I used `ushort` instead of `uint` because 16-bit values give better throughput on Apple GPUs.
2020-10-21 06:51:48 +00:00
float4 swap_horiz = spvQuadSwap(float4(20.0), 0u);
bool4 swap_horiz_bool = spvQuadSwap(bool4(true), 0u);
float4 swap_vertical = spvQuadSwap(float4(20.0), 1u);
bool4 swap_vertical_bool = spvQuadSwap(bool4(true), 1u);
float4 swap_diagonal = spvQuadSwap(float4(20.0), 2u);
bool4 swap_diagonal_bool = spvQuadSwap(bool4(true), 2u);
float4 quad_broadcast0 = spvQuadBroadcast(float4(20.0), 3u);
bool4 quad_broadcast_bool = spvQuadBroadcast(bool4(true), 3u);
MSL: Support SubgroupLocalInvocationId and SubgroupSize in all stages. MSL prior to 2.2 doesn't support these natively in any stage but compute. But, we can (assuming no threads were terminated prematurely) get their values with some creative uses of the `simd_prefix_exclusive_sum()` and `simd_sum()` functions. Also, fix a missing `to_expression()` with `BuiltInSubgroupEqMask`. For KhronosGroup/MoltenVK#629.
2019-05-31 18:42:36 +00:00
}
Reference in New Issue Copy Permalink