Item — HN Reader

Subgroup operations[1]. In SIMT, they are fairly easily modeled[2] as communication between the different threads running on the same SIMD, and in most cases explicitly expose the predication mask. In fact, subgroupBallot(true) is a common idiom to extract that mask, and also to determine the subgroup size if run in subgroup uniform control flow. Generally they have good performance characteristics.

In SPMD, subgroups aren't easy to model, and would generally be emulated by inter-thread communication if it's important to match the semantics of a source program that includes them. Performance in that case would be terrible.

[1] In Vulkan, OpenGL, OpenCL, and WebGPU they are called "subgroups". In Nvidia including CUDA they are called "warps". In D3D and AMD they are called "waves". In Metal (often running on the same hardware) they are called "simdgroups". This fragmentation of terminology is part of the flavor of working with GPUs.

[2] "Fairly easily" by the standards of GPU infrastructure. In fact, the exact semantics have never been nailed down, and in particular "reconvergence" is poorly defined. Working through this is one of the things blocking subgroups in WebGPU (https://github.com/gpuweb/gpuweb/issues/4306). Even so, they're used commonly in practice, especially for things like fast matrix multiplication where you need to shuffle lots of data into place.

1 comments

This is true to some extent, although certainly SPMD has subgroups and reduction operations on those are fairly common.

And you are essentially repeating my point from before, that the fact that an operation among the SIMT threads can be scheduled as one SIMD op is the only special behavior. My example was regular vectorized SIMD ops, but your example of reductions are essentially the same point. Shuffles and horizontal adds are present in SIMT just like they are in regular SIMD.

These same reductions exist in contexts where the model would traditionally be called SPMD. But because the concepts map onto a different level of the system we might not easily recognize them as the same. The same kind of subgrouping happens in a multinode SPMD context in terms of NUMA regions and nodes, and an MPI implementation can handle reductions on a within-node or within-NUMA-region group of ranks more efficiently than a group containing ranks assigned to multiple meaningful hardware regions.

So I agree with you that the one thing SIMT has to differentiate it from SPMD is the SIMD-like scheduling of multiple threads -- and this was exactly my point from before -- but even here we find that SPMD has the same concepts, just not as explicitly, because the stack typically used in SPMD is more flexible, and SIMT can take advantage of specialized hardware ops.

Optimizations like this do not necessarily mean that the model isn't SPMD though. It's still a single program (a kernel) processing multiple data (indexed by the thread id). The one key idea of SIMT is really all about the batch scheduling of threads, the fact that multiple threads from a warp can be executed on one SM partition (using CUDA terms here) using a single instruction and a mask.

SIMT emphasizes the GPU scheduler, but it still fits completely within SPMD. All SIMT technologies are SPMD technologies, but not all SPMD technologies are SIMT technologies. I'm not convinced there is enough new in SIMT to warrant the branding using a new architecture type in the taxonomy. They could have just as well emphasized the thread scheduling as an optimization in an SPMD context.