Item — HN Reader

DirectX12 has ok compute capabilities, but is increasingly being left in the dust by Vulkan. It doesn't have pointers or a memory model, but it does have device scoped barriers (which Metal lacks), so you can do things like single-pass prefix sum (monoid scan) operations, which Metal and WebGPU can't. Also, the Nvidia "cooperative matrix" operations (which access the "tensor cores") exist in CUDA and Vulkan but not DirectX12.

Programming GPU compute outside CUDA is still quite painful; the developer experience of tools is terrible and the ecosystem has a lot of catching up to do. I'm most bullish on WebGPU going forward, but there are definite growing pains.

2 comments

Vulkan Compute hardly matters and DirectX future is on mesh shaders.

Also all major graphics vendors tend to create their hardware designs in collaboration with Microsoft as part of DirectX, and eventually add them as extensions to Khronos APIs.

So it hardly matters what Khronos is doing with Vulkan.

all major graphics vendors tend to create their hardware designs in collaboration with Microsoft as part of DirectX

That's pretty sickening, I hope the EU will give them a rap across the knuckles for that. Very anti-competitive.

It is up to Khronos to make it more interesting to do otherwise.

Hardware Transformation and Lighting, CG/HLSL, RayTracing, MeshShaders and more recently DirectStorage, are all examples of features that appeared first in DirectX, with AMD or NVidia hardware implementation, before showing up as OpenGL or Vulkan extension.

Could you elaborate on how device scoped barriers are benefficial? I’m not sure I follow where that would help for your example, but I’m definitely curious (and Google searches have become increasingly useless of late)

Sure! I have blogged about this[1], but I'll summarize here. Basically a barrier helps you do a "message passing" pattern, where one workgroup prepares some data then sets a flag, and another workgroup can read the flag and then read the data from the first workgroup. However, in Metal (and hence WebGPU) the barriers aren't powerful enough to guarantee you won't see stale data. Thus, to do prefix sum on Metal, you need to at least two dispatches, one to aggregate reductions over partitions, then another to do the sum within a partition. That's less efficient, both because of the dispatch overhead, and also because you need to read the data at least twice. You also need a bunch of different versions of your shaders to handle different problem sizes, which is really annoying (in fact, Vello can't handle more than 64k path segments until I write the larger version). Vulkan, CUDA, and DirectX12 (even DirectX11) can all do this; it's one of the ways in which Metal is an inferior basis for doing GPU compute.

Btw, this is one of the reasons I feel a bit burned by MoltenVK, as it happily and silently translates correct SPIR-V into MSL that's lacking the correct barriers. In my experience, GPU translation layers are some of the leakiest abstractions around.

[1]: https://raphlinus.github.io/gpu/2021/11/17/prefix-sum-portab...

Thanks for the great write up!