Item — HN Reader

Adequate for what purpose? My personal feeling is that if you are sufficiently motivated, you can get your work done in them, but they are painful and tedious enough that in most cases your GPU is sitting there idle while the CPU with approximately 1/10 the raw computational throughput does its best to chug through compute workloads that could be done on GPU instead. And virtually nobody[1] is using them for machine learning; CUDA is so overwhelmingly better that people will buy into vendor lock-in to use it.

[1] kompute.cc is the exception that proves the rule

2 comments

Aside from being easier to use, what other advantages does Cuda have? Does Nvidia hinder Vulkan compute in any way?

It's not just easier to use, but more powerful in many ways. A huge advantage of CUDA is the ecosystem of high quality libraries, including cuBLAS, cuDNN (machine learning), CUB, and Thrust. In the compute shader landscape, there's nothing remotely comparable, largely because GLSL doesn't support abstractions to express concepts like "do a prefix sum (scan) with this custom monoid", so you end up having to custom code everything yourself in C.

In more recent iterations (ie since Volta/GTX 20XX), CUDA also supports independent thread scheduling (allowing mutexes with thread granularity) and cooperative groups. For really advanced workloads, that's well beyond what compute shaders can do, and I suspect it will be a decade or so to catch up, if they ever do. Of course, part of what I enjoy about compute shaders is the challenge of making algorithms run well in a more constrained computing model.

Nvidia does not hinder Vulkan compute in any way. I have a bunch of experience with Vulkan implementations and Nvidia's is certainly one of the best. In particular, their implementation of the Vulkan memory model is top-notch, with no correctness issues I've uncovered, and with a measurable performance improvement from using the fancy atomic semantics over older-style barriers. In fact, this is arguably one way in which compute shaders are more advanced than CUDA, which still does not have a formal memory model.

> CUDA also supports independent thread scheduling

And that’s technically only present on NVIDIA and Arm Mali GPUs as of today. (Both do independent thread scheduling, not so for the other GPU vendors)

I think the problem is not with compute itself but supposedly with higher level languages like GLSL and usability / expressiveness of their abstractions. You don't write your logic in pure SPIR-V.

The SPIR-V OpenCL profile is suitable, the Vulkan one is much less suitable. (you can’t even transpile OpenCL C 1.2 to it fully, with quite heavily limited pointers…)

Metal has the best GPGPU story of the graphics APIs, with D3D12 coming second after that. Both at least can have OpenCL C transpiled without catches to them…

Microsoft is writing CLon12 on that front: https://github.com/microsoft/OpenCLOn12

> (you can’t even transpile OpenCL C 1.2 to it fully, with quite heavily limited pointers…)

Is that still true as of Vulkan 1.3? We have buffer device address now, which I think counts as a real pointer.

> Metal has the best GPGPU story of the graphics APIs

I wish I could agree wholeheartedly with this. Yes, it's wonderful in many ways, but the lack of device-scope barriers means an entire class of advanced coordination patterns is unavailable.

Of course, you're talking about transpiling OpenCL C 1.2, which I consider incredibly primitive and limited compared to modern compute shaders: no subgroups, no device-scope barriers, no 16 bit float.

> Is that still true as of Vulkan 1.3?

In practice every GPU vendor that mattered already implemented it to 1.2, not much of a change on that front.

The core problem, that pointers are an abstract type with no known compile-time size remains.

> but the lack of device-scope barriers means an entire class of advanced coordination patterns is unavailable.

(And OpenCL on Apple Silicon seems to have the opposite problem, stuff being promoted to device wide barriers when it should not be…)

That still sounds like not a good situation, if the only option if C-like expressiveness level I suppose.

Can something like Rust level of expressiveness be compiled into OpenCL profile SPIR-V?

Basically I'm talking about the higher level language, not about just IR being able to express compute use cases. Metal, D3D and CUDA are all dead ends becasue neither of them is really free of some kind of lock-in.

Cuda is c++ with all the expressive power it has

That is a good example of why they aren't adequate (if that's indeed the case).