▲ 205 points
back
173 comments
For context, the submitter of the issue is Anush Elangovan from AMD who's recently been a lot more active on social after the SemiAnalysis article, and taking the reigns / responsibility of moving AMD's software efforts forward.
However you want to dissect this specific issue, I'd generally consider this a positive step and nice to see it hit the front page.
https://www.reddit.com/r/ROCm/comments/1i5aatx/rocm_feedback...
Also know as the AMD representative who recently argued with Hotz about supporting tinycorp.
Is that a bad thing? Good for him to stand up to extortion.
hey thats me. Happy to help answer anything here and look forward to your constructive feedback to make AMD software better. We got work to do and look forward to it.
Ok, why does running koboldcpp with a "BLAS Batch Size" of 512 via Vulkan on an RX570 crash my entire computer? You know, to the point where I manually have to turn it on again.
I personally couldn't think of a better reason to never buy AMD GPUs ever again by the way.
Which SemiAnalysis article?
ROCm is a mistake. It's fundamentally broken by compiling to hardware specific code instead of CUDA's RTX, so it will always be plagued with this issue of not supporting all cards, and even if a certain GPU is supported today they can stop supporting it next version. It has happened, it will continue happen.
It's also a strange value proposition. If I'm a programmer in some super computer facility and my boss has bought a new CDNA based computer, fine, I'll write AMD specific code for it. Otherwise why should I? If I want to write proprietary GPU code I'll probably use the de facto industry standard from the industry giant and pick CUDA.
AMD could be collaborating with Intel and a myriad of other companies and organizations and focus on a good open cross platform GPU programming platform. I don't want to have to think about who makes my GPU! I recently switched from an Intel CPU to an AMD, obviously to problem. If I had to get new software written for AMD processors I would have just bought a new Intel, even though AMD are leading in performance at the moment. Even Windows on ARM seems to work ok, because most things aren't written in x86 assembly anymore.
Get behind SYCL, stop with the platform specific compilation nonsense, and start supporting consumer GPUs on Windows. If you provide a good base the rest of the software community will build on top. This should have been done ten years ago.
Agreed.
Honestly, the problem isn't just which devices, but even more so, this (from the page, not your comment):
> No guarantees of future support but we will try hard to add support.
During the Great GPU Shortage, I bought an AMD RX5xx card for ML work. It was explicitly advertised to work with ROCm. Within a couple of months, AMD dropped ROCm support. EOLing an actively-sold product from being used for an advertised purpose within the warranty period was, if I understand consumer protection laws in my state correctly, fraud. There was no support from either the card vendor (MSI). No support from AMD. No support from the reseller. Short of small claims, which was not worth it, there was no recourse.
This is on a long list of issues AMD needs to sort out to be a credible player in this space:
* Those are the kinds of experiences which cause people to drop a vendor and not look back. AMD needs to either support cards forever, or at the very least, have an advertised expiration date (like Chromebooks and Android phones).
* Broad support is helpful from a consumer perspective from the simply pragmatic point of view that only a tiny fraction of the population has the time to read online forums, footnotes, or fine print. People should be able to buy a card on Amazon, at Best Buy, and Microcenter, and expect things to Just Work.
* Being able to plan is essential for enterprise use. I can't build a system around AMD if AMD might stop supporting their platform on 0 days notice, and the next day, there might be a security exploit which requires a version bump.
I'm hoping Intel gets their act together here, since NVidia needs a credible competitor. I've given up on AMD.
PTX does provide a low level machine abstraction. However you still target some version of hardware ( https://arnon.dk/matching-sm-architectures-arch-and-gencode-... ). However a lot of software effort has gone into it to make it look and work seamlessly.
Though AMD doesn't have the same "virtual ISA" as PTX right now there are increasing levels of such abstraction available in compiled flows with MLIR / Linalg etc. Those are higher level and can be compiled / jitted in realtime to obviate the need for a low level virtual ISA.
We already fought and lost this battle with 3D APIs for GPUs. What makes you think that winning strategy would play out any other way for tensor processing?
Really telling they have to ask us for what cards we want as opposed to supporting all cards by default from day 1 like Nvidia.
All because they went with a boneheaded decision to require per-device code compilation (gfx1030, gfx1031...) instead of compiling to an intermediate representation like CUDA's PTX. Doubly boneheaded considering the graphics API they developed, Vulkan, literally does that via SPIR-V!
Really telling who comments before reading :)
The author of the issue comments that they'll eventually support all cards. What he really is asking for, is what cards people want them to prioritize, not just support.
I’m constantly baffled and amused on why AMD keeps majorly failing at this.
Either the management at AMD is not smart enough to understand that without the computing software side they will always be a distant number 2 to NVIDIA, or the management at AMD considers it hopeless to ever be able to create something as good as CUDA because they don’t have and can’t hire smart enough people to write the software.
Really, it’s just baffling why they continue on this path to irrelevance. Give it a few years and even Intel will get ahead of them on the GPU side.
If I were Jensen, I would snap up all the GPU software experts I possibly could, and put them to work improving the CUDA ecosystem. I'd also spin up a big research group to further fuel the CUDA pipeline for hardware, software, and application areas.
Which is exactly what NVIDIA seems to be doing.
AMD's ROCm software group seems far behind, is probably understaffed, and probably is paid a fraction of what NVIDIA pays its CUDA software groups.
AMD also has to catch up with NVlink and Spectrum-X (and/or InfiniBand.)
AMD's main leverage point is its CPUs, and its raw GPU hardware isn't bad, but there is a long way to go in terms of GPU software ecosystem and interconnect.
I've never understood why they have such a fractured approach to software:hardware support. I remember reading and writing comments about this on hn nearly a decade ago now. It's a long time to keep making the same mistake.
They had the exact same kind of support issues back in the OpenCL days, where they didn't manage to provide cross platform, cross card support for same versions of the platform.
I have never been able to reconcile it with their turnaround and newfound competence on the CPU side.
> I’m constantly baffled and amused on why AMD keeps majorly failing at this.
i wonder if you've considered the possibility that there's some component/dimension of this that you're simply unaware of? that it's not as straightforward as whatever reductive mental model you have? is that even like within the universe of possibilities?
My wishlist for ROCm support is actually supporting the cards they already released. But that's not going to happen.
By the time an (consumer) AMD device is supported by ROCm it'll only have a few years of ROCm support left before support is removed. Lifespan of support for AMD cards with ROCm is very short. You end up having to use Vulkan which is not optimized, of course, and a bit slower. I once bought an AMD GPU 2 years after release and 1 year after I bought it ROCm support was dropped.
FWIW, every ROCm library currently in the Debian 13 'main' and Ubuntu 24.04 'universe' repository has been built for and tested on every discrete consumer GPU architecture since Vega. Not every package is available that way, but the ones that are have been tested on and work on Vega 10, Vega 20, RDNA 1, 2 and 3.
Note that these are not the packages distributed by AMD. They are the packages in the OS repositories. Not all the ROCm packages are there, but most of them are. The biggest downside is that some of them are a little old and don't have all the latest performance optimizations for RDNA 3.
Those operating systems will be around for the next decade, so that should at least provide one option for users of older hardware.
As the underdog AMD can't afford to have their efforts perceived as half-assed or a hobby or whatever. They should be moving heaven and earth to maximize their value proposition, promising and delivering on longer support horizons to demonstrate the long term value of their ecosystem.
I have a mi50 with 16gb of hbm thats collecting dust (its Vega bases, so it can play games, I guess) because I don’t want to bother setting up a system with Ubuntu 20.04, the last version of Ubuntu the last version of ROCM that supported the MI50 works on.
With situations like this, its not hard to see why Nvidia totally dominates in the compute/ai market.
Seeing Radeon VII on the deprecation list is a little saddening, unless they start putting out more 16gb+ GPUs that aren't overly expensive...
They should have at a minimum 5 year support release cycle.
AMD has separate architectures for GPU compute (Instinct https://www.amd.com/en/products/accelerators/instinct/mi300....) and consumer video (Radeon).
AMD are merging the architectures (UDNA) like nVidia but it's not going to be before 2026. (https://wccftech.com/amd-ryzen-zen-6-cpus-radeon-udna-gpus-u...)
I can understand wanting to prioritize support for the cards people want to use most, but they should still plan to write software support for all the cards that have hardware support.
I've long since given up on my 5700xt getting supported. AMD is just not a good pick if you care about non graphics compute.
Imagine Nvidia not supporting CUDA on any of their cards. Unthinkable.
this is a-posteriori development.. we have no idea of how hard it is to implement with older GPUs
rocm is kind of a joke. Recently I wanted to write some golang code which talks to rocm devices using amd smi. You have to build and install the go amd smi from source, the go amd smi repo has dead links and there is basically no documentation anywhere on how to get this working.
Compare this to nvidia where I just imported the go nvml library and it built the cgo code and automatically links to nvidia-ml.so at runtime.
Is this the repo you are referring to https://github.com/amd/go_amd_smi ? Would having a prebuilt version there help you ?
AMD supports only a single Radeon GPU in Linux (RX 7900 in three variants)?
Windows support is also bad, but supports significantly more than one GPU.
Imagine nvidia supported only the 4090, 4080 and 4070 for cuda at the consumer level. With the 3090 not being supported since the 40xx series came out. This is what amd is defending here.
I honestly can't figure out which Radeon GPUs are supposed to be supported.
The GitHub discussion page in the title lists RX 6800 (and a bunch of RX 7xxx GPUs) as supported, and some lower-end RX 6xxx ones as supported for runtime. The same comment also links to a page on the AMD website for a "compatibility matrix" [1].
That page only shows RX 7900 variants as supported on the consumer Radeon tab. On the workstation side, Radeon Pro W6800 and some W7xxx cards are listed as supported. It also suggests to see the "Use ROCm on Radeon GPU documentation" page [2] if using ROCm on Radeon or Radeon Pro cards.
That link leads to a page for "compatibility matrices" -- again. If you click the link for Linux compatibility, you get a page on "Linux support matrices by ROCm version" [3].
That "by ROCm version" page literally only has a subsection for ROCm 6.2.3. It only lists RX 7900 and Pro W7xxx cards as supported. No mention of W6800.
(The page does have an unintuitively placed "Version List" link through which you can find docs for ROCm 5.7 [4]. Those older docs are no more useful than the 6.2.3 ones.)
Is RX 6800 supported? Or W6800? Even the amd.com pages seem to contradict each other on the latter.
Maybe the pages on the AMD site only list official production support or something. In any case it's confusing as hell.
Nothing against the GitHub page author who at least seems to try and be clear but the official documentation leaves a lot to be desired.
[1] https://rocm.docs.amd.com/projects/install-on-linux/en/lates...
[2] https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com...
[3] https://rocm.docs.amd.com/projects/radeon/en/latest/docs/com...
[4] https://rocm.docs.amd.com/projects/radeon/en/docs-5.7.0/docs...
Super annoying. I have an RX 6600 XT and can't get ROCm to work on Linux.
Vulkan ML however worked perfectly out of the box, so at least I got something.
Just weird the official thing doesn't work.
I found that striking as well. Does AMD expect everyone wanting to try out PyTorch or LLMs on Linux to splurge on Instinct servers?
Add support for every APU. They can have much more RAM than discrete graphics.
Why are people in AMD assuming other people don't want more software support for their GPUs by default? This is not nice.
Because they don't have infinite resources like nVidia so they're asking what people want the most to prioritise it.
Please read the link before commenting on future. We do that here. This info is is an early comment by an AMD employee.
I figure that list is only what’s officially supported, meaning things not on that list may or may not work?. For example, my 6800 XT runs stable diffusion just fine on Linux with PyTorch ROCm.
What’s the performance like? Was it easy to set up?
A lot of people think rocm is basically a big pile of crap.
What are the chances for amd to consider alternatives: - adopt oneapi and try to fight Nvidia together with intel - Vulkan and implement pytorch backend - sycl
They should just support all cards. Just like Nvidia does.
And they drop support too quickly too. The Radeon Pro VII is already out of support. It's barely 5 years since release.
This way it will never be a counterpart to CUDA.
Really hoping for support for an AMD Radeon Pro W5700 I have kicking around.
As someone from the rendering side of GPU stuff, what exactly is the point of ROCm/CUDA? We already have Vulkan and SPIR-V with vendor extensions as a mostly-portable GPU API, what do these APIs do differently?
Furthermore, don't people use PyTorch (and other libraries? I'm not really clear on what ML tooling is like, it feels like there's hundreds of frameworks and I haven't seen any simplified list explaining the differences. I would love a TLDR for this) and not ROCm/CUDA directly anyways? So the main draw can't be ergonomics, at least.
users mainly use PyTorch and Jax and these days rarely write CUDA code.
however separately, installing drivers and the correct CUDA/CuDNN libraries is the responsibility of the user. this is sometimes slightly finicky.
with ROCm, the problem is that 1) PyTorch/Jax don't support it very well, for whatever reason which may be partly to do with the quality of ROCm frustrating PyTorch/Jax devs, 2) installing drivers and libraries is a nightmare. it's all poorly documented and constantly broken. 3) hardware support is very spotty and confusing.
Vulkan doesn't do C++ as shading language for example, there are some backend attempts to target SPIR-V, but it still is early days and nowhere close of having the IDE integration, graphical debugging tools and rendering libraries that CUDA enjoys.
Examples of rendering solutions using CUDA,
https://www.nvidia.com/en-us/design-visualization/solutions/...
https://home.otoy.com/render/octane-render/
It is definitely ergonomics and tooling.
Cuda the language is an antique dialect of C++ with a vectorisation hack. It's essentially what you get if you take an auto-vectoriser and turn off the correctness precondition, defining the correct semantics to be that which you get if you ignore dataflow. This was considered easier to program with than vector types and intrinsics.
Cuda the ecosystem is a massive pile of libraries for lots of different domains written to make it easier to use GPUs to do useful work. This is perhaps something of a judgement on how easy it is to write efficient programs using cuda.
ROCm contains a language called HIP which behaves pretty similarly to Cuda. OpenCL is the same sort of thing as well. It also contains a lot of library code, in this case because people using Cuda use those libraries and don't want to reimplement them. That's a bit of a challenge because nvidia spent 20 years writing these libraries and is still writing more, yet amd is expected to produce the same set in an order of magnitude less time.
If you want to use a GPU to do maths, you don't actually need any of this stuff. You need the GPU, something to feed it data (e.g. a linux host) and some assembly. Or LLVM IR / freestanding c++ if you prefer. This whole cuda / rocm thing really is intended to make them easier to program.
i really need amd to make an apu with eight channels and ddr5
Latest AMD miniPCs APUs have DDR5 memory (up to 96GB). Don't know about the channels.
But this APU hack might work:
https://blog.machinezoo.com/Running_Ollama_on_AMD_iGPU https://github.com/ollama/ollama/pull/6282
Linux since 6.1 made some changes to allocate the memory to the GPU from userspace, but it seems the GTT method has degrated performance even more.