alt Hacker Newsbut you have to rewrite all your software to ROCm. And ROCm, to this day, still sucks.
ZLUDA basically crash on high memory demand, and only accounted for ~70% of CUDA API coverage (and it is still buggy).
But hey, at least it does run on Rust-CUDA. I'm one of the few who ported it and fixed a few bugs on ZLUDA. I used it to run a simple SHA256 kernel and it ran sure, but I gave it up because of those fundamental problems on AMD GPUs. You can't believe how messy ROCm is. I wonder if Vulkan compute kernel using SPIR-V would be a better choice.
Replies
Co-founder and Chief Compute Officer Anthropic:
> I think the thing that we were thinking about originally was whenever we're bringing up a new hardware platform, it's a big effort. It's like a huge thing. And so as we were thinking about this, we started doing our own evaluation of MI 355. You guys generously got us a rack to start working. And we expected this to be kind of a big process.
> Our actual experience was we had one engineer who start doing it. They spun up Claude, asked it, hey, bring up this machine, left it going over the weekend. And we ended up with a graph of the actual performance of our leading model on it, just going up and up and up over the weekend.
Spir-v is already happening: https://rocm.blogs.amd.com/software-tools-optimization/spir-...