alt Hacker NewsAlternative(s) to run CUDA on non-Nvidia hardware
Comments
Most of these "alternatives" focus on CUDA C++, and overlook what actually makes CUDA interesting.
Already in 2020,
https://developer.nvidia.com/blog/cuda-refresher-the-gpu-com...
In this context AdaptiveCpp should also be mentioned. Started as a SYCL implementation, but recently-ish added a compiler for compiling a CUDA dialect to GPUs and CPUs from basically all vendors
Alternatives exist, but little demand outside hyperscalers and special uses.
Neocloud customers just want plug-and-play CUDA. It works, it's tested, it adapts faster, and has known performance. Alternatives give no significant benefits.
Things can change, but they are not changing now.
every CUDA alternative follows the same arc: bold launch, works for 3 operations, then a Discord server where the last message is 'any updates?' from 2024
There is also ZLUDA, which is open source and works on pre-compiled binaries.
Why should I not just port my kernel to Triton? What's the appeal of Scale?
Isn't the future of the industry specialized chips like those that Broadcom and Cerebras are making? I don't know how much longer I can tolerate 50 tokens per second. It feels like the dial-up era.
aren't llms smart enough to directly write custom kernels for custom hardware from cuda code?
I have been trying for cuda -> metal, to run it on mac, https://github.com/lulzx/cuda-metal
Its easier to just get rid of your legacy code entirely and use Vulkan for compute, or have your compiler emit SPIR-V directly.
No reason to tie yourself to Nvidia's moat.
[dead]
[dead]
There's nothing wrong to run CUDA on non-Nvidia hardware. CUDA has an interface that is reasonably well-designed, well-documented/reverse-engineered, and battle-tested for decades. What we need is not to invent another interface just under the name of 'open standard', but to implement the same interface. ROCm is exactly doing this, and so are other hardware SDKs such as MooreThread and Alibaba T-Head.