alt Hacker NewsCubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
Comments
This reminds me of Halide (https://halide-lang.org/).
In Halide, the concept was great, yet the problems in kernel development were moved to the side of "scheduling", i.e. determining tiling/vectorization/parallellization for the kernel runs.
Love it. I've been using cudarc lately; would love to try this since it looks like it can share data structures between host and device (?). I infer that this is a higher-level abstraction.
Very interesting project! I am wondering how it compare against OpenCL, which I think adopts the same fundamental idea (write once, run everywhere)? Is it about CUbeCL's internal optimization for Rust that happens at compile time?
Gotta say, the constant dance between all these GPU frameworks kinda wears me out sometimes - always chasing that better build, you know?
See also this overview for how it compares to other projects in the Rust and GPU ecosystem: https://rust-gpu.github.io/ecosystem/
Praying to the kernel gods for some Rust FP8 training
wow, what's the downsides to this? It feels like it could be one of the biggest leaps in programming in a long time, does it keep rusts safety aspects? How does it compare with say openCL?
I'd recommend having a "gemm with a twist" [0] example in the README.md instead of having an element-wise example. It's pretty hard to evaluate how helpful this is for AI otherwise.
[0] For example, gemm but the lhs is in fp8 e4m3 and rhs is in bf16 and we want fp32 accumulation, output to bf16 after applying GELU.