alt Hacker NewsYou probably can just use Vulkan. Why don't you think Vulkan is an alternative? Can it not handle BLAS and memory access? What is a graphics card supposed to be doing that isn't efficient in Vulkan?
You probably can just use Vulkan. Why don't you think Vulkan is an alternative? Can it not handle BLAS and memory access? What is a graphics card supposed to be doing that isn't efficient in Vulkan?
Sometimes it’s just that a particular backend is poorly optimized or has a regression on a particular platform as compared to the “mainstream” backends. For example, whisper.cpp’s Vulkan backend performs 2-3x worse than CPU on my Snapdragon X2 laptop when using the ggerganov v3 turbo model. It’s probably a simple fix, but it does need to be fixed.