logoalt Hacker News

metalliqaztoday at 2:48 PM3 repliesview on HN

better than Vulkan?


Replies

cpburns2009today at 2:58 PM

In my experience using llama.cpp (which ollama uses internally) on a Strix Halo, whether ROCm or Vulkan performs better really depends on the model and it's usually within 10%. I have access to an RX 7900 XT I should compare to though.

show 1 reply
0x457today at 5:30 PM

For me Vulkan performs better on integrated cards, but ROCm (MIGraphX) on 7900 XTX.

hrmtst93837today at 4:06 PM

Wrong layer. Vulkan is a graphics and compute API, while Lemonade is an LLM server, so comparing them makes about as much sense as comparing sockets to nginx. If your goal is to run local models without writing half the stack yourself, compare Lemonade to Ollama or vLLM.

show 1 reply