logoalt Hacker News

djsjajahtoday at 12:19 AM1 replyview on HN

trl. give me a uv command to get that working.

But even in the amd stack things (like ck and aiter) consumer cards are not even second class citizens. They are a distance third at best. If you just want to run vllm with the latest model, if you can get it running at all there are going to be paper cuts all along the way and even then the performance won't be close to what you could be getting out of the hardware.


Replies

latchkeytoday at 12:21 AM

It is not perfect, but it isn't that bad anymore. Tons of improvements over the last year.