logoalt Hacker News

3abitonyesterday at 9:18 PM0 repliesview on HN

To be fair, it's "possible" to run such setup with llama.cpp with ssd offload. It's just abysmal TG speeds. But it's possible.