logoalt Hacker News

ahartmetz • today at 7:31 PM • 0 replies • view on HN

Running locally is not a problem if you allow it to be slow. I get about 3.5 tps with Qwen3.8-27B:UD-Q4_K_XL on my 64 GB laptop and about the same tps with Qwen3.8-Flash-Next:UD-IQ4_XS (IIRC, or UD-Q4_K_XL) on my 128 GB desktop with a 16 GB GPU.