logoalt Hacker News

Leif24 • today at 6:39 PM • 1 reply • view on HN

Re: running qwen 3.8 27b on a 16gb card - that isn't a MoE model, was it just heavily quantized? How much context window were you getting?

Just starting to dive in on local AI with my own 16gb card, and the struggling with evaluating what tradeoffs are worth it.


Replies

lrvick • today at 7:42 PM

On a 16gb card check out Bonsai2. I run it on my 6800XT.

➕ show 1 reply