logoalt Hacker News

saejoxyesterday at 10:12 PM1 replyview on HN

Most of those people will be dissapointed when they experience Q4 variants of those models getting stuck in loops.

I would wait till the ram crisis is over to fetch a future 64gb ram gpu to run Q8 models. Cloud inference until than.


Replies

etoxinyesterday at 10:44 PM

Switching from Q4 to Q8 was a game changer when I upgraded