logoalt Hacker News

syntaxingyesterday at 4:42 PM1 replyview on HN

Yes and no. Are you using open router or local? Are the models are good as Opus? No. But 99% of the time, local models are terrible because of user errors. Especially true for MoE, even though the perplexity only drops minimal for Q4 and q4_0 for the KV cache, the models get noticeably worse.


Replies

acidtechno303yesterday at 4:59 PM

Sounds like you're accusing a professional of holding their tool incorrectly. Not impossible, but not likely either.

show 1 reply