logoalt Hacker News

selcukalast Monday at 6:48 AM0 repliesview on HN

We are talking about accuracy, though. I don't see the point of MoE if a 120B MoE model is not as accurate as even a 32B model.