logoalt Hacker News

hedgehogtoday at 7:26 PM0 repliesview on HN

Separating knowledge from reasoning so you only pay for what you use is a big rationale for MoE, the big problem being MoE training has historically been hard to get right. In a dense model every single token you're paying a cost to determine whether you're now talking about the flavor of durian.