logoalt Hacker News

bearjawstoday at 4:19 PM0 repliesview on HN

Given how "smart" some of the 26b dense models are now, I would not be surprised to see a strong 40b MoE.