alt Hacker NewsIf all you're going to do is chat with an AI, you don't need a 400B parameter MoE model.
13 TPS is unusable for agentic coding, because if decode is that slow, then prefill is absolutely abysmal, and agentic work live and die by prefill speeds.
Replies
genxy • today at 7:07 PM
> If all you're going to do is chat with an AI, you don't need a 400B parameter MoE model.
Why wouldn't I want the smartest possible model I can get? I only "chat" about smart person stuff.
I’ve been agentic coding at home with like 15 TPS on average. I don’t mind it at all. I can give my agent a big task or a list of little ones and just… do something else. Why do I need to watch it work?