alt Hacker NewsIt strikes me that more tokens likely give the LLM more time/space to "think". Also that more redundant tokens, like local type declarations instead of type inference from far away, likely often reduce the portion of the code LLMs (and humans) have to read.
So I'm not convinced this is either the right metric, or even if you got the right metric that it's a metric you want to minimize.
Replies
make3 • today at 5:22 AM
With Chain of Thoughts (text thinking), the models can already use as much compute as they want in any language (determined by reinforcement learning training)
➕ show 1 reply
I think separating thinking tokens from "representing" tokens might be a better approach, like what those thinking models does