I think because we have handicapped them with a set of tokens from human language. But (at least I think so) thought happens outside of language.

So more tokens/variability and slow or fewer tokens and fast.

There seems to be a threshold tho, like taalas is super fast but that model is so dumb, being dumb faster doesn't work, seems to be some minimum requirements.

Currently AGI has been re-defined mostly as "can do any tasks (of the same modality) a human can, equal or better". I feel the "equal or better" includes properties like how fast they can get the task done, not just that eventually they can succeed at it. If you want to have AGI stock buy/sell for example, speed of decision will matter, and there are many other use-cases for which it would matter, not all, but definitely for many tasks, speed does matter.

You can imagine with more operations being available to be done more cheaply and quickly the LLM doesn't need to "one shot" a solution. It could try many solutions, test them, throw some away, wiggle some of the parameters like a genetic algorithm, see how that changes the result, and converge on an optimal solution (based on whatever the cost function is). Basically producing a good result could become like an optimization problem. That would be way too expensive and slow right now.

Imagine Luna at 10x tps and 1/100 of current cost.

At that point you will be able to "brute force" basically everything.

IMO also a lot of problems with memory and context rot will be solved too.