logoalt Hacker News

Davidzhengtoday at 9:10 AM0 repliesview on HN

Even if they are "simplistic projections", which I don't think is the correct way to think about it, there's no reason that more LLM thoughts in middle layers can't also exist and project down at the end. Though there might be efficency issues because the latent thoughts have to be recomputed a lot.

Though I do think in human brains it's also an interplay where what we write/say also loops back into the thinking as well. Which is something which is efficient for LLMs.