alt Hacker NewsA long time ago I noticed that I sometimes already had a complete thought before my inner monologue turned it into words. A few times I tried skipping the inner monologue because I'd clearly already thought the thought. Turns out the bit of my brain that creates the inner monologue from the thought, can generate a sense of annoyance that the rest of my brain can feel.
Not that it matters, there's evidence that while LLMs output one word at a time, they've got forward-planning going on, having an idea of the end of a sentence before they get there.
Indeed, and it seems like they would really struggle to output coherent text at all if there was not some kind of pre-planning involved (see how even humans struggle with it in games where you have to construct a sentance by having each person shout out one word at a time). Even GPT-2 likely had at least some kind of planning for the next few words in order to be as coherent as it was.