I really like the distinction between "prediction" and "generation" here. Thank you for putting that into words so clearly.
The generation is done by... prediction. It is making a guess based on weights (statistical likelihood) plus a pinch of wobbliness.
It's worth reflecting on the original domain jargon for selecting the next token: completion.
alt Hacker News
The generation is done by... prediction. It is making a guess based on weights (statistical likelihood) plus a pinch of wobbliness.
It's worth reflecting on the original domain jargon for selecting the next token: completion.