alt Hacker NewsI don't think that's a suitable mental model of how LLMs work. Their output is not just the statistical average of all the training data. If it was then the output would be quite useless.
I don't think that's a suitable mental model of how LLMs work. Their output is not just the statistical average of all the training data. If it was then the output would be quite useless.
An LLM trained on the entire internet incorporates essentially all (human-known) aspects of things.
Like a superposition of all kinds of writing styles, all perspectives on every topic and all kinds of ways to do things right and wrong.
In post-training (and of course already by sensibly curating the input data in the first place), people try to select useful submanifolds from that.
Ideally, you get a combination of all the extant strengths of people combined into one model. Self-improvement then leaves the sphere of human capability for good.