alt Hacker NewsI need to do some more research on this. Because I've always been told that AI, more specifically LLMs, are "glorified autocompletes". Yes they have huge data sets but ultimately they can only provide answers based on data that they were trained on.
So how can they possibly solve problems that humans have worked for decades or centuries on without being able to solve? How did they find something new? Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
Replies
Because the "glorified autocomplete" crowd likes to ignore the technical detail of how the autocomplete actually works.
They have access to a much larger corpus of information than any single human can manage in his one brain. No human mathematician reads all the papers and has perfect recall on them. The answers have always been there, but no individual has been able to make all the right connections to pull them together and formalize a new discovery or proof.
> Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
Yes, both of these + an infinite token/compute budget and SOTA+ models. It's not like some hobbyist out in the woods with a GPU running Qwen 27B can do this.
[dead]
The following certainly does not describe all mathematics or all LLM operations, but its my mental model for why LLMs are so effective in several fields.
Lots of mathematics entails recognizing that a tool/techniques that works in one area can be adapted to a completely different problem. For a human to do this requires having a robust mental model of the technique and the problem itself, and probably having just the "right" mental model to see how they fit together. And we are talking about extremely hard/sophisticated techniques and extremely hard/sophisticated problems, so the ability to deeply appreciate either can require years of study. And on top of that, even if the mathematician has sufficient understanding to develop the intuition that the technique might work, they will need profound fortitude, skill, and patience to see the solution through.
Unlike human mathematicians, who will develop a deep understanding of only a relative handful of techniques and problems in their lifetime, LLMs have ALL written techniques, problems, solutions, etc stored in their internal representations. So when they ably match a problem to a technique, we can imagine the operation as a supercharged autocomplete, or even an extremely proficient search engine. Their stroke of genius is really a consequence of capacity, moreso than cleverness.