alt Hacker NewsI worry this is looking at where the ball is now instead of where it's going. The recent disproof of an Erdos conjecture should put to rest the idea that LLMs will reach a skill ceiling before they reach superintelligence.
I believe we are headed for a world of superintelligent AI where LLMs are much better at logical thinking than humans, the same way that chess engines are much better at chess than humans.
In that world there's really nothing humans can offer in terms of logical thinking other than their humanity itself. An 8 year old with Stockfish can beat Magnus Carlsen, and an 8 year old with Codex (and daddy's credit card) will be able to beat me at software engineering.
Replies
I personally don't see that much sense worrying about this scenario because if it comes true then it doesn't really matter what I do.
Are you confident in putting a timeline on this prediction?
One of the reasons I'm increasingly skeptical of this prediction is that I've now lived past a few of the dates I heard people put on the achievement of this level of superintelligence in previous years.
I don't buy that at all.
It doesn't matter how great the LLMs get, the act of creating software using them will still require a great deal of skill.
Most people just don't think in terms of software.
Try asking a non-developer in your life what their dream software would be for their work, or their hobby. If they don't have what Nilay Patel calls "software brain" I'd be surprised if they came up with something actionable.
(For more on software brain see "THE PEOPLE DO NOT YEARN FOR AUTOMATION", which makes the point I"m making here but much, much better: https://www.theverge.com/podcast/917029/software-brain-ai-ba...)
You could give a non-developer the smartest LLM in the world and they wouldn't be able to create GitHub with it, because creating GitHub requires an enormous amount of understanding of what software developers need from a cloud source control tool.
Sure, you can argue that the LLM "knows" what GitHub needs already and can guide their human-user to that, but why would a human-user who doesn't understand the domain ask an LLM to do that in the first place?