alt Hacker NewsI wrote this partly because nobody seems to know about Gödel who is arguably a more important figure than Turing.
The piece follows four threads where formal limits show up in AI practice today: learnable problems that mathematics cannot sort, neural networks that exist but cannot be trained, self-improving systems that dropped their safety proofs for benchmark scores, and the fact that “will this AI cause harm” is, mathematically, an unanswerable question.
The piece is long and the math is dense. I’m curious whether people see the connection as tight or as overextended—whether Gödel’s structural limits actually do constrain what we’re doing now, or whether I’m collapsing categories that should stay separate.
Replies
Gödel, Escher, Bach (1980, Pulitzer prize) was how I first got exposed to Gödel.
I’m curious how much you used AI to write it, as it’s just chock full of Claude-isms.
I think there is a loose connection. Hofstadter's 1979 masterpiece, Gödel Escher Bach (https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach --- mentioned in an earlier reply), has a chapter called Contracrostipuntus (https://genius.com/Douglas-hofstadter-contracrostipunctus-an...) which describes an arms-race tension between Crab's "perfect" record player and Tortoise's mischievous records designed to be unplayable on that record player, with titles such as "I Cannot Be Played on Record Player X". Crab's then invents a "Record Player Omega" which includes an ahead-of-time "scanning" step to reconfigure itself based on the record it scans...
All of which reminds me of virus-scanning on computers, and AI efforts for "safety".
Making the connection between GEB and AI isn't new, apparently: https://matthodges.com/posts/2025-08-26-music-to-break-model...
If the theorem is truly general ie any algorithm not just certain classes of neural net training methods, then is a moot point. Computers, humans or any other physical system can't solve it.
> the fact that “will this AI cause harm” is, mathematically, an unanswerable question.
No it isn't -- this is a fundamental misunderstanding. What is unanswerable is "For all x where x is an AI, will x cause harm?" ... but there is an infinity of specific AIs that provably will or won't cause harm.
Likewise this a very common misunderstanding of the Halting Problem -- Turing proved that there is no TM that can prove whether m will halt for all possible TMs m ... but there are myriad TMs that provably do halt or provably don't halt.
This is a wild statement to me that “nobody seems to know about Gödel”… I think I learned about him when I was like 12? And while I’ve certainly met plenty of people who haven’t heard of him, until 10-15 years ago the same was true about Turing, mostly changed by a some Hollywood and/or literary references… and Gödel had a tiny bit of that in Oppenheimer if i recall, though not to the same degree? I bet if you asked 100 comp sci students if they’ve heard of Gödel at least 50 would say yes? On what basis do you think no one has heard of him? Not many have read the original math, sure, but… the idea that he was a mathematician who wrote an incompleteness theorem? VERY widely known.