alt Hacker NewsDidn't the Postgres team just have to stop a release because the agents found a slew of bugs in human code from domain experts that "understood and verified it"?
Perhaps I'm being overly generous to the parent's point, but it stands to reason that we make a lot of choices of where and how much trust we give away. This is just another one of those points.
I would say that a calculator and a database like postgres differ quite significantly, though. How many PRs are merged per month on your average calculator? How many lines of code is in that firmware?
Trusting that a calculator works is like trusting that there will be another day tomorrow, while trusting the code of an LLM is like trusting that the current weather will hold for the next hour.
Also, LLM finding bugs in human written code doesn't mean it wouldn't find much more in LLM generated code, and spend much more money doing so. That is, of course, after ruling out the false positives (of which there are a few, judging by the PRs I've received on my open source projects)