alt Hacker NewsBut if I may steelman the article a bit:
What was good enough in the past may not be good enough now. In the past these overflow defects were as hard for attackers to find as they were for developers, because they had the same tools available.
Now we have LLMs, and they can apparently find all sorts of problems that, for whatever reason, weren’t being found with manual review, static analysis and fuzzing. We have to assume that hackers will use the technology to find vulnerabilities. If maintainers don’t do the same, then they are ceding an advantage and leaving their users unnecessarily exposed.
I don’t know that I completely agree with the above. (For example, I don’t know how scrupulous GNOME has been in the past about non-AI tools for automated defect discovery, or how well they compare to AI.) But it at least feels like a much more charitable interpretation of the article’s main thrust.
It remains viable as long as frontier models remain subsidized, too. How long will it take before these providers have to raise the prices to such a degree that such attacks would be reduced to only the most determined, financially capable attackers?
And even then there is the risk that the frontier model providers collapse. There’s no indication yet that these companies are going to become profitable. And open source models simply piggy back on frontier ones and are generally not powerful enough for this level of adversarial attacks as far as I know.
And finally are LLMs the only method to hardening software? There is still a lot left on the table that could still allow a project to resist attacks from a frontier model.
Sure, humans are bad at catching this stuff. But in order to drive an LLM you have to be able to catch this stuff. Otherwise your only option is to trust the model and give up.