alt Hacker NewsIt remains viable as long as frontier models remain subsidized, too. How long will it take before these providers have to raise the prices to such a degree that such attacks would be reduced to only the most determined, financially capable attackers?
And even then there is the risk that the frontier model providers collapse. There’s no indication yet that these companies are going to become profitable. And open source models simply piggy back on frontier ones and are generally not powerful enough for this level of adversarial attacks as far as I know.
And finally are LLMs the only method to hardening software? There is still a lot left on the table that could still allow a project to resist attacks from a frontier model.
Sure, humans are bad at catching this stuff. But in order to drive an LLM you have to be able to catch this stuff. Otherwise your only option is to trust the model and give up.
To the point about LLMs being the only method to harden software - that's why I'm not sure I'm convinced by the argument. The article says the bulk of the defects were integer overflow bugs. This is out of my league a bit, but that feels like the kind of thing that should be detectable with static analysis, possibly both less expensively and more reliably than with LLMs.
For example: https://link.springer.com/article/10.1186/s42400-020-00058-2
Perhaps using static analysis generates false positives in cases where the code can't be proven safe? But when I was working on a project where we used Sonarqube, we ended up deciding as a team that we'd prefer changing the code to eliminate false positives over "wontfix"ing them, and I was happy with that decision. It led to more regular coding practices that ultimately made the codebase easier to read and understand. For largely the same reasons as Dijkstra was getting at in "Go To Statement Considered Harmful."