alt Hacker News_Is_ AI generated code notoriously terrible at security?
My experience with AI-code is that it's extremely defensive and good at minimizing 'obvious' mistakes.
Are we comparing to old models generating code? Or mythos-class models?
Replies
Physician heal thyself?
Tell an LLM to break into its own system as part of a feed back loop, but with no context of the initial creation? It should iteratively get to 'safe' shouldn't it (subject to the tokens you want to spend)?
It is not just code, it is also deploying Software. I constantly have to correct or steer the 'ai' in conversations towards safer directions or better following even the most basic security principles. Here I feel it is still more the overambitious intern that needs to be carefully overseen, but has a lot of time and energy at hand.
Yeah they are pretty bad at it still, latest models tried to ship a basic SQL injection flaw. Was also happy to build out an xss flawed module end-to-end. Definitely have to keep a close eye on it.
My vibe code friends have pretty much infinite security issues. Don't even have basic data modeling at client-server boundaries or anything. Also had Google analytics embed that read and saved everyone's credit card details as clear text in analytics.
Seems to make pretty much infinite mistakes if you don't take care and tell it what not to do or how to build certain features.
What models are people actually using to pump out code? That is the relevant comparison, not the absolute best in class models.
It's been shown to be bad at secure coding _unless_ you specifically prompt models to watch out for common vulnerabilities, at which point they're actually quite good at containing the issues, though the studies here are somewhat older (eg https://arxiv.org/abs/2211.03622).
The issue then is that most people aren't actually good at keeping an eye out for security issues in code proactively, so things slip through the cracks (whether human-written or not), just at a much accelerated pace.
like most software, do you assume people are driving lamborghinis, or the cheapest make and model they can find...
It very much depends on how you generate it, from prompting to test suites, but most importantly, your attitude. The extra security costs you time and money.
There are still folks, many of them, who think that time to market is everything, and anything beside it should be ignored. "It compiles? Ship it!" is a phrase from 1990s; LLMs just make this approach much easier. We will have plenty of machine-generated software with the security posture of a colander hit by a buckshot.