alt Hacker NewsEverything I've read indicates that yes, this is the new normal.
When Mythos was first announced, pretty much every security expert I talk to freaked directly out. A significant portion of my job now revolves around mitigating the impact of AI-generated exploits.
The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.
Replies
Can you explain the asymmetry you are claiming that exists where some models would be excellent at finding faults in code/systems and exploiting them often through coding the required tools to exploit them... but at the same time they would be bad at avoiding this kind of faults when coding the system itself?
Feels like you should be able to prompt the kind of exploits you want to prevent and the model should be able to make the system secured against them, no?
You just need to use AI to test your AI code.
(What a wonderful new world we live in)
> The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.
I think this was true in the past but AI is helping us finding exploits which have been in our code for many years. All out there by humans. We’re seeing fewer customer reported issues, and despite the pen tests getting far more rigorous, we’re passing all of them. Obviously this is one data point, but I’m hearing similar things from my friends in security and ops.
Agreed. Shouldn't continuous pen testing be a thing by now?
Yea most the people I know working in cyber security weren't worried about mythos in a serious way. All the people selling cyber products spread fear over it because profit.Everyone knows there are exploits laying in wait. Always has been always will be.
Most the cyber security people I know were upset once llms came out that could do basic os int and author their own emails, and mimic speech/video. That's how the majority of real losses occur. No one even talks about it. It's mind numbing hearing about the zebras and unicorns when there are 10 billion ants entering houses a year. Exotic stuff sells...
Now lots of sane people are genuinely and correctly worried that turning all engineers into 24/7 sloppers is and will continue to make the world far more exploitable. That's the real risk as far as I can tell
> because AI-generated code is kind of notoriously terrible at security.
and vaccines notoriously cause autism. But you might investigate why that belief is so widespread and you might not believe it that much anymore.
Can you point at something specifically that indicates this statement is true?
> The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.
This seems incongruent with what we're witnessing. Right now we're witnessing vulnerabilities that are 6 years old be exposed. That code wasn't generated by AI slop.
Security has always been about attacker/defender symmetry. So if the attacker has an LLM so does the defender. Reality is, the tax is getting higher as security tooling extracts more perceived value.
Let’s not forget the AI industry developed the technology and infrastructure for that security nightmare to exists. And no, none of that was inevitable. At every single step the AI leadership took decisions that makes it more likely that we face the current threats
> The rate at which we're churning out AI-generated code is going to make this worse
Not only this, a significant number of developers have stopped looking at code, trusting the bots. I've seen major security regressions, bot rolled auth implementations, and falsified benchmarks... just this last week, all in projects that get hyped.
Rule #2 - do not blindly trust Ai output
> The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.
So you're saying it's good at finding exploits but bad at generating code? I can tell you don't know what you're talking about because pretty much all the exploits that are being found were exploits that were enabled by human written code. So let me correct your statement for you:
There are exploits everywhere and we humans have always been bad at writing code without exploits.
"New normal"? Is that phrase back again? It was heavily pushed in 2020.
_Is_ AI generated code notoriously terrible at security?
My experience with AI-code is that it's extremely defensive and good at minimizing 'obvious' mistakes.
Are we comparing to old models generating code? Or mythos-class models?