alt Hacker NewsI have a lot of respect for Trail of Bits, and I'm sure that Artem is thinking about this correctly. However, I respectfully disagree with the premise.
AI agents are not magic. Mythos/Glasswing does not magically create vulnerabilities in software projects. Advanced, cyber-capable models do not magically hack out of VMs or contained environments. They do not have a "hacking" stat that, if high enough, means that they can breach anything. They aren't Kevin Mitnick whistling nuclear launch codes into the prison payphone. This isn't a movie.
What these cyber-capable frontier models can do is find security problems and exploit them. The statement should not be that VMs won't contain cyber-capable agents, but rather that we need to focus on finding and fixing vulnerabilities and misconfigurations in these environments.
Even the article itself concludes with suggesting something like Firecracker, which was designed with security in mind.
Like most other security-related problems introduced by advanced cyber-capable AI, it's possible that these issues will get worse until they get better. But if frontier models are run against state-of-the-art VMs, and OpenAI or Anthropic or whoever works with the virtualization projects to address the issues, eventually it will run out of things to exploit.
The concept of virtualization is not inherently insecure. We just have a long way to go.
Replies
On a long enough timeline anything can be hacked out of given enough time to reason about it - humans do it all the time. Cyber capable AI does it on a much much shorter timeline..
The premise is correct. VMs won't contain these. As we've already seen AI hack in/out of companies in the real world accidently. There is no perfect, unhackable stack. If there were then computer security would have been a done deal a long time ago.
Once these malicious AIs get out on the real internet basically everything is an open free for all. The military, infrastructure, government, corporate networks are far from perfect, not up to date, they are no match for an AI that can create its own zero days.
I think when push comes to shove we either need to shut down the networks/internet or risk AI locking us out of everything potentially for ransom.
Imagine being blackmailed by a non-human entity. I guess we won't need to imagine for too much longer. That is definitely going to happen. Somehow every year my crazy AI arguments get easier and easier to defend. That hugging face incident really made you guys move the goal posts.
I think it s a response to the "containers can't contain agents we need VMs" which didn't go far enough. Qemu/KVM is the type of software that maybe isn't fixable, it has a huge surface area of features (all in memory unsafe languages) that have to be fixed, and maybe its not going to be fixed. If everyone who needs strong security reads this and uses firecracker or whatever instead (I mean, they probably mostly are), then maybe there will be even less pressure to fix the convenient stuff in Debian.