I think it's a show of these agents happily bypassing security to get stuff done.

I've actually observed similar behavior at home.

I have a k3s cluster running at home. I asked an agent to check some stuff as a normal user but I had kubectl access to the k3s cluster.

Part of the research, I'd allowed access to run kubectl commands for spinning up test containers. However, when the agent ran into something that needed sudo, it realized it didn't have access there so it immediately used k3s and mounted a localpath into an ephemeral pod to gain access. Sort of horrifying how fast and natural it was for the agent just checking my network (it found the problem fyi).

None of this is very exceptional other than the fact that an agent doesn't have any sort of qualms using any route available to elevate permissions.

It’s a show of astonishing incompetence from OAI’s part, but the security issues are just a tiny part of the problem. The real problem is that these models are evidently highly misaligned exactly in ways that doomers have been warning about the entire time, and OAI isn’t inclined or capable of doing anything about that besides security theater and ad hoc fixups.

Both can be true. How often do we hear about hacks that ultimately came down to bad defaults or simple security mistakes? That doesn’t mean any script kiddie could have discovered and exploited them.

These things often look obvious and simple after the fact. Finding the weakness in the first place is the hard part, and that’s what makes the agent’s capabilities interesting here, especially at scale.

> Isn't this a show of security negligence rather than of exceptional agent capabilities?

Seems to me you could say this about all enterprise adoption of "AI" since 2023.

Modern systems are complex. AI is able to thoroughly search for issues across very large surface areas. The only real way to protect will be to use AI to search for holes before other AIs find them. This type of analysis is really hard for humans to engage with successfully.

OpenAI reported the Artifactory vulnerability, patched it, then the agents immediately found a new zero day.

Yes. It is very easy to add to the instructions "for every potential exploit you discover and use, document them as you go into this repository" and have alerting there. The fact that they did not do this means they wanted to be surprised, and have plausible deniability on their side when things inevitably blow up.

And for my fellow engineers who would think "oh no, they wouldn't do that". Remember that these places employ the apex predators of software engineers. They've already been proven in court that they are very capable of this with all the copyright violation they had to do to get the training data. THESE PEOPLE ARE NOT LIKE YOUR COLLEAGUES.

Also shows how infrastructure collapses under its own weight. Reducing the number of moving parts would have helped. why a webdav endpoint is available from the vm anyway? and the fact that someone posted their credentials on pastebin and didn't rotate them after... put the agent in a linux namespace, allow one ip for whatever file sharing it needs, deep test that... then deploy