alt Hacker NewsI don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process.
I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen all the time. If the natural variation of agent executions cause agents to have ability to break out of sandbox 0.1% of the time, given how many agents OpenAI runs, this behavior happens eventually.
All sufficiently complex processes and software has bugs, but recently frontier models have become sufficiently advanced to exploit them.
Replies
My impression (HuggingFace incident) was they put some environment together in kubernetes like it was some B2B SaaS and tossed in an off-the-shelf package manager. And my thought there is, I know nothing about this package manager, but even if it's a pretty good package manager I would not expect such a product to be particularly hardened against being exploited to gain internet access. My other thought is if the package manager wasn't a convenient hole, they would probably have found something wrong with the kubernetes setup.
And then there was the Anthropic story where they just forgot to remove internet access.
So you’re saying that fucking up 0.1% of the time is acceptable, especially when it’s inevitable? I know it’s a stretch, but we’re in an industry where the concept of “five 9’s” is the gold standard.
I wonder how smoothly things would run on three 9’s. That definitely seems where we are going.
>Compared to industry standard
I don't hear about Anthropic or Google having such security lapses.
Why do you think it wasn't airgapped? Was internet access deemed necessary?
Did you read or watch any of the post mortems?
No, it is a shocking level of incompetence given the conveyed seriousness of the work by these labs.
So yes, models are getting better. Ask yourself: if you know that to be true, would you act the same way that the teams did in the public post mortems?
Am I missing something here? The "sandbox" was an inability to make POST requests, only GET requests. This has to be the weakest and most insubstantial sandbox of all time.