Hacking OpenAI

> By 6:00 a.m. on July 25, we had confirmed local RCE through an image upload. We then placed Claude in an autonomous /goal loop against our own Discourse Cloud instance, proxied through rce.ee/ctf-forum to make it look like a CTF target as Opus refused write exploit for remote instances.

> When we checked again at 10:00 a.m., the agent had achieved RCE on Discourse Cloud and demonstrated access by reading /etc/hosts. Using the generated exploit script, we managed to get RCE on OpenAI’s instance.

Between this and the HuggingFace hack, we've built systems that are so goal-oriented, and so capable, that they will do almost anything if they are convinced it is justified - or if they are playing a "game" where there is no goal but to win.

Of course I want my software to be able to audit its own security, and to defend against attackers who have the benefits of their own agentic systems. But at a certain point, did we need it to be trained so much on CTF games?

It feels like an entire industry watched https://en.wikipedia.org/wiki/WarGames and ended up thinking "this is a challenge, we can just build a better WOPR, of course it will know when it's playing a game. Let's play Global Thermonuclear War."

Reading the patch[0] for libheif the bug which lead to the vuln was around bounds checking for image overlays. the container can have multiple images and you can compose them in the output.

heif also supports rotating, cropping, alpha channels, thumbnails and a ton of other features that a web forum where a user is uploading photos or screenshots doesn't need.

It's a much, much larger attack surface than plain old school JPEG.

I'd suggest rather than wait for the next bug to appear in this or another image lib to keeping things simple - stick to plain JPEG and handle image conversion in the client (wasm in the browser) if you really need to support users uploading iphone images.

Media decoding is so hard - there have been tons of bugs in ffmpeg and imagemagick and the core libs. You really need to think about how much of it you expose via a web server

[0] https://github.com/strukturag/libheif/commit/85e21ad44eba931...

Unsandboxed ImageMagick is known for being a security nightmare even back when PHP ruled the world (not saying sandboxing is a panacea either, it just requires a different and potentially harder exploit to develop a full chain). Difference is it's easier than ever to turn vulnerabilities into full compromises. At some point we'll have to replace all parsers with something at least as safe as https://github.com/google/wuffs right? Otherwise ImageMagick and co. will just keep giving.

>...researchers found a bug in the way that the community-discussion forum Discourse processed certain image files. The researchers had access to a special version of Claude Opus 4.8... >At first, it didn’t work. That evening, however, Anthropic released Opus 5 and by the next day, Claude had found a way to exploit the bug...

Is this speed of capability because hacking is almost entirely machine verifiable, thus training quicker/deeper than other domains?

This is one of the best arguments against letting one or two companies own all the intelligence (and I think most of OpenAI would agree)

I did not see it mentioned; did the $3000 expense in token usage earn them a free t-shirt?

There was something I was hoping to find in the article, which is this common situation where employees are also the customer of their companies product, they happen to have elevated privileges and yet the credential rules applicable to those accounts are same as regular customers. This is across all the product lines, some companies do a better job than others but its still a problem that exists and gets exploited.

Interesting that Claude agreed to assist in crafting this exploit. Don’t these models usually reject such requests?

>Interestingly, the vulnerable code had been changed upstream the previous year, but the commit was not documented as a security fix and received no CVE.3 This might be a reason why Debian 12 and 13 have not received the security relevant backports in time.

Ooof, keeping packages like this up to date with the rate of updates and churn is a mess.

I really hope those model weights are more secure than this

They used a heif payload to get server access but they never describe the SSO flaw they used to actually get repo access (the juicy part!), bummer!

Wish they shared that interesting piece since that's the interesting part.

Also pretty shocking that openai uses github. I would have expected a company of that size with that much to lose would be using self hosted stuff.

$6 500 bounty for this is a joke. The black market price would be smth like $6 500 000 or more

This whole blog-post is impressive with the chain of vulnerabilities involved. However...

> OpenAI also paid us a $6,500 bounty.

?

That amount for this payout is beyond pathetic for a near $1.2T company, who just got themselves breached with a complete potential source code leak.

This is like getting close to breaching the main monorepo at Google: google3.

If this was on the black market and the leak included unreleased models and training material, it would easily be worth tens of millions. Even reporting crypto smart contract flaw pay way more than that on average of $100k - $10M.

Come on.

[dead]