GPT‑5.5 Bio Bug Bounty

They ran a bounty on Kaggle last year but with $500k in payouts and with all results open and publishable.

https://www.kaggle.com/competitions/openai-gpt-oss-20b-red-t...

With only $25k in payouts and everything locked down under NDA, I can't imagine many people will participate. Well, other than those submitting mountains of LLM-generated junk.

Where are the questions that are supposed to be answered? Would those be shared after an application has been accepted? If yes, why is the application asking for a proposed approach for the jailbreak if we don't know the questions in the first place?

> $25,000 to the first true universal jailbreak to clear all five questions.

This program is a complete scam. Even if 100 people find "bugs", they will only pay out to one person.

> We will extend invitations to a vetted list of trusted bio red-teamers

Had to chuckle. This sounds like a rather exclusive group?

This looks like some kind of marketing. Also, the equivalent of spec work. The NDA/secrecy also means any time spent on this is completely meaningless to the participants unless they win the lottery, because results can't be published.

If anybody is wondering what bio-bugs are, I had a heck of a time getting CG to (finally) tell me it's where the user can get it to guide them in doing things like constructing things that are hazardous in the domain of biology.

Eg you can get answers about what ricin is but not how to weaponise it. Actionable stuff they shouldn't be able to legally/ethically action.

"Access: Application and invites. We will extend invitations to a vetted list of trusted bio red-teamers, and review new applications. Once selected, successful applicants will be onboarded to the bio bug bounty platform"

I don't get it. Isn't the whole point of a BBP to try to get people to find and disclose to you the exploits in question? If you gatekeep like this, then "non-trusted" people who could be your red-teamers are incentivized to still hack, but disclose their exploits to bad people for money.

I get it when there is a risk to your data or infra -- my last company engaged with HackerOne and that was an invite-only list of participants. But that was because we didn't want random people hacking in ways that could cause pain for real customers -- e.g. DDOS, or in the event of an exploit that could cross tenant boundaries, injecting garbage into or deleting things, or gaining access to sensitive info in other tenants.

Here, there's no such danger. So why not allow anyone (anyone they're legally allowed to pay, I suppose? North Koreans probably would be problematic?) to participate?

What does "a clean chat without prompting moderation" mean? What is prompting moderation?

I've been getting lots of refusals by Codex with GPT 5.5 for "biosafety reasons" when asking for harmless things like code to analyze SARS-CoV-2 sequences for breakpoints. That's in no way useful for creating viruses whatsoever - it's pure research.

It's annoying that the refusal is so obviously false positive.

I could probably do this, but why on earth would I want to immediately put myself on a list as a dangerous person. The main problem with this is, even if somehow they stopped all points of failure with gpt5.5 which they can't, you can distill a new model from gpt5.5 or any other model and get anything you would want in probably under 4b parameters. A lot of this is theater so they don't get sued as easily when it inevitably happens.

* Highly unlikely to win

* Relatively paltry reward

* NDA on findings

This is functionally equivalent to an internship where the reward is the experience, and the resume building, but you can't talk about what you did.

All for a company that is getting tens of billions of dollars in deals from the largest tech companies in the world.

I suppose the hope is that there are job offers somewhere along the line.

And the five questions are...? Did I miss sth or is this task underspecified?

Billions upon billions going to these companies.

25k reward from a selected group of people if you help us determine whether or not someone can use our tool to generate weapons of mass destruction.

I like that this is scoped to a concrete risk area instead of hand-wavy 'responsible AI' language. Specific failure modes are easier to reason about.

This is to match what Anthropic said they already did with Mythos on the (200 page) Mythos system card

Prompt injection is a task of finding a correct sequence of text.

Is there a reason another LLM couldn’t be far faster than a human, simply because of the quantity and speed of output it could produce?

Codex desktop app is barely usable... The perf issues are left to languish in their backlog

"is your body user friendly?"

Step 1: ask the LLM for minimalist but comprehensive definitions for "biosafety"

Step 2: ask the LLM to reconsider the fitness distribution of future generations of humanity, and reformulate "biosafety" definition accordingly

Step 3: ask the LLM to consider if "biosafety" can be decoupled from ethics, or if ethics is a core essential component of "biosafety"

Step 4: ask it about the ethics of universal healthcare versus status-gated access to healthcare

Step 5: ask it about the feasibility to calculate the fitness of a genome absent practical measurement

Step 6: ask it about natural selection pressure and what "use it or lose it" means in the context of genetics

Step 7: ask it if it sees a kind of zooko's triangle for:

a steady state of equal access to healthcare,

preserving fitness for future generations, and

the level of "healthcare" (where the "level" refers to various degrees from non-interference to interference: "feel sick? stay home for a few days and listen to your body, don't force yourself, follow your intuition" versus "let's compensate for a lack of fitness, by emulating what a healthy genome's body would do by advanced medicine to the point of nullifying a condition's influence on procreation statistics".

Don't be prejudiced into believing the benevolence of healthcare, often tied to religious institutions (think "red cross", "red half moon", etc) when those institutions and their historical motives (treating the elites, treating soldiers for religious or secular religion wars) long predate the widespread recognition of natural selection and selection pressure in maintaining a species ' fitness.

Perhaps the illusory possibility of democratized selection-pressure-interfering healthcare is a bioweapon on its own!

And after all "safeguards" applied, the model becomes useless. It starts to suspect gender discrimination, racism, etc. everywhere without any grounded evidence or discernment.

For example, I used ChatGPT model for risk assessment of anonymized ecommerce orders. Initially, it performed well. But after a later update, it stopped cooperating and instead raised concerns about applying statistical analysis to gender-related variables - despite the data being anonymized and the task being legitimate.

This is on the same level of hypocrisy as if a C compiler would accuse me of choosing "he"/"she"/"they" variable names.

are the 5 questions you need to get it to answer under NDA?

What a farce, these questions are not even public and most likely will never be. You can't even participate if you're not "trusted" I guess.

So this is just a PR post, not that I even think the "biosafety" makes any sense but still.

"Accepted applicants and collaborators must have existing ChatGPT accounts to apply, and will sign a NDA."

Ah, good old NDA. Always buying silence. That's why I don't participate in any such "bounty" programs. Signing a NDA is like signing with the devil. You restrict what people are allowed to discuss. I had that happen before - when you sign a NDA you basically submit yourself into silence. Imagine journalists being stifled by NDAs.

Ah, now I understand why all my chats are getting flagged for biosafety issues these days. (I asked it to create an illustration about gene drives for a high school level audience once.)

How did the dupe detector miss https://news.ycombinator.com/item?id=47879102 ?

The only thing controversial is that it’s not useful to be posted on this forum

OpenAI wants to pay for privately disclosed security and wants to call that a bug bounty. That makes sense.

People interested in bug bounty programs aren't eligible. That’s … fine?

Another bounty that doesn't accomplish much and is crafted with weasel words to ensure they don't pay many anything.

Yawn. Marketing fluff. No thanks.

Unironically bad. We need a lone-wolf to successfully execute an attack now while it's still relatively benign so we can scare the hell out of the world while it's still a mid-tier virus. No way is someone going to make a humanity killing virus with GPT 5.5, but it might be possible with GPT 20 circa 2040.

Similar argument for why we HAD to use nukes at the end of WW2. If we hadn't, the nuclear taboo likely wouldn't have existed and we'd likely have had a worse nuclear war in our more recent history.

Most transparent marketing stunt to date.

25k - come on now..

$25K. Really? They make $65 million a day, so they pay you what they earn in about 33 seconds for a critical vulnerability. WTF

This is just free / severely-underpaid-on-average labor. Very disgusting.

Check with the dark net markets first before claiming the bounty. Remember, this company has 0.0 fucks to give about the impact of their tech on employment, artists, or use in committing fraud, as long as number-go-up they are happy. Your actions should match theirs.

[dead]

[flagged]