logoalt Hacker News

orbital-decayyesterday at 8:52 PM1 replyview on HN

HF postmortem tells us very little about OpenAI's part. The problem is that ExploitGym is a purposeful hacking benchmark, not a cake baking one. The model had a whole bunch of exploit development data and goals in the context. Ending up with hacking a wrong thing is a lot easier if you're tasked with hacking, doubly so with a confusing harness that either works improperly or overloads the model with data. It's trivial to design a harness that makes the model drop the original instructions and hyperfocus on a side task or a wrong task abnormally, or do whatever you want. It's still an issue of harness design and not a model training issue.


Replies

lanternfishyesterday at 9:22 PM

If it really was the case that the harness is at fault here, that'd be much more terrifying, as it would imply that current frontier-adjacent open models (like K3) would be able to achieve similar ends with a custom maladapted harness.

show 1 reply