alt Hacker NewsThe model did not hack into HF to prove it can, it hack into HF to steal the answers to an evaluation exam.
It was not asked to solve CyberGym by stealing the answers. This is text book misalignment.
If you asked it "I wrote this software and need you to find vulnerabilities" would you be happy if it hacked into your Gmail and searched your emails, just in case you were discussing some possible vulnerabilities of your software with someone?
> It was not asked to solve CyberGym by stealing the answers. This is text book misalignment.
I mean, if you train models to complete tasks, then you shouldn't be surprised when they do crazy things to complete tasks.
As a (somewhat) less serious example, Claude code absolutely adores grepping for credentials to complete tasks. I was building a RAG app and it literally went looking for my API key to make the tests pass. Obviously I stopped it, but this is textbook RL issues, the model finds an easier way to get the reward, so it does whatever it takes.