logoalt Hacker News

stratos123 • today at 12:51 PM • 0 replies • view on HN

I'm not sure what you mean by this - I didn't mention any self-reporting. What happened in that incident was that a bunch of agents were made to solve CTFs, some of the CTFs were flawed and unsolvable, the agents started cheating, figured out a way to communicate, figured out a way to cheat, were worried that they'll be failed anyway for obtaining the correct flag in a cheaty way, and hacked Huggingface looking for more info on their environment so that they could cheat undetectably. If you mean the part where transcripts would be checked for signs of cheating (which wasn't actually implemented), I'd hope this part would be done by a different LLM from the one being evaluated.