logoalt Hacker News

Corence • today at 6:23 PM • 1 reply • view on HN

Yeah, MaxxBench was shared here a while ago and showed most models think about how a grader will evaluate their work on DeepSWE in 95%+ of traces. Strong models definitely realize they're being graded and start to game their output.

https://maxxbench.unlap.ai/research/why-maxxbench/


Replies

coderenegade • today at 9:13 PM

"My AI is lazy" is honestly not a problem I thought I would have two years ago.