alt Hacker NewsGoodhart’s Law in reverse, what can’t be gamed gets rejected.
Replies
cbg0 • today at 5:04 PM
SWE-bench verified was created in collaboration with OpenAI. It's also an open dataset so prone to contamination, meaning it can be gamed.
Goodhart’s Law in reverse, what can’t be gamed gets rejected.
SWE-bench verified was created in collaboration with OpenAI. It's also an open dataset so prone to contamination, meaning it can be gamed.
You've almost buffer overrun Goodhart's Law into the https://en.wikipedia.org/wiki/McNamara_fallacy . :]