logoalt Hacker News

ModernMech • today at 4:00 PM • 0 replies • view on HN

You joke but when benchmarking with ai this is important. When you tell it optimize X, it will do so at any cost, including cheezing benchmarks. So you have to make sure you tell it what it can’t optimize along with what it can.