logoalt Hacker News

evalstate • today at 5:13 PM • 0 replies • view on HN

Just use a harness that has been properly integrated and tested with each model, with published traces and comparisons (author of) https://fast-agent.ai/benchmarks/

GitHub and Copilot CLI have started to do the same btw (not sure if they publish the traces/reward hack results). But we should normalise this as consumers shouldn't be forced in to playing this game.