alt Hacker NewsHuh? What are you talking about?
https://www.anthropic.com/engineering/infrastructure-noise
Is anthropic benchmark maxxing and cheating on terminal bench too? They don't follow the strict resource "limits" either
Huh? What are you talking about?
https://www.anthropic.com/engineering/infrastructure-noise
Is anthropic benchmark maxxing and cheating on terminal bench too? They don't follow the strict resource "limits" either
What that link describes is basically the motivation to go from terminal bench 2.0 to 2.1. The latter simply fixed the common issues/complaints. There is a long github discussion on tbench's about it