logoalt Hacker News

scuppernong • today at 4:13 AM • 0 replies • view on HN

https://frontierharness.org/ allegedly this is on the Pareto frontier, though I don't know how good of a benchmark this really is. it seems to focus on one-shot type tasks, whereas the real utility of one harness over another seems to make itself known in long running tasks.

also this is a very limited static snapshot with one model as backend, I wish there were more consistently refreshed and diversified harness benchmarks.