alt Hacker NewsBasically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks.
Pro is quite limited on the web UI I reckon. This approach can be highly effective for reasonably verifiable task, for example, write comprehensive unit tests pointing out a tricky bug, get multiple agents to swarm at it.
Replies
cubefox • today at 8:34 AM
> Basically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks.
It's unclear how they would do this when there is no signal that provides an objective ground truth.
It's been very successful at frontier math tasks - a bunch of the Erdos questions have been solved by it - more than any other model.
https://www.erdosproblems.com/