alt Hacker News> Gemini
That's definitely part of your problem.
In my recent experience, error rates for astra/fable are at or below human level. Just like when directing humans, it pays to ask probing questions ('Are you sure about X?', 'Did you check for Y?', 'Please run Z just to double check.') if you really care about the result being correct.
And if you're building something of any importance, you need to have verification steps at checkpoints. It's honestly just engineering.
Weak models tasked with review can catch a decent amount of the mistakes that weak models make and help them be much better, especially if you have them verify against authoritative sources. Strong models make far fewer mistakes to begin with. And, for the mistakes they do make, a swarm of reviewers (same model or somewhat weaker, reviewed by the stronger model) can really help reduce the error rate further.