alt Hacker NewsHow can you be cheaper if you query multiple models at once, compared to me just using a single model, which will always have the correct caching configured?
How can you be cheaper if you query multiple models at once, compared to me just using a single model, which will always have the correct caching configured?
If we cut off an unconfident/expensive model early in thinking, then we don't have to pay for additional thinking cost at all. For many tasks, this is quite a large saving.
That being said, whatever models we select to race at any point will ALL have their cache updated, so if they're chosen on the next race they'll be a lot cheaper.