alt Hacker NewsThis is essentially what Anthropic does with the advisor mode and what GitHub does with their multi-model pipeline using the HydraFusion model, which is just a large set of different models with different qualities used in a pipeline to produce an answer.
What I don't understand is how they all make it work with caching in general. Looking at how the advisor mode in claude work you seem to only be billed the output tokens and cache hit on the whole session when it calls out to the advisor, making it super cheap compared to running the more expensive model for the entire session.
Replies
lemonish97 • today at 7:58 PM
I've found hydrafusion to be great, although it sometimes does an unnecessary review pass at the end for simple tasks too.
Pretty sure on advisor it needs to send the entire context to another model. The cost is mitigated a bit because there would be few output tokens from the advisor model normally.
Orchestrator seems to automatically delegate things to subagents as required is my read of it. This should be fine for caching.