alt Hacker NewsDoes a mid-session provider switch result in loading the entire context into the new model, inflating session cost?
I don't think I understand the token/cost implications of this feature
Replies
gunalx • yesterday at 10:12 PM
Its nice if you used local, but needed å beefier modell, or more context Window. It will eat input tokens, but you do that all the time unless you have input caching.
Yes you pay a big burst right after switching. After that, everything is cached and it's smooth sailing.