alt Hacker NewsIf they are literally sending the entire transcript and asking for a summary, wouldn't almost all of the tokens be cache hit, and therefore fairly cheap?
Replies
buzer • today at 4:43 AM
Didn't they change the cache TTL to 5 minutes earlier this year? So depending on exact timing it could potentially be invalidated between the updates.
➕ show 2 replies
I don't want to sound rude but even at cache hit you are burning almost input * 0.10 * context amount of tokens multiple times.
If it happens 10 times you have already accepted the same cost as the entire session fresh prefill.
People on HN are either too rich or wayyy too rich