logoalt Hacker News

rixedtoday at 4:46 AM2 repliesview on HN

Only "generally"? I'm curious what API has moved away from this protocol that seems mode adapted to conversaions with humans than agentic loops.


Replies

_fluxtoday at 8:25 AM

To me it would certainly make sense if the protocol just said "append this text to context window id/sha256", in particular as the data is cached in tensor level in the provider side, so they need to first do that lookup anyway. So I would be surprised if they don't have that.

In addition, this protocol could make it more transparent to say "oh we cannot proceed as we dropped the this cache, are you sure you want to proceed and consume a whole lot of expensive uncached tokens?". Oh, maybe that's a reason not to do it..

bluegattytoday at 5:03 AM

So the standard API you pass it all along but I think there are some odd open ai apis that are different.