logoalt Hacker News

someguyiguessyesterday at 8:24 PM2 repliesview on HN

Yes. It’s perfectly reasonable to expect the user to know the intricacies of the caching strategy of their llm. Totally reasonable expectation.


Replies

jghnyesterday at 11:27 PM

To some extent I'd say it is indeed reasonable. I had observed the effect for a while: if I walked away from a session I noticed that my next prompt would chew up a bunch of context. And that led me to do some digging, at which point I discovered their prompt caching.

So while I'd agree with your sarcasm that expecting users to be experts of the system is a big ask, where I disagree with you is that users should be curious and actively attempting to understand how it works around them. Given that the tooling changes often, this is an endless job.

show 1 reply
coldteayesterday at 9:41 PM

It's not like they have a poweful all-knowing oracle that can explain it to them at their dispos... oh, wait!

show 1 reply