logoalt Hacker News

jahooma11/07/20241 replyview on HN

I think managing context is the most important aspect of today's coding agents. We pick only files we think would be relevant to the user request and add those. We generally pull more files than Cursor, which I think is an advantage.

However, we also try to leverage prompt-caching as much as possible to lower costs and improve latency.

So we basically only add files over time. Once context gets too large, it will purge them all and start again.


Replies

handfuloflight11/07/2024

> However, we also try to leverage prompt-caching as much as possible to lower costs and improve latency.

Interesting! That does have 5 minute expiry on Claude, and your users can use Codebuff in an unoptimal way. Do you have plans in aligning your users towards using the tool in a way that makes the most use of prompt caches?

show 1 reply