alt Hacker NewsDoesn't it autocompact only if it is about to run out of context? On these newer models the context is 1M tokens so it's quite difficult to reach. I run compacts manually when I stop in a good place and tokens are over 300k
Doesn't it autocompact only if it is about to run out of context? On these newer models the context is 1M tokens so it's quite difficult to reach. I run compacts manually when I stop in a good place and tokens are over 300k
1m is too high, they used to autocompact a little too soon, maybe I tweaked defaults, it was kind of annoying, 200k tokens is too soon for me, I feel like the sweet spot is around 400k tokens.
Funnily enough Groks API charges less when you are below 400k tokens.
Half a million can get you reasonably far with enough of your codebase within the models context window.
I lowkey blame the 1m context window as the start of Anthropics worse woes earlier this year.