alt Hacker NewsI don't know if this is why they did it, but I think it is generally a mistake to go beyond this context size anyway.
I don't think people realize how much dumber the models get at larger contexts and how much more the token cost is. I never let claude get about 300k and in general I don't compact, I just divide up the work in chunks where I can fit it into 300k and I try to keep really clean and slim docs and modular code bases.
It's still kind of annoying, sometimes the larger context is useful for one time tasks, but I think if you are regularly going beyond 300k you are losing a lot and probably have a poorly designed code base.
Replies
Exactly my experience. I would even draw the border at 100-150k. Long contexts would be nice, but they just don't work well, even if the models support them.
>> I don't think people realize how much dumber the models get at larger contexts and how much more the token cost is.
It does not match my experience that the model gets significantly dumber. It does get slower and more expensive, yes, but that's a sacrifice that needs to be made when working on anything complex.
My process involves having the main agent use subagents to explore what is needed for the given task. Then it writes a plan. Then it has the plan adversarially reviewed by more subagents and hardens it. After all is said and done, the 1M token window is 30-40% full. This flow would never work with 272k context, and in fact I've had to tone it down significantly for 5.6 Sol. Which, now that I think about it, probably explains why the results I get with it are inferior.
Yeah I also try to compact/restart at 250k. I noticed though that it tracks with how big the project is so I guess the people needing more are just working on bigger projects