alt Hacker NewsI know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately.
Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine?
The lack of long context is the main reason that I still end up using Anthropic.
The worst is when you need it to hold for example a number of papers in its head, or large and complex materials that it needs full resolution on and your context window ends up being perennially at 16%. You have about five minutes of conversation and it compacts and then you have to wait for it to read that again, get to 16%... and repeat.
372 was not perfect, but it was so much better and a godsend. It turned that 12 to 20% into more like 40%.
Replies
That seems quite different from my design process. I write a plan.md that goes through multiple revisions. The plan is the memory. Restarting and reading the plan again to do another review is a good way to get a different perspective.
I agree. Compaction sucks, so I made tools that let the LLM selectively delete (and recall if needed) chunks of its context. You might want to try context bonsai if you're routinely hitting the auto-compaction wall.
Instead of compaction, I very aggressively do everything with teams of subagents. The main agent keeps the high level context, and all of the nitty gritty detail and code details not needed by the main agent never even make it into the context.
The solution isn’t to stuff everything it needs to know into the context, it is to have it write out what it needs to know for every part. It should write out plans, infrastructure decisions, etc, and then use those in future work as needed.
All depends on context coherence. E.g. in Claude Code with the "1M context" models I am reluctant to push past 30%. So if that Codex 272K is 100% coherent than it is less of a difference. Still, even then the "hard fail" boundary has moved. Longer running agentic processes will surely run into this more often.
I suffered this same problem with Anthropic, ironically. Mainly with reverse engineering things. That would usually end up requiring a ton of context it had to remember, and then compaction would kick in and it would just forget half of it (that half being really important too), and even switching to Claude Code and having it document everything wouldn't really fix the problem. It was honestly quite sad because I've usually gotten better results with it, but it falls a part the moment you try doing something like that that's really complicated and requires a ton of different attempts to get it right.
The best way around this for me is planning a project well ahead of time and using subagents to execute individual tasks while the primary agent acts as a PM.
Pay attention to what it forgets, and start telling it to proactively note down those things into nominated files while it works, e.g. indexing topics covered by each chapter/section/page of each paper/material. GPT-5.6 is really good at finding context again, if given the smallest hint where to start looking.
You can also ask it to read your past sessions, find places where it was wrong after compaction, and figure out a strategy to persist the most important summary information.
I have several Hermes threads that have each had >10 compactions with a 200k context limit, and with the right instructions on note-taking, they require at most a "double-check that against our past decision records" to be put back on track after a slip-up.
I think I've seen the opposite end of this as well: I get the most value from any LLM when I ask it one very targeted question with a followup. I don't fall outside of any sort of context window sizing when I am doing that. That isn't to say that I don't have a few long engineering discussions that are helpful, but most are very to the point. It helps that I have decades of experience at this point and effectively know what to ask to narrow the scope quickly.
What. Anthropic's models have million token context windows! I'm switching to OpenAI next month, I can't believe they're still stuck on ~300k... Better start adapting to the new reality I guess.
Depends on what you are using. I have been experimenting with various compaction techniques in piclaw (https://github.com/rcarmo/piclaw/blob/main/docs/pipelined-co...) and Codex has native "opaque" server-side compaction at a dedicated OpenAI endpoint.
sometimes compaction is a death knell and other times it's totally fine. it really just depends on whether the thing you were doing at the very end has high correlation with stuff from early on in the session, or if there was an obvious ~break s.t. you were just on some specific subtask. the situation has clearly gotten a lot better than even 6 months ago-- i think openai handles compaction better in my own experience. but it's exactly the kind of thing that yegge was trying to get at with gastown-- i used to run every session s.t. i could re-animate it every time the compaction boundary came... now, when the compaction boundary comes i am so pot committed that i just hold my breath.
Compaction is a feature like swapping out a car for a tricycle is a feature for greater economy.
Curious, what kinds of tasks do you do that require such a large context window? Anything specific?
What plan are you on?
With Anthropic I run out of quota very fast on a Pro plan with long contexts.
Is long context primarily for Max tier or for API usage? Or are there special ways of working with long context on a Pro plan?
A typical suggestion is to make the agent create or update a .md file once in a while, so the agent would remember the new important things that have shown up since the last update. The problem is, how would the agent know which things are really important. If it knows that well, then /compact should work well too...
If you’re using the pi.dev harness, I found this tool to be quite good as a compaction alternative: https://pi.dev/packages/pi-blackhole — it keeps a memory so every prior conversation piece is recallable, not lost.
If you want stuff to be referenced regularly, why are you doing it in the conversation instead of just putting the documents in the directory and referring to them in AGENTS.md or whatever?
You can mad customize it if you're willing to set up like, oh my pi or whatever.
I didn't quite get it tuned up enough to be a daily driver but I'm basically sure it can be hotrodded however you want. It comes out of the box with like four different vendor hostile compact strategies, including compacting into images at the smallest size Claude can read, which the "coding is basically solved" geniuses conveniently leak the resolution heuristic out of their website along with all the other side channels they print for the MSS.
OpenAI compaction is very different than Anthropic compaction, especially in 5.6
I use GLM5.2 and gpt-5.5 and 5.6 and never noticed any quality degradation near the limits of the context or due to compaction.
Why arent you using /goal?
> The lack of long context is the main reason that I still end up using Anthropic.
Anthropic's "long context" is mostly marketing bullshit. It semi-reliably holds about the same amount of info, and then starts suffering the same issues.
The fact there is no way to disable auto-compaction and no way to go back in the conversation history to before a compact makes codex a no-go for me on any codebase > 5kloc. Even worse it seems to fire randomly at 10-20% of context left. So really you have 80% of 272k as real usable context. Compaction kills my sessions, it hallucinates and is worse than starting fresh. I've had enough times screaming at my computer when it burns tokens on a large codebase, gets to 15%, auto-compacts, and hallucinates so bad it has to read the entire codebase agin, gets to 15%, auto-compacts....