alt Hacker News> fit a lot more work inside the model's context window before you're forced into a slow and lossy compaction step
This is something I never understood. Why the reasoning is not included until the context is full, then the reasoning stripped optionally to allow the conversation to continue. and only then when its truly full offer a compaction. Was it to optimize caching? Well I guess it doesn't matter now that you hinted that this choice was made because of prior limitations and may change very soon
> Why the reasoning is not included until the context is full, then the reasoning stripped optionally to allow the conversation to continue. and only then when its truly full offer a compaction.
Models are typically trained (at longer conversations/more turns) either with or without the reasoning still in the conversation. If you train a model with those, then using it without them, the model will perform a lot worse, same vice-versa if you train without but then end up using the model with them.
That's why you'll see some models have it and others don't, and trying to use them another way, will make them worse, they weren't trained like that.
So why aren't the models trained with both? I'm guessing that sort of permutation in the training would lead to double the amount of training time being needed, as you know effectively will have two variants of every session you train on, with and without the reasoning.