alt Hacker NewsI've notice the same pattern even in GLM5.2. It has always been a thing, but it seems to be getting worse in recent models
It does feel like the kind of thing beam search would fix. The LLM starts the sentence with a claim like "Pay attention to the order of arguments". Around that time it "notices" that the order doesn't matter, but it's already committed to the sentence and has to complete it in the best way still possible
Maybe at some point someone figures out how to train models with a backspace token
Replies
Diffusion language models do this. I’d guess they are the future.
Yes, LLMs are prone to semantic consistency traps. I see a version of this in a system I have that transcribes a lot of noisy low quality audio (radio comms).
Sometimes it works out, in that an unrecognizable word or two is replaced with reasonable assumptions based on the semantics established by the words that came before the signal degraded.
But the bad audio might also result in words that don't align well to what came before, or represent alternate (mis)interpretations. Now this is part of the context and the next several tokens align to this new path regardless of what is said in the audio.
In pre-LLM transcription, you might get a nonsense word or two when the audio transiently degrades, but the specific meaning of the nonsense words doesn't influence the transcription of audio following the degradation.
Have noticed similar things using it for code review. "Issue #3: This variable is defined but never used. It is actually used later in the method though. So this is not a real issue to be concerned about", that sort of thing.