That phenomenon and others is what made it obvious that COT is not its "thinking". I think COT is a process by which the llm expands its processing boundary, in that it allows it to sample over a larger space of possibilities. So its kind of acts like a "trigger" of sorts that allows the model to explore in more ways then without COT. First time I saw this was when I witnessed the "wait" phenomenon. Simply inducing the model to say "wait" in its response improved accuracy of results. as now the model double checked its "work". funny enough it also sometimes lead it to produce a wrong answer where otherwise it should have stuck to its guns. But overall that little wait had a net positive affect. Thats when i knew COT was not same as human thinking as we dont care about trigger words or anything like that, our thinking requires zero language (though it does benefit from language) its a deeper process. Thats why i was interested in latent processing models and foray in that matter.