Because it is somehow incapable of separating the conversation with its human operator from the code it is generating and commenting on. Incidentally, this is also why prompt-injection works.

No one wants to know about the three other approaches tried when reading the first sentence of a function's documentation. No one cares that the implementation was planned in six phases and "Phase 3" will implement this interface in a concrete type. But the LLM internalizes absolutely everything and you have no idea that it is producing slop because you included some "load-bearing" phrase that sent it on some unwanted tangential vector in its latent space. And you will not be able to debug the problem with closed models because you cannot see it referencing this phrase in its internal traces.

I don't understand why this isn't the highest priority for the big labs to fix. This is anti-productive.