I didn't mean to imply that it was. But when you reply to it, if you just say "no" then it's aware that you could've just not responded, and that normally you would never respond to it unless you were asking for something more.

It just doesn't make any sense to respond no in this situation, and so it confuses the LLM and so it looks for more context.