You are correct that it can all be correct in one sense, and wrong in the product (“what the user wants”) sense. Of course you need to monitor the output, that’s a given. You need to question the entire system. That is the frontier for engineers.

“How is the agent lying to me?” Etc.