A Canadian politician did this during a speech in front of the house/parliament or whatever. In the middle of the speech the LLM wrote "Here's a more natural, flowing version of that section that reads like a legislative speech" and he read it verbatim and then continued on, not even realizing what he just read.