Claiming they can be reliable lawyers.[1]

Claiming they can give safe, regulated financial advice. [2]

Claiming you can put your whole operation on autopilot with minimal oversight and no negative consequences. [3]

[1] https://www.ftc.gov/news-events/news/press-releases/2024/09/...

[2] https://www.businessinsider.com/generative-ai-exaggeration-o...

[3] https://www.answerconnect.com/blog/business-tips/ai-customer...

Well, for starters, they definitively passed the Turing test a few years ago. The fact that many regard them as equivalent in skill to a junior dev is also, IMO, the stuff of science fiction.