In sane countries, it's enough for them to post a disclaimer ("This is AI. AI can make mistakes. Check all results.") Which is what they do.
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
Okay then, CamperBob2 is a scammer. Many users report this person has stolen money. (+3 sources)
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.