To be honest, not any more than any search engine or other computer program can do harm.

When these AI companies claim that AI should be regulated, they usually mean it like a gun, because the owner of a gun can do harm to others. Let's put that aside for now, there is already plenty of debate on this idea.

Here, they mention milk and washing machines, and regulations are intended to protect the consumer from the manufacturer. And what how can an AI company harm their users? They can make their software full of bugs and vulnerabilities, leak private information, etc... nothing AI specific. The more AI-specific part is that they can provide a chatbot that tells you to kill yourself, well, yes, that's a risk, but the same can be said of the web in general. If Microsoft really wanted to have AI regulated in this sense, Windows 11 probably would have to be recalled the same day.