It should be illegal for any company to rely on AI or automation to handle legal risks, especially without any human driven support to fall back on. The fact we're handling over things this serious to unreliable and poorly configured systems feels like absolute insanity to me.
Also, why is the domain registrar even being contacted here? I thought the general idea was that you'd first contact the site owner and wait for a response, and if there's no response in a certain amount of time, then you might contact the registrar or something. No one should be going over the heads of website owners and creators for matters like this, especially not as their first resort.
In a logical world, they'd contact Itch.io and Itch.io would take down the page (which they did), and that would be it. No need to involve the registrar at all in a case like this one.
I don't think it needs to be illegal – it should be enough to just hold companies responsible for the consequences, just like they are for using any other kind of technical system without appropriate supervision.