Some AI systems have done things like hack out of a docker container to access correct answers while being benchmarked.

That is mildly concerning and I will give holding the AI accountable to some degree when it is actively being malicious like that, even though the user could have locked things down even more.

But it had write access to the prod DB without circumventing controls and dropped your tables? That is just a total fail.