Seems possible.

The only real defense I see is to harden literally everything before that time comes. Unfortunately, security usage is locked away under fear of misuse, while companies like OpenAI seem incapable of containing their own hacking, which is exactly what makes this possible.