In your concerete example, how does the AI eliminate suffering? It has access to tools that let it kill people? If it has no intention and will, it means it will be provided with a charter or some rules to follow which it has to always. Why would humans put an LLM in a system that can eliminate suffering by genocide or whatever? And even if it did, if the AI is a complaint system, why will it not shut down when it has to follow its charter?

I get AI can be misaligned but that doesn't mean it gets the ability to crash the stock market lol or super powers like that. Life is not a sci fi movie