LLMs don't need to be the dangerous ASI thing themselves for them to still be dangerous. It suffices if they

- destabilize the world just by what they do: making previously very hard things (generating fake everything, manipulating people, hacking, bio-weapons research) easy enough fast enough that our civilization doesn't cope with it and/or

- accelerate AI research to the point where they make the dangerous ASI thing appear years earlier than would otherwise have happened