I don’t know how you quantify a very low p(doom), but this is why mine is high enough to worry me.
A million AI monkeys at a million AI typewriters, banging away at random, could do amazing damage.
Which will happen first: amazing damage, or reproduce a Shakespeare play?
Especially when they cross into the physical realm as in not properly secured and air gapped control systems. SCADA is scary.