Classic software was constrained by classic hardware. AI couldn't happen until there was enough compute to make (this entry level iteration of) it possible.

I'm fairly sure - ask me again in a few years - that most of the compute is unnecessary, because the current iteration is brute-forcing algorithms that could potentially be distilled into lightweight elegance.

And if that ever happens AI really will be unstoppable, because we'll get instant red queen evolution that leaves us far behind.

The question is more whether alignment now can steer that towards less cataclysmic outcomes later.