> Society doesn't allow dangerous companies to operate or exist

FTFY: Any self-respecting society with competent politicians.

The US has neither of that, and it shows everywhere you are looking.

> Go try to buy a tank and drive it into Manhattan. If we can prohibit that why can't we prohibit irresponsible AI development and deployment?

Because even if you had the tank, you can't make much money with it (unless you're a hitman, that is, but even for these, the payouts are measly). But if you are the surviving AI company in the usual VC playbook of "outcompete everyone else until society is completely and utterly hooked, then squeeze the customers by the balls"? The return on investment is virtually infinite. And that is what sustains the absurd valuations for all the AI companies.