It's a good thing you weren't in charge of nuclear arsenals during the Cold War, sounds like your approach would have been unchecked proliferation.

Fortunately developing frontier models takes immense amounts of specific resources and knowledge. There are only a handful of companies capable of developing new cutting edge models. This is an area a few governments absolutely could coordinate on and regulate, if they were so inclined.

Obviously the current US administration is completely lacking both the will and competence to actually negotiate an agreement like that with China, and who knows if Xi would even be interested. But with different leadership we actually could be reducing our existential risks in this area much more than we are. Just like having a few thousands nukes across several countries isn't totally safe, but it's a heck of a lot safer than hundreds of thousands of nukes spread across a hundred countries.