I'd prefer something akin to the Biological Weapons Treaty which prohibits development, production and transfer. If you think it isn't possible you have to tell me why the bioweapons convention was successful and why it wouldn't be in the case of AI.
Because bioweapons suck, this is why. On the other hand AI sucks too, but it has at least some use
because bio-weapons labs take more to run than a workstation pc under your desk with a good graphics card. both in equipment material and training. Its hard to outlaw use of linear algebra and matrix multiplications.
Don't compare general intelligence to bioweapons. A bioweapon cannot defend against or reverse the effects of another bioweapon.
> bioweapons convention was successful
Was it successful? The jury is still out.