It's not asymmetric cryptography itself. It's the fact that it takes enormous resources to manufacture modern SoCs, such that the economy only makes sense if you're churning them out by millions at least. It's also the fact that they can't be modified after they've been manufactured.

It's basically those people who can manufacture chips having technological supremacy over the rest of the humanity.