Within the larger spec, you can whitelist a set of known devices, such as only allow Yubikey's, etc. Which would prevent the private key material from getting into your password manager.