> There has to be a better way.

Probably Clevis and Tang, network disk decryption that can only decrypt if most of your servers are online. https://github.com/latchset/clevis https://github.com/latchset/tang

Or network decryption (SSH into initrd). https://github.com/gsauthof/dracut-sshd