Anything else could be lost. I can always get new SIM card for this number. I don't need to backup it and I can't accidentally delete it. That's the biggest reason for me to link phone number everywhere. I'd hate to lose access to my GitHub account.
It's also not very hard for scammers to get a SIM card for your number, unless you're using a carrier that specializes in not allowing SIM swapping attacks.
I don't see how I could simultaneously lose my three hardware keys (laptop, phone and Yubikey) and backup codes.