I think the op was maybe on a bit of a tear and misspoke, but the sentiment is correct. These days even saying hello can be used to make a decent voice clone with some reasonable (say 50%) chance that it is you (your phone number is linked to a ton of information). I would personally try to minimize my exposure to this risk even if it is somewhat paranoid.