There must be something unique about my GV number. It's even allowed on WhatsApp (knock on wood).

I registered it about 13 years ago. I didn't transfer it from a landline/cell phone, it was picked from a list of Google Voice numbers available in my area code. I've never had Fi.