That's why in 2020 the FCC belatedly mandated SHAKEN/STIR to authenticate Caller ID in the US using public-key cryptography. Deployment is still work in progress, and it does not cover SMS/MMS, however.

A bigger problem is Russia or Saudi Arabia using the SS7 signalling network to track their dissidents in the US because those legacy telco protocols have basically no authentication whatsoever, and won't blink if a Saudi Telco sends Verizon a MAP message saying "what is the cell location of Jamal Khashoggi's phone?"