There is no way for you to know that you aren't slightly less sharp having offloaded the memorization of those phone numbers. Who is the judge? It's a nonsensical question from a scientific perspective because it's impossible to prove either way.

We could speculate that simple acts like memorizing phone numbers probably do make the average person slightly sharper, in a similar way to trivial brain games helping to stave off alzheimer's.