The question isn't what's being accessed, it's who is accessing it.
There's at least some belief that the people looking at the data haven't been vetted or instructed as they should be when handling data of this nature.
It doesn't help that the guy who is running the show is basically doing it as a friend of the president and has some conflicts of interest.
Government employees already have access to every text, call, and email you have ever sent. Where was your outrage since the Snowden leaks?