It's an interesting argument. Is there a highly-credible, authoritative source? Maybe someone like the EFF or ACLU? There are lots of ideas online about the law, of varying credibility, and I'd hesitate to risk a lawsuit over Internet advice.
While I concur with your hesitation, my first reaction on hearing about the fee was "Didn't they say you couldn't fly without a realid? Why am I able to fly without one then?" The idea that they may not be able to bar you without one jives with how this is playing out. Another commenter in this post also mentioned flying without id, which I also thought wasn't possible.
The author has been qualified as an expert witness in several venues.