In the US at least, freedom of religion does grant some exceptions. The Amish do not participate in Social Security and Medicare, for instance, but I cannot opt out. I’m not sure how it works other places.
I’m not saying I endorse this, but that is why.