Then you're not acknowledging that a group for men that has a specific kind of power within a certain sphere that has excluded women is very different than a group for women in that same sphere. There is no uniform rule that can be applied, which is where you're getting tripped up. Uniform rules often favors control by those who have power.

I'd go with "you have never feared that you might be raped and killed while walking alone at night".