Nothing you said is really in contradiction with what I said, you're just looking at the why from an individual standpoint.

All of what you said can be true and also others can and do decide to allow or disallow a group to exist.