A community _will_ form around any piece of software with enough users and developers, regardless of if it has a code of conduct or not. And communities will almost inevitably develop community standards and cliques. The code of conduct simply formalizes that. Someone who gets kicked out of a community with a code of conduct is equally likely to get kicked out of a community without a code of conduct

And besides that, someone can always make their own community around FOSS software. Fork and move on. But the new community will compete with the old, and it's possible that the fact that people stay with the old community means that they actually prefer that community