To expand on this, consider the historic importance of culture for improving survival odds and thus conformism as a natural consequence. So it makes sense that people in smaller groups would exhibit associated tendencies, and also that people who exhibit those tendencies would tend to gravitate towards smaller groups.

Somewhat related recent discussion: https://news.ycombinator.com/item?id=46989124