It is a chicken and egg problem. As long as the majority of people who would maintain the social environment are avoiding the social environment, the healthy consensus/operating regime can never emerge.
It's not. Pass a law that continuing to be noisy or disruptive on a bus or train after a warning results in 10 years of prison time with no parole and consistently enforce it. The problem will solve itself without a chicken and egg problem. Problematic people can simply be removed from society to make for a good social environment. Adding more good people is not the only option and in fact only hides the problem instead of solving it.
In my experience the majority consensus is to maintain a quiet, generally polite environment on trains and buses.
But that's precisely the problem, it only takes a very tiny minority to change this. If one group, one person sometimes, in a carriage of 50 people decides to go against this, then that's that. It's not even particularly common, but it happens, it's random, and so it's just something that must be contended with.