Because you only slow down the more common thing once, and the less common thing is slower in absolute terms.