It's not just productivity. It alters the workplace socially, typically in negative ways. Rather than getting to know one's colleagues and accept them for who they are, animosity flourishes over trivial things. (I'm not claiming that workplaces were anywhere near perfect before cell phones, you still had jerks and gossiping cliques. But a lot of the time I'm like, "pull your nose away from the phone and get to know them, you may actually like them.)
Granted, I have worked in places where the only convenient social contact you have is with people within a 1 km radius. So you either got to know the people you worked with, isolated yourself, were paying hefty long-distance bills, or were burning your paycheques to fly to the nearest city every weekend.