Languages were made up by mankind at some point. They are not backed by the rest of physical reality. They can only be learned by induction, and there is no guarantee that people will get the same "version" of it.

To speak in your analogy, people walk around with different maps of the same territory and realizing this is the self-awareness mjburgess is talking about.