It's a linguistics thing, it's about word usage more than about colour. You ask someone to get a book off the shelf, and you say "get the blue book" and the person is confused because they see a green book.

We are usually not specific in our day-to-day language, and this exposes/clarifies the issue.