This is still quite clearly something different than being unable to see the different colors, though.

Their mental model, sure. The way they convey it to others, sure.

But you can easily distinguish between two colors side by side that are even closer in appearance than wine and the sea, even if you only know one name for them. We can differentiate between colors before we even know the words for them when we're young, too.