The downside is that you loose the place to store the metadata that a column is supposed to store a date.

Which is why I prefer not to use them.