It isn’t just familiarity. Some people experience some fish sauces as having vividly foul flavor. This includes people who routinely eat anchovies, cured fish, etc.

It is clearly an issue of sensitivity.