Kerning does affect readability sometimes. Some fonts cause more issues with dyslexia than others.

I wouldn't say it is so much that the population doesn't care about fonts, more a case of them interacting with them in ways they don't notice consciously. The fonts on packaging affect the public perception of a product (and whether they buy them). Other fonts can hint at something being more modern or more traditional.