Indeed, I remember being in Oxford in the 90s and an older man approached me and spoke to me in English and I couldn’t understand a word he said. My ex-wife, who’s an ESL speaker who speaks fluently and without an accent has trouble with English accents in general. Similarly, in Spanish, I find it’s generally easier for me to understand Spanish speakers than Mexican speakers even though I learned Mexican Spanish in school and it’s been my primary exposure to the language. Likewise, I generally have an easier time understanding South American speakers than Caribbean speakers and both sound little like Mexican Spanish. (The Spanish I understand most easily is the heavily accented Spanish of non-native Spanish speakers.)

Accents have diverged a lot over time and as I recall, American English (particularly the mid-Atlantic seaboard variety) is closer to what Shakespeare and his cohort spoke than the standard BBC accent employed in most contemporary Shakespeare productions).