> sounds to me as bad as "should of"

Interesting that you've used the spelling mistake which is perhaps why you hate it?

If you heard "should've" or "should have" then perhaps it wouldn't annoy you so much??? Also listen for would've / could've

But listening to https://www.grammar-monster.com/lessons/should_of_would_of_c... made me wonder if people do clearly pronounce the "of" in "should of"... Now I'm worried that I'm going to hear the mistake and be annoyed.

There is nothing more annoying than being told something annoying, and then learning to be annoyed by it.

Try not to internalise that dictum or it will recursively annoy you.

Next time you hear a really annoying vague repetitive/intermittent sound at work, mention it to a coworker if you wish to ruin their worklife.

(Minor edits). I often hear people who have learned English make a particular class of mistake (usually pronunciation) that is a result of being taught English by reading from books. Modern schooling for languages causes certain types of mistakes. There is a natural mimicking art/skill to learning a language by ear. Unfortunately the art isn't taught and is hardly even recognized: perhaps because it works best with intense one-on-one interaction and intent. Book learning was the default that our society used, and some well-educated people prefer books. When learning spoken English it is important to try and ignore spelling. Natural English speakers learn the spelling after learning the language and are in an environment where we have tricks to learn pronunciation of unfamiliar words. There is a strong classist/academic ridiculing of people that make the mistake of pronouncing a word as it is spelled (knowing how words are "properly" pronounced is an important distinction to many people - as is received wordplay).