What I'm saying is that people in certain regions actually say "should of" instead of "should've".

And yes, there can be and often is a difference between saying "should've" and "should of".