> I get the impression that CPU designers in the 80s and early 90s massively underestimated just how beneficial even a small predictor can be.

It's got a lot to do with how cpu clock speeds were getting way faster, but ram wasn't. That's what makes deeper pipelines attractive, and if you give a cpu a deeper pipeline, it's gonna want a good branch predictor.