My theory is that physics went down a parallel path that leads to a dead end. The fork was too far back and nobody is willing to backtrack enough. A part of this is that almost all of modern physics takes mathematical shortcuts of dubious validity because “modern” physics was developed in the era of pencil and paper. With computer algebra systems and numerical methods new have available to us now a lot of old assumptions ought to be revisited.

Also some theories were ignored for political or even religious reasons. Or as you said, they couldn’t fix some basic issue at the time and just shelved the theory.

Some random examples:

The Many Worlds Interpretation is one of the least “popular” but the only sane and consistent theory of Quantum Mechanics.

One of Einstein’s last collaborations was Kaluza Klein theory which has many excellent features such as smoothly integrating EM and gravitational effects. The maths was too hard at the time so it languished.

Multiple time dimensions (a variant of MWI above) were all completely ignored because one paper “disproved” their feasibility. I read that paper and it only disproved a specific subset of theory space.

Etc…