Tossing in my two cents here to agree with you. I worked remotely on and off from about 2014 onward until post-COVID RTO brought me into an office for 18 months before I became remote again. During that time (and across a bunch of companies) I went from desktop support to senior sysadmin to security on the cusp of senior security engineer.

In my experience the biggest factor in teams usually came down to the middle management layer. If their "style" was "watch over your shoulder, butts in seats" type of micromanagement then juniors didn't tend to progress unless they were self motivated to seek it out.