> All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.

That's not entirely true. Locomotion is well addressed by RL in sim. It's true that there is still a PD layer, and the RL policy produces setpoints for it.