Unless materials science advances to the point where a display like the Vision Pro weighs as much as a pair of glasses, I don’t think there’ll ever be mass adoption of wearable VR beyond anything more than a novelty, for exactly the reason you stated.

Wearing something heavy on the front of your face is simply not a pleasant experience.