You nailed it. Im a tech nerd by day, rather occasional casual gamer, and besides that - I dont want to interact with more technology than needs be. The thought of having additional nonsense overlaid on my view is actually nauseating.
It could make sense selectively during construction, mining, etc, but even then its a sometimes thing and it needs to be completely unobtrusive. Meta dont seem to be aiming towards or catering to this.