Related online demo on using mediapipe for flying spaceships and camera/hand interaction to grab VR cubes (2nd link for the demo). There was a discussion on hackaday recently [2].

[0] https://tympanus.net/codrops/2024/10/24/creating-a-3d-hand-c...

[1] https://tympanus.net/Tutorials/webcam-3D-handcontrols/

[2] [https://hackaday.com/2024/10/25/diy-3d-hand-controller-using... DIY 3d hand controller