alt Hacker NewsA bit of a tangential topic — what would you recommend to someone who wants to get into computer vision and 3D (NERFs, photogrammetry, 3DGS etc)?
For someone who has a middling amount of math knowledge, what would you recommend?
I went to uni 15y ago, but only had "proper" math in the first 2 semesters, let's says something akin to Calculus 1 and Linear Algebra 1. Hated math back then, plus I had horrible habits.
Replies
I've been working in the novel view synthesis domain since 2019 and I would recommend starting with "nerfstudio". The documentation does a good job of explaining all the components involved (from dataset to final learned representation), the code is readable and it's relatively simple to set up and run. I think it's a nice place to start from before diving deeper into the latest that is going on in the 3D space.
For learning 3dgs (and its derivatives) I would recommend grabbing the original 3d Gaussian Splatting paper + repository and going through it and using an LLM to ask many questions.
LLMs aren't that great at explaining concepts a lot of the time so when you get stuck there, google around and learn that subtopic. E.g. you will come across "Jacobian" that you may or may not have seen before, but you can search Youtube and find a great Khan Academy/3b1b collab explaining it.
Get the code running also, play around with parameters, try to implement the whole thing from scratch, making sure you intuitively understand each part with the above method.
Obviously time scales vary for everyone, that having been said: I'd guess if you have a decent technical background, are OK feeling uncomfortable with the maths for a while (it is all understandable after a bit of pain), and are willing to keep plugging for a few hours a day you will have a very decent understanding in 6mo, and probably be "cutting edge" in a year or so (obviously the learning never ends, it is an active area of research after all!)