alt Hacker NewsMy intuition is that in theory focus stacking should not be necessary as preprocessing step for 3dgs (or photogrammetry). Does anyone know if there is any recent developments in this regard?
Focus stacking generally is not perfect process and can lead to artifacts/errors and I'd imagine those can then compound when stacked images are used for 3dgs. Also the image focus actually provides some depth data in itself that could be useful?
Replies
If you don't focus stack and try to train on partially unfocused images, the optimizer will try to match the rendered view to be also partially unfocused.
You would have to mask out the blurry areas for each image. I guess one could just implement a feature where the optimizer only optimizes gaussians within the sharp distances relative to the camera.
This is an interesting question. There has been some attempt to model the camera better than just a pinhole camera, that could work. https://dof-gaussian.github.io/ https://github.com/leoShen917/DoF-Gaussian
My take.. at a macro scale, the dof is usually so small, that it's hard to get a reliably track. So you'd need some sort of way to tell that these stacked photos belong into a series, and then you sort of are doing focus stacking :-) I do think the alignment algorithm could be improved. Maybe the approaches I linked could be used to make a much more robust focus stacking algorithm, that also corrects for 3D geometry. That would be really cool!