alt Hacker NewsI am not sure you need something so complex. If objects on scene could be tagged with their location, and the camera movement tracked to identify when it brings the same objects back into scene, then the model can use its positional information to validate if the next rendered scene contains all the objects that should be expected.
Though, I am not sure well this retains temporal stability. I am just thinking about how in 3D rendering, objects are tracked as individual things and only culled, but all the data about them still exists.
Maybe the models need to reason as a 3D space first, then turn chosen objects into renderings?
The trick is to render a larger frame in lower fidelity and then only render a smaller output frame in full fidelity.
Or render a low fidelity 3d model for coherence and then raytrace that onto a 2d frame that is then rendered at higher fidelity.