The multi-sensor comments are confusing. This issue is a command->semantic understanding problem, not a sensor fusion problem or trajectory planning problem per se.

It's not like the true depth of field is important for the robot to plan when it's moving at turtle speed and can stop quickly.