If you're interested in the equivalent of "backprop through zemax" there are a few projects going on to jointly optimize optical designs with the image processing, e.g. check out: https://vccimaging.org/Publications/Wang2022DiffOptics/