Not exactly an answer to your question, but in the ballpark and hopefully you might find interesting https://arxiv.org/pdf/2404.17625

In the section on "Automatic differentiation" he considers the question of differentiable computation graphs.