Aren't deep learning models themselves a case where we have hints of some deeper underlying logic to why some things are more effective than others, but we lack the mathematical tools to properly work it out for anything of practical size?

All we're able to do is apply flawed analogies, generic information theoretical models, trial and error, post-hoc rationalizations and benchmarks without really understanding why.