I think you can. If you have a neuron whose input weights are 100,-1,2, with threshold 0, you can know the output of the neuron if the first input is enabled, as the other 2 dont matter, so you can skip evaluating those.

I'm not enough of an expert to see if there's any actualy merit to this idea, and if you can skip evaluating huge parts of the network and keeping track of such evaluations, is actually worth it, but it intuitively makes sense to me that making an omelette has nothing to do with the Battle of Hastings, so when making a query about the former, the neurons encoding the latter might not affect the output.

Afaik, there's already research into finding which network weight encode which concepts.

MOE is a somewhat cruder version of this technique.