Are they forbidding the model to implement backprop to figure out useful weight changes? If not, I’d expect this to be observed eventually.