The real reason is that backprop is basically matrix multiplication and multiplying from left to right is way cheaper from right to left. Since on the left side you will have a scalar loss term and you keep vector - matrix multiplication through the network instead of doing matrix by matrix multiplication from the right side.