I don't think it makes sense to allow derivatives of a function f to have a larger domain than the domain of f.

>which is why they’re defined via limits

They're defined via studying f(x+h) - f(x) with a limit h -> 0. But, your example is taking two limits, h->0 and x->1, simultaneously. This is not the same thing.