Yes that's a big part. The proofs get hairier otherwise.

But I think there is more to it, the convergence to Gaussian also gets slower.

In practice, we deal with finite averaging, so speed of convergence matters. For some non-iid case, the convergence may be so slow that the distribution cannot be approximated well by a Gaussian.