I think the linear least squares is like a shear, whereas the eigenvector is a rotation.