> At some point, because these models are trained on existing data, you cease significant technological advancement

What makes you think that they can't incrementally improve the state of the art... and by running at scale continuously can't do it faster than we as humans?

The potentially sad outcome is that we continue to do less and less, because they eventually will build better and better robots, so even activities like building the datacenters and fabs are things they can do w/o us.

And eventually most of what they do is to construct scenarios so that we can simulate living a normal life.