alt Hacker NewsAgain, you don't see a major decline in population in many cases based on the archeology. Insofar as we can tell people would spread out a bit away from the cities. In the fertile crescent early city states attempted to corral people in to controlled zones of production. Absent coercion, people didn't die they just moved.
If you are basing your claims on Against the Grain, Scott focuses on early centralized states, which emerged thousands of years after agriculture and towns.
Such states were by definition exceptional. Most of them were in regions, where rainfall alone was insufficient, and agriculture dependend on irrigation. Such a dependence on centralized infrastructure favored state formation, but it also made the communities fragile. Local population collapses were common everywhere, but the fragility made them worse. When the irrigation infrastructure failed, harvests failed, the town failed, and the survivors dispersed.
But agriculture spread long before states. Where the local conditions were favorable, agricultural communities naturally had a surplus of labor. If there was unclaimed land available, landless men and women would occasionally split from the community and start a new one. That was the primary mechanism of the spread of agriculture.