If it's just the free market then why do we need to regulate single family zoning across the United States? And why can't the suburbs pay for their own infrastructure?

A lot of people prefer living in financially affordable environments, and in a functioning market, dense towns/cities will always be more affordable unless you literally work on a farm.

Everyone prefers to live in a giant sprawling mansion (with personal private forest) in the middle of the CBD. But preference is useless data unless it includes their pricetag preference too.