I think this all depends on what time and place you're living in.
I'm definitely living in a very weird bubble in eastern Portland. It was a working-class, blue collar town of single-family homes. It was colonized in two separate waves, in the 90s and in the 2020s, by people who couldn't afford rents or homes in city areas of San Francisco or Seattle. But it never really took off economically as an international city, and it's still stagnant, and much more reminiscent of Seattle in the 1980s.
There's a lot to like about that pattern, and under Georgism it would make lots of sense for people to move here because the land value close to downtown isn't really that great. Or to move to Cincinnati, or Lexington, or Des Moines. If all people had to pay was fair rent on the land, new local communities and diverse downtown areas would be springing up all over America.
Not everywhere that people rush into and want to live because it was shown on the Real World needs to instantly resemble the same monocultural sort of housing that a big city like Los Angeles or Seattle has had to adopt.
Moreover, there's nothing inherently immoral about retaining the shape of a city when more people move into it; presumably, they are moving there because they like the city.
When I see working people being priced out of Seattle or LA, they're being priced out by financial groups and offshore investors, with the help of local boards that allow homes to be torn down and replaced with half-empty investment apartment blocks... destroying the very thing they're investing in.
We may be clinging on as homeowners and holding to a way of life that is incompatible with the kind of colonization other cities experienced, but: We take care of our parks. We know each other. I can't go 4, or 8 or 12 blocks that I don't see people I know. Up in the Amazon boxes, we meet the people who come for a year and go.
What equity is there for us in the plan to remove single family zoning? Not even as a parable:
10 miles west of Portland, Intel built a giant facility, and there is a commuter train stop out there which has Amazon boxes for miles around it. But no SFHs in their new construction, nor anything that would be attractive to anyone over the age of 29.
Why should I move out there and they move here, when in reality, that wouldn't even exist without the ancient core of SFHs that defines the character of the city they ostensibly want to live in?