alt Hacker NewsRent isn't high because of collusion. It's simple supply and demand.
There may be fewer people in manhattan, but that's mostly because fewer people live in each living unit. The same number of living units is being demanded by the market because of evolving living preferences.
If you allow sufficient living units to be built, it doesn't matter how much landlord try to collude, they won't be able to keep rent high. Someone will break when the vacancy rate reaches 15%.
Rent is high due to supply and demand, but collusion lowers supply. Ironically enough, "affordable housing" arrangements and rent-control, which is common in NYC, are examples of such collusion and end up raising rents over time compared to the alternative where the collusion isn't there.