My problem is that the wealthy get wealthier while the middle class gets subjugated to permanent renter class.

Go ahead, develop! And sell equity in the property. I've wondered if I would support my city requiring a certain percentage of dense urban lots be flats for sale vs. rental.