No, it's rentiers. The government takes about a quarter but the rentiers easily take 2 times as much in interest, monthly fees, and other costs that I have to pay in perpetuity. You just don't consider that because you think those people are necessary for living a good life. In reality their purpose is to extract as much money from you for as little work as possible.
Yes, the rent is too damn high. The way to address that is BUILDING MORE! Which is what YIMBY is all about.