It's not just public housing it's all housing. Even some public housing advocates argue against housing reforms that would make it possible to build public housing!

I've been supporting an advocacy group looking to build social housing and at least half the pushback is from anti-housing "tenant advocates" that work for non-profits funded by extremely foundations with boards that are all very wealthy with multiple homes, and don't see the need for more housing. The "tenant advocates" seem to view housing similarly and only support public housing to the extent that it doesn't actually get built.