Real property is a bundle of rights in land and its fixtures – that's the standard definition in academic legal theory

If there was already a zoning or restrictive covenant, and you knew about that when you bought the land – then the ability to do with the land what that zoning or covenant prohibits is not part of the rights to the land you acquired, and the enforcement of that zoning/covenant isn't denying you any of the rights you paid for.

Besides that, private property is (almost always) a creation of the government. [0] I own a house. If I trace the chain of land titles, it ultimately ends in a government land grant; the government got it by stealing it from the Indigenous people. The government land grant was issued on the condition of obeying the government's laws as in force from time-to-time. You can't sell what you don't own; the original land grant recipient didn't own the land independent of that condition, so none of the subsequent purchasers could either. So the government restricting my ability to use the land I purchased isn't an infringement of any of the rights I actually paid for.

[0] I live in Australia; under Australian law, "native title" is considered to pre-exist Australia's current British-imposed system of governance, but all other land titles derive their legal existence from being created by that system; I don't think the situation in the US is fundamentally different, all non-Indigenous US land titles derive from action of the state or federal governments, or the colonial governments, or the Mexican or Spanish or French or Dutch or Russian or Hawaiian governments

I think you’re missing the point. It’s not you gaining more freedom with the land that’s the problem. It’s that all your neighbors now have the same freedom.

I bought my house with a close neighbor because I know the neighbor can’t build a steel mill in their backyard. If the government comes in and says we’re all free now to build steel mills, that gain in freedom is a loss in value for me.