> That billionaire can probably just buy up some land and put their house there, though, since "affordability" is not part of the equation.

Not in California; we have an entire bureau, the Coastal Commission, that exists to prevent that very thing.