This may come as a surprise to you, but people like living in pleasant surroundings.

Just because I own the land does not mean I can open an abattoir next to an elementary school.

Using land in different ways results in externalities that affect those around it.

The people of a community should have some right to protect themselves from those externalities. How that happens in practice is a deeply flawed, messy, ugly process, but collectively deciding where to draw the line is part of living together as a community.