Then they should be owned by governments outright. Provided that the community consent to it and are aware of the cost.

Government provides crucial services that increases land value, offsetting any losses in tax revenue through public utility. Perhaps the same thing can happen with historical buildings.

However, let us note that cities are for living in. It is not a museum.

Ultimately, only the public can determine the balance of concerns to be struck.