We absolutely want politicians to listen to lobbyists in a way that is proportional to some parameter besides the number of people they represent. The should listen to who is right, not which lobby represents the most members. Self-interested groups lobby for bad legislation all the time. When the harm to society is diffuse, they don't always have a large lobby opposing them.