How do you know that people use flags to further an agenda? I for one both downvote and flag pretty often, but it's largely because I don't like the tone of the discourse, not because of some overarching ploy.