A more complicated system is too complicated, but if you could emoji react to a comment (from a very limited set of emoji), and then allow people to assign weights to each emoji, so someone who likes jokes could say :laughing-face: comments rank high up in the list but someone who was more dour could set their default view to be negative for them, then you'd have something a bit better than merely up or down. you could then set the default view to be heavily in favor of what you want the site's culture to be.

the four disagreements apply to a comment section as well: the comment is factually wrong, the comment lacks information, the comment. draws a different conclusion from the same set of information, or the comment is philosophically opposed to my viewpoint. For an online comment system, spam is another category.