So that's how you tell, salutes. Sometimes I worry that I inadvertently downvoted someone by mistake. "But you can just unvote if that happens" sure but how do you tell what was voted.

Anyhow, I think if this was my forum I would put the downvote selector at the end of the comment title and have the upvote selector at the beginning.