It's important to consider context when talking about people's moral positions over time.
We don't judge the founding fathers' morals by the same metrics we judge our contemporaries. That much should be obvious.