A quick search suggests to me that those same abolitionists were born into families that were slave owners, but rejected their upbringing to become abolitionists. It doesn't suggest that a large fraction of practicing abolitionists were contemporary slave owners.
And then this interacts with the above analogy in whatever manner.