Trust / spam score is the largest one I think, second to consumer ISPs blocking the necessary ports for receiving mail.

Even if your "self hosting" is renting a $5/month VPS, some spam lists (e.g. UCEPROTECT) proactively mark any IP ranges owned by consumer ISPs and VPS hosting as potential spam. I figured paying fastmail $30/yr was worth never having to worry about it.