There are two things: trusting the person's intentions and trusting the person's competence. Federation makes both problems worse, because you need to trust an unbounded number of organizations rather than a single organization. Even if you take it for granted that I trust all of those orgs intentions, there's no way they are as competent as the multimillion and multibillion dollar organizations running the big names.