The major flaw in Panspermia is that it all had to start somewhere without Panspermia. If it did that there, why not here?
One theory is that the early plasma shortly after big bang had the right conditions to set the life building block chemistry.
Imagine the entire universe contains those buildings block.
It had to start somewhere which is favourable to preserve the necessary molecules. Early Earth was not such place.
One way to think about that is time required:
If earth is about 4 billion years old, but it takes say 400 trillion years for natural processes to produce this chemistry, then it happened out there not here.
This was a key reason why Hoyle preferred a steady state model of the universe — the part of the universe we inhabit needs to be very, very old for this stuff to work out, according to his thinking. A minority opinion, for sure, his rejection of the Big Bang model and timelines lost him a lot of respect among his peers. And his ideas could be wrong, I’m just pointing out that historically panspermia proponents have taken this position as to “why not here”.
We know 2 things that are apparently incoherent:
1 - Abiogenesis is incredibly rare. We don't know how much exactly, but it's a lot.
2 - Abiogenesis happened on Earth about as soon as it became possible. Where "as soon as" means within half a billion years, but it's still way quicker than its rarity implies.
A lot of people think panspermia is what made those two happen. Life had about a full billion years to appear in meteors before they could appear here.
There are some problems, e.g. that each meteor only stayed chemically active for less than that half-a-billion years Earth had. Or that all the meteors that fell on Earth had only a fraction of the material that was later available here. But IMO, the largest issue is that just doubling the time is absolutely unsatisfying.