As a layperson myself - how can we tell that a distant celestial object is not made from antimatter? Afaik, it doesn't behave differently from matter, other than it annihilates on contact.
Also even if we could, that would only describe the practically and theoretically observable universe, which doesn't rule out other places looking very different (on its own)
Because distant celestial objects accelerate a lot of matter away from them as parts of different processes. For example if there were a sizeable chunk of space were antimatter than at the edge of this space there would be a constant glow of annihilation. We don't see that anywhere (well except things like quasars doing this at small scales.
So, yea, it would have to be outside the observable universe in which that means we can do no empirical science on it therefore it not being real to us.