You are confusing hardware noise and whether the algorithm itself is verifiable as if parity bits were not a thing. We don’t trust computers because the medium itself is infallible, but because the logic is sound.

With statistical models - such as LLM’s - there is no logic as such, but statistical assumptions based on given data. The output can ge very good or very bad, but you are fool to trust it blindly. Therefore you need a deterministic way to verify, whether meat- or software-based.