I mean, do we? For all we know all he has done is had access to unlimited compute on higher power models than available to the general public. His general complete unwillingness to divulge any information about any of the results he has obtained (AFAIK, we are still waiting on any details for the Jacobian conjecture counterexample or high rank elliptic curves or Hadamard matrices or S^6 complex structure or...) combined with the general ability of current models to make progress with minimal prompting means the burden of proof is on him.
If he truly did have non-trivial mathematical input to this it should be trivial for him to give talks on at least the overall strategy of the proof. Before LLMs this was a bare minimum expectation for basically all mathematicians and their results, and I see no reason why we should give up on it now, especially so since the previous primary artifacts (the papers themselves) are now on their way to being completely automatable.
I mean, your average guy absolutely has no chance of producing anything of value without having a lot of natural intelligence, no matter how good LLMs are.
A good article on what goes into writing such a proof by a non-mathemtician is here: https://overreacted.io/how-i-vibed-a-proof-of-conways-conjec...