Have you tried... giving it the proof?

I tried to use Sol to:

- double check the proof (provided it with the prompt and proof artifacts)

- double check some of the claims made in this comment section (no math involved newer than 30 yo, no human contribution or review, no mathematician affirmations, proof assistants not being developed enough in this area to support machine checking a proof like this)

- check for any mathematician feedbacks

It stalled out (bad first impression much? lol). I then retried with 5.5, expressing the same request and my personal skepticism, and it returned to me with cautious optimism and no obvious issues found.

I think the fact that I provided it with the actual artifacts in question vs. you simply asking it to speculate about them is a really interesting UX difference. Like certainly, a coveted 50 year old math problem having a few pager proof is not going to be very likely. But then skim reading the proof by a frontier model is not going to yield any obvious issues either. Both responses are perfectly defensible given the context (I don't necessarily think these qualify as sycophancy), but we'd walk away with entirely different impressions if we didn't know about each other's requests.

And I'm not even trying to suggest you were wrong to not approach it in the ways I did. It's a perfectly reasonable and human way to prompt it the way you describe. It's just not the way I'd do it, but I have a hard time articulating why. And it's clear that the model was never going to help with this difference either.

Half a century of computing, and we're still trying to make the machine think on the users' behalf :)