Terence Tao has some observations that seem to be directed at this,
https://mathstodon.xyz/@tao/117237320796901560
> "We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field."
>"While it may be technically infeasible to completely prohibit the use of automated tools to perform indiscriminate solution extraction, I believe that we can still designate many classes of problems as being desirous of a careful analysis that not only solves the problem, but identifies insights from the solution process, and learn more about the difficulty landscape for nearby problems, and for which raw solutions without such analysis would be of negligible or even negative value for these purposes."
Not sure I agree with this. AI generated proofs can still be analyzed and mined for useful insights. I suppose he's saying the process of banging our heads against the wall on a problem can itself yield useful insight? But what is stopping us from analyzing a proof after the fact. And if we can generate many different versions of a proof that should help us develop a much deeper understanding of the problem than we would have without being able to perceive the "proof landscape"...
No matter what happened this must be a wake up call for all of us. We’re basically sharing everything we have with these companies/AI systems. This is wildly different than a human wiretapping our private messages. Because it is systematic and automated in an astronomical scale. There is no real privacy in this new world. Law? I think “National Security” is a good enough excuse to screen anything constantly, including foreign researchers in case they are close to a breakthrough.
Isn't it safe to say that all famous unsolved math problems will get a "massive amount of AI-powered effort" pointed at them regardless?
I think he's suffering from a sort of static-universe fallacy. People aren't going to keep doing what they are doing, but secretly.
What is going to happen is a complete revaluation of things like "finding a counter example to a famous problem". Even if someone finds a solution to a problem like this with pencil and paper, nobody will believe it, and they will assume that there was an AI involved.
Further, sitting and doing math with a pencil and paper will no longer be a reasonable strategy to build a reputation or career, beyond the benefit a mathematician gains to their own intuition and skill. People who work hard to build intuition and also use AI effectively will dominate the field.
In a world where everyone is using AI, the open problems that remain will be the ones that are AI resistant. This is no different that how things work now, mathematicians wait until they are fairly confident someone won't rapidly solve their problem before they start talking about it. They will do the same thing in the future, except in the future AI will be part of the toolset they use decide if they are ready to share yet or not.
Edit: Ok I believe I was generally right here, but I just read the details of what OpenAI did. They didn't solve a longstanding problem, they got tipped off to an approach a mathematician was using and would likely result in the solution very soon and they finished it first. If this turns out to be true I think my take above is not correct, in the short term people will have to stop sharing updates because otherwise openai will dishonestly race to finish their work.
That’s not untrue. But it’s also a misstatement of mathematical history. Many leading mathematicians historically have been highly competitive — Gauss comes to mind. Woe betide the lesser intellect that sent Gauss some ideas. The Newton Leibniz controversy was very serious business at the time in the UK and the continent. It was considered at the least a sin to reveal that sqrt(2) was irrational to those outside Pythagoras circle.
Mathematics has always been highly competitive.