> can't empathize with treating mathematicians as some sort of AI refugees
They may be a vanguard.
> find it hard to believe that AI doesn't represent a net potential benefit to human understanding of mathematics. Yes it can be misused
Have you read Tao’s criticism of AI in mathematics? The concern isn’t mis-use, it’s ecosystem collapse.
A good analogy would be offshoring. If AI causes higher mathematical thought among humans to stop happening for a generation, we could wind up in a state where the field stops progressing because AI is unable to generate new interesting questions or cross-link findings into practical applications.
More directly, the risk is that happens in America but not in a more-thoughtful country.
This reprioritization of math would release a tremendous amount of human capital to do other more productive things other than posing yet another abstract puzzle which may or may not have an use in a few hundred years. It's not even clear if the human race can survive until then nor is it justified to spend that resource researching that far ahead instead of just when we need it.
But it's entirely up to humans whether or not higher mathematical thought ceases due to lazy over reliance on AI.
Just because AI is capable of churning out vomit inducing prose, doesn't mean we need to surrender to a future filled with it. Humans who actually care about what they're doing will use AI when and where it should be used. I trust the intelligence, curiosity and tenacity of mathematicians to learn to work with AI. I don't understand the sky is falling worst case scenario Tao and others seem to be taking as the most likely future.