Well in the Navier-Stokes example the initial proof was completely unintelligible to humans, but other LLMs understood it and were able to answer questions about it.
If that's not the standard for understanding something, what is?
Being able to teach it to someone less knowledgeable.
That’s an instrumental definition, but it captures the idea that understanding has to have causal, conceptual linkages to a wider body of knowledge. Imagine a magical, infinite catalog of true vs. untrue statements. Every bit of knowledge is there, but we wouldn’t say it has understanding & it couldn’t teach us anything beyond its (impressive!) raw content.
Perhaps it could be considered understanding, but it’s an unsatisfying brute force sort of understanding.
I think of it like a spaghetti codebase with a giant densely connected graph of interactions. Today’s LLMs can easily “understand” and answer questions about it, but only by way of fitting all of the complexity into context at once.
The more useful form of understanding is being able to compress that giant graph down into a far smaller set of independent subgraphs. To factor it into simple, obvious, reusable building blocks, and use those abstractions to express the problem itself in a much simpler way.