You see that https:// part of the URL in your browser? The 's' started out as pure mathematical research into elliptic curve cryptography based on algebraic geometry and number theory, a solution looking for a problem if there ever was one. When the problem eventually came along, the solution was obvious thanks to pure math.

I wouldn't be the least bit surprised if work done on the behavior of vectors in high-dimensional space by Hong and her colleagues someday becomes relevant to practical neural-net R&D. You never know what you might need in the future, but it will suck if engineers have to stop and figure this stuff out from first principles when they do need it.

von Neumann did a lot of theoretical work, e.g. https://www.academia.edu/download/38046446/Ohta_15_von_Neuma... where he was hanging out intellectually with McCullough and Pitts. He not only wouldn't be surprised at the ANN renaissance under way now, he'd ask us what took so long. He even cited M&P in his EDVAC report. Way ahead of his time, envisioning how hardware that wouldn't be practical for another 80 years might work.