alt Hacker NewsNoting the theory papers on Kolmorogov complexity. For those not familiar, Ilya argues that the reason why neural networks generalize -- why they work at all -- is because they are effectively finding a simple description of their training data, converging down onto the limit of the Kolmorogov complexity. [1]
That's true of all statistical models, it's not some magic property of neural networks.