> Human brains do this by deep double descent-style overparameterization, and adopting a scaling strategy of extremely high-learning-rate training of extremely overparameterized models on small diverse highly-filtered datasets.
That’s an extremely steep claim with no source other than vibes. Last time I checked my biology notes, model parameters are neurons, and they cost a ton of energy to maintain. Your hypothesis is really far removed from any actual neuroscience. Also, where are those filtered datasets coming from? Do you think genetics hands them to us? There’s about zero evidence for this claim as well.
I like new concepts for ML research but please do not make up theories of human cognition when you clearly have no idea about it.
alt Hacker News
> Human brains do this by deep double descent-style overparameterization, and adopting a scaling strategy of extremely high-learning-rate training of extremely overparameterized models on small diverse highly-filtered datasets.
That’s an extremely steep claim with no source other than vibes. Last time I checked my biology notes, model parameters are neurons, and they cost a ton of energy to maintain. Your hypothesis is really far removed from any actual neuroscience. Also, where are those filtered datasets coming from? Do you think genetics hands them to us? There’s about zero evidence for this claim as well. I like new concepts for ML research but please do not make up theories of human cognition when you clearly have no idea about it.