alt Hacker NewsAgreed. This isn't being oversold.
And actually, the fact that the brain is actually two "developmental" systems combined into one seems like big news to me. We might not know its other uses, but probably for cognitive science and even AI research this might be super useful. In AI, maybe two models should be developed and trained in conjunction that talk to each other, each with different functions and roles, because brains work better this way.
And from what I read, evolution did this type of combined evolution somewhat often. The parts of the ear evolved from different "Gill slits" from when we were fish, meaning the ear isn't evolved from one group of cells in the embryo, but from multiple groups of cells that evolved into one organ system, the ear. Yeah, the middle ear evolved from a different "gill slit" from the outer and inner ear.
Replies
> And actually, the fact that the brain is actually two "developmental" systems combined into one seems like big news to me.
It was known and discussed in evolutionary-developmental neuroscience, but not much outside that.
> maybe two models should be developed and trained in conjunction that talk to each other, each with different functions and roles, because brains work better this way.
I mean, is that not essentially what a Mixture of Experts model is?
"The brain does it" is not in itself a reason to do the same in AI.
Sometimes, what the brain does is genuinely a good solution to a given task - one that's good regardless of whether your neural network is of artificial or biological variety. But sometimes, what the brain does is an evolutionary kludge, or a hack that works around one of the "being made of flesh" issues - of which there are a great many.
We have known examples of both - and a few known features that might go either way.
The brain does have some known useful features that we are yet to plunder - usually because we know they're there somewhere but not how they work. We don't know how the brain stabilizes online learning, for example. Or what low k-complexity priors and data augmentation processes does it use to enable its sample efficiency.
But a two-system split? Useless by itself. Splitting a network in two is easy - but if we don't know what that split does, what it buys us, what useful bias does it impart? We're just adding complexity. See: the investigation into HRMs, and how the "hierarchical" part proved to be a lot less meaningful than anticipated.