logoalt Hacker News

ACCount39yesterday at 9:45 PM1 replyview on HN

"The brain does it" is not in itself a reason to do the same in AI.

Sometimes, what the brain does is genuinely a good solution to a given task - one that's good regardless of whether your neural network is of artificial or biological variety. But sometimes, what the brain does is an evolutionary kludge, or a hack that works around one of the "being made of flesh" issues - of which there are a great many.

We have known examples of both - and a few known features that might go either way.

The brain does have some known useful features that we are yet to plunder - usually because we know they're there somewhere but not how they work. We don't know how the brain stabilizes online learning, for example. Or what low k-complexity priors and data augmentation processes does it use to enable its sample efficiency.

But a two-system split? Useless by itself. Splitting a network in two is easy - but if we don't know what that split does, what it buys us, what useful bias does it impart? We're just adding complexity. See: the investigation into HRMs, and how the "hierarchical" part proved to be a lot less meaningful than anticipated.


Replies

computerdorktoday at 3:15 AM

Eh, think you might not have read my comment well enough, because actually I partially agree with you in that two models may not be the best way to train an LLM. Yeah, I did say "In AI, maybe two models should be developed and trained in conjunction," meaning this is just some random idea based on the two system idea that might be useful.

Was just implying we don't know all the useful ideas (not just in tech but in medicine, genetics, evolutionary bioligy, mathematics...) that might be developed because we now know that the brain is two separate systems. It's all good though.