logoalt Hacker News

fc417fc802today at 1:04 AM0 repliesview on HN

> they lack an explicit architecture for the executive control of attention found in humans

Deceptive terminology strikes again! The "attention" mechanism in transformers appears (to my understanding at least) to have about as much to do with human attention as the "neurons" in a multi-layer perceptron have to do with biological neurons.

That said, the core premise of building in something that mimics executive function is an intriguing one (which I assume has been explored before but it's not something I'm familiar with).