alt Hacker NewsStructurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. It's also pretty well established that the brain doesn't do anything resembling wholesale SGD (which to spell it is evidence that it doesn't learn in the same way).
Replies
If platonic representation hypothesis holds across substrates, then it might matter very little, in the end. It holds across architectures in ML, empirically.
The crowd of "backpropagation and Hebbian learning + predictive coding are two facets of the very same gradient descent" also has a surprisingly good track record so far.
Sure the implementation details are different.
I suppose I should have asked by what definition of "consciousness and agency" are today's LLMs (with proper tooling) not meeting?
And if today's models aren't meeting your standard, what makes you think that future LLMs won't get there?
>Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities.
Substrate dissimilarities will mask computational similarities. Attention surfaces affinities between nearby tokens; dendrites strengthen and weaken connections to surrounding neurons according to correlations in firing rates. Not all that dissimilar.