alt Hacker NewsDouglas Hofstadter: Analogy as the Core of Cognition [video]
Comments
He was extremely skeptical of the capabilities of AI for years but eventually came around to calling it truly capable after testing advanced LLMs.
He is still an AI skeptic in as many ways that he can reasonably be, but doesn't deny the raw ability. Actually he thinks it will eclipse humans and he is a doomer.
He sees it as being an extremely empty type of intelligence though.
But I hope that people who have an intuitive understanding of contemporary machine learning (not me) will sometimes watch videos like this and think about things at a higher level. LLMs have a LOT of assumptions built in.
The drive toward the formation of metaphors is the fundamental human drive, which one cannot for a single instant dispense with in thought, for one would thereby dispense with man himself.
Strong agree. It's the finding of symmetries and folds along non-obvious crease lines, but in semantic space of language <3
It's akin to the amino acid interactions in proteins that hold biological matter together, and determine it's shape and active form. Protein folding and narrative/storytelling have strange homology :)
(I work in this area via collective intelligence, and these ideas are very dear to me during the past decade. It's neat to see the intuitions seemingly becoming validated in language models)
See George Lakoff/Mark Johnson, Metaphors We Live By (1980):
https://george-lakoff.com/books/metaphors-we-live-by/
Took a course with Lakoff as an undergrad and it was compelling.
Every category and noun is an analogy uniting things unlike in at least some details. We don’t even see most analogies, because they are more common than air and just as necessary.
Scott Aaronson wrote a good post last week on how Hofstadter's theories of intelligence have held up: https://scottaaronson.blog/?p=10046
(The post primarily emphasizes self-referentiality rather than analogy, but I suspect similar things could be said about analogy.)
This seems like a worthwhile listen. As it stands I think that LLMs sort of suck at this. Either that or I don't understand what analogous thinking is or people who use LLMs to facilitate this kind of thinking are bad at it too.
But I think that LLMs are bad at something that has to do with taking seemingly disparate concepts and assimilating one into the other to convey a novel idea.
Like these articles...
<https://spectrum.ieee.org/jimi-hendrix-systems-engineer>
<https://zed.dev/blog/agentic-xanadu>
...are anything but convincing once you read past the gravitas that the LLM lends the prose.
I lover his book ”I am a strange loop”. It has shaped how I view myself.
This is very similar to George Lakoff's work. https://en.wikipedia.org/wiki/George_Lakoff
A good place to start for anyone that is interested is his book Metaphor's We Live By.
(2009)
Anyway, ML was already onto analogies with word2vec, which famously could answer questions like "man is to woman as king is to ____" mathematically. This stuff seems quaint now.
It's like Uber for thinking.
While I've not watched this video I don't think this is an uncommon idea. Or at least many people may see it in practice but never really follow thru on it.
People with more knowledge, especially practical working knowledge over many fields, tend to have much more freedom in finding solutions.
Now, an interesting question is how good at LLMs are at analogy, especially deeper transferable concepts?
It's been a long time since I read Hofstader's book "Fluid Concepts and Creative Analogies" that this is presumably based on, so I'll be interested to watch the video as a refresher, later. :) He also wrote a follow-on book, "Surfaces and Essences: Analogy as the Fuel and Fire of Thinking", published after this video was made, that I've never read.
From what I recall, to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process all the way from perception on up, and is the mechanism by which we form object categories in the first place.
As always Hofstader's ideas are interesting, but I can't say I agree with them. It seems that the key evolutionary benefit, and function, of a brain is prediction, which is the superpower that moves us from being stuck in the present to being able to "see" (predict) the future, and therefore from being merely reactive to being able to proactively plan and predict future outcomes (what will the sabre-tooth do, where is the water supply?) based on our experience.
Given the never-same-twice nature of sensory perception, before you can predict you need to be able to generalize/categorize, which I think Hofstader would regarded as analogy making (how is this thing I'm seeing similar to what I've previously seen?), although it seems the actual mechanism involved is embeddings or embedding-like representations where similar inputs have similar representations, and what might more simply be considered as associative recall provides the generalization from view/instance to identity/category.
So, is it really analogies all the way up, or are our perception and cognitive processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations, but also match the evolutionary needs that we would expect to exist?