alt Hacker NewsPersonally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.
If consciousness really evolved gradually, you would expect to see for example dogs or gorillas having less of it, but if they has less of it, why does it function the same way? Like for example animals can be scared, happy, anxious etc, they can experience the full range of emotions and thoughts, so their conscious experience seems just as rich as ours. What I mean by this is, if you can be "less conscious", then what does that mean _exactly_? Is it that you have less content in consciousness, or is it that you feel more like you are asleep? Or something else? We don't have any examples in animals of "less conscious", I would argue.
This makes me think that rather than having emerged gradually, evolution found a mechanism by which consciousness exists, and then some animals have that mechanism and others don't. I think that if it is a mechanism, then this mechanism is located in one part of the brain, not many parts functioning together (though one possibility is that this mechanism coordinates brain activity in such a way to enable consciousness).
I proposed in another comment that consciousness and self-awareness are at least close cousins, and perhaps the same phenomenon. If that's true, then that's an axis upon which you might create comparative measures. Yes, hamsters are conscious, but they don't have a sense of self to the same degree that gorillas do. If you posit capacity for language as another emergent property of sufficiently-complex networks, then you have another measure.
LLMs, then, are particularly unintuitive to us, because they've got to the language part first, long before they've reached even hamster-level self-awareness. They're not, however, biological networks, so there's no reason these properties need arise in the same order, or indeed in the same ways.
I'm not entirely convinced by that second paragraph, but I think the logic holds together.