> Why consciousness is more likely a property of life than of computation
This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious.
And neither "life" nor "consciousness" are scientific terms with fixed definitions. If we encountered something that was clearly conscious in our judgment, we would likely expand the definition life in whatever way we needed to in order to encompass it.
Anil Seth has made the arguments in TFA many times and they have been rebutted many times. The arguments are complex and sophisticated and people are often most swayed by which one they read last. Teasing out the various logical errors and sleight of hand (I see multiple strawmen and other errors in TFA) really requires extensive reading of the literature in Philosophy of Mind, neuroscience, robotics, etc. and setting aside one's priors.
If you are interested in consciousness, and also interested in going beyond the usual philosophical approach, you might like our book Journey of the Mind https://www.goodreads.com/en/book/show/60500189-journey-of-t...
One thing that is implicit, which I now think we really should have expanded on in the book, is that consciousness is fundamentally an engineering problem. And that's also why it's mystified everyone who's brushed away the engineering as the easy plumbing and focused on the other seemingly magical aspects.
The fundamental question is phrased as "Why does it feel like something?" it being perception, which is essentialy computation. Now that AI does look like it does all of the understanding we do, many including people like Hinton think they have consciousness or are very near it. Well, they are capable of simulating consciousness but that is not the same as experiencing it.
To answer "Why does it feel like something?" we can focus on what's been glossed over: Who is feeling it? The "I" does not come from as a free lunch and it must be constructed moment to moment from some decentralized cohesion and consensus of a billions of neurons. And this consensus must be metabolically cheap, and we need to do it again and again for everything from feeling a pin prick to hearing a rustle and determining if that's a snake or a leaf to pondering thought.
This is the insane engineering, the energtically cheap chorus that creates the transient I and this cheap chorus is where both feeling and feeler emerge. Perception, understanding, computation can occur without this cheap chorus, but then there is no feeler, and so no feeling.
What this points to is that we will have conscious AI when we have decentralized autonomous swarms where a few million of them have to cohere, collude, collaborate, conspire. My worry here is that it will first be done for harmful purposes (autonomous military drone swarms) and not for anything good.
Consciousness in others isn’t something we prove. It is something we grant and we should be very careful about whom or what we grant it to.
how dare he tell us we cannot make a computer out of cheese
personally I think conscious AI will be unavoidable because there will be benefits to consciousness. evolution probably selected conscious entities for some fitness reason
Consciousness might not necessarily emerge when there is enough computation, but we also don't know that it doesn't emerge. We really know almost nothing about consciousness. It's possible that the question of why the presence of consciousness is correlated with certain kinds of material organization is in principle not accessible to scientific investigation. We don't even know that it is so correlated in general, we just know that for example people report being conscious when they are not asleep and not drugged in certain ways, and report having had no experience of consciousness during dreamless sleep or when drugged in certain ways. Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.