The last five hundred years or so of scientific progress have been a series of discoveries that we are not so special:

- The sun does not go around the earth

- It is just one sun out of countless billions

- We were not designed, we were once apes

- We do not know ourselves as well as supposed - we can endlessly mislead ourselves about our own actions (Freud etc)

The first surprise with LLMs, so far, is that competent language use can arise out of a mass of unstructured training data and some fairly simple principles and operations - this was suspected for a long time but now we are really seeing it. A few years ago we were the only competent language users on the planet, now we share that title with LLMs. The second surprise is that competent language use can (at least) appear to overlap with reasoning.

As such I think its likely that consciousness will also turn out to be something not particularly special or magical, something along the lines of information combined and processed at a large enough scale with a handful of (lets call them) 'tricks'. Perhaps LLMs represent one of those 'tricks' and we're about 25% of the way there. Add a few more tricks over the next few decades and we might have something. (Of course, I have no inking of what the next steps will be)

Dennett didn't get everything right but his main message was that consciousness is probably something very different to what it naively appears to be. A simple example - our visual consciousness seems continuous but we know from eye saccades that it must not be.

A yardstick for me will be when we start discussing legislation about the use of AI in computer games, with a view to protecting AIs from inhumane suffering - at what point does increasingly detailed simulation of suffering become actual suffering? As a society we might have to make a call on that sooner than you might think.