I'm not sure if that's the best framing. Could you not extrapolate that to anything else?

"The only way that '(trees, shrimp, ants) are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway)."

Hmm let me try then. AI agents do not experience real time. This is understandable given their design, but it’s also something we have good empirical evidence for. They cannot, especially over the long horizon, track how much real time has passed as they complete their tasks. And they are not off by a few minutes but often bizarrely off, even mixing across past present and future.

Biology, on the other hand, is nothing but timed processes in a loop, the most obvious to us being the circadian cycle. As estimators of wall clock time, biology isn’t great, but when it comes to internal processes, and most certainly learning, memory, sensing, locomotion… biology is rhythmic in behavior, and the rhythms go all the way down to gene expression. More, these rhythms are, except during sleep, constantly entraining to signals from the environment that indicate time, most importantly light.

I think it’s a fairly unremarkable claim that agency and consciousness are temporal processes that depend on systems having an internal sense of time. How else can you anticipate? How can a system that can be literally turned off ever succeed in an environment where time never stops?

> The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).

I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this.

The claim is that the content of subjective experience depends upon how that subjective experience is instantiated. If you get a text "I'm fine" from two different people, those two different people are not necessarily having the same subjective experience even if they produce the same output.

In general, people that behave similarly in many contexts can have quite different experiences in those contexts.

There are many different ways that you can instantiate a forward pass in an LLM. Even if these are doing the exact same computation, we should not necessarily believe that they have the same experience, or that they even have a coherent single experience associated with the forward pass.

Subjective experience presents itself to me. To the best of my understanding, this subjective experience seems to be correlated with this biological human body (brain, heart, eyes, etc.).

To the best of my understanding, other biological humans have subjective experience, but this subjective experience seems that it can be much different than mine, in ways that are sometimes difficult to understand.

LLMs are instantiated in a radically different way from biological humans. Forward passes happen across many different physical machines, spread across time, batched and interleaved with many other computations and forward passes.

Given this, I think that it is very likely that LLMs have radically different sorts of experience to me and other biological humans.

I don't think that we have a good understanding of how subjective experience is instantiated in the physical world. I hope that we will gain a better understanding of this.

I think that chimpanzees have experience much more similar to us than LLMs do, even though the output of LLMs seems much more similar to humans in some contexts.

Essentially, until we gain a better understanding of how subjective experience is physically instantiated, we should bias towards believing that more physically similar architectures have more similar types of experience, and should think that very physically different architectures (such as LLMs) likely have very different subjective experiences.