Why though? There are only two reasonable reasons I can think of:

1. Scale. There's clearly a scale of consciousness (a jellyfish isn't very conscious but an ape is). Current LLMs are so small (on the scale of brains) that they must be not very conscious at all.

2. Structure. There might be some fundamental structure of computation that is required for consciousness that LLMs don't have yet. For example the ability to form memories (they don't really have that), or some structure of weights that gives consciousness.

My guess is it's definitely 1. and probably also 2.

But I don't think that means thinking LLMs are conscious is completely ridiculous, just a bit premature. I would not be surprised if they are conscious in 50 years.

The real question is how would we ever know that they are. The only reason we know animals are conscious is by direct experience, and we'll never have that for AI.

Anyway this blog post looks like typically worthless philosophy.