I honestly don't think a language model is enough for self-awareness, regardless of the exact model of awareness.

A language model (or an image model or whatever) cannot even be sentient, and I think sentience is a prerequisite for awareness.

Even if we express a lot of our subjective experience with words, the language is just a symbolic representation of those experiences. The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.

You can't have an understanding of what hunger or physical pain feel like if you have no need for food or a sensory capacity for feeling pain. You can't understand what loneliness or pride at an achievement mean if you don't have a neural network wired to value social connection or status. We value connection because we're social animals that have needed each other for survival.

Even the more abstract of our subjective experiences are in some way rooted in our physical evolution.

I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.

AI awareness might actually be more believable if that awareness manifested itself in an entirely different way than in humans. But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience, I think we're seeing something that isn't actually there.