I see no reason to seriously consider that current AI is in any way conscious. It's fun as thought experiment people like to play with. The technology is not some black box, it was designed by humans who understand it well enough to keep building and improving it.
We do not actually know what consciousness is, but we know we have it, in spite of our brains also just being a big blob of simple biological weighted functions that react to the hallucination our brains generate from external sensors we evolved. There is no guarantee other humans even perceive the same simulation we do!
The reality is we have no way to measure consciousness or how it emerges, or how to tell if other humans actually experience it the same way we do, so confident statements about consciousness being unable to emerge from simple weighted functions is baseless projection, wishing ourselves more special than an emergent property of the bag of weighted functions in our heads.
My argument isn't that current AI is definitively conscious or that it will be in the short term. It's that nobody has a cohesive theory of consciousness, and that we should therefore approach questions of what's conscious and what's not with humility.
For LLMs in particular, it's notable that they both 1) originate from a lineage that attempted to model biological brains mathematically and 2) are the only other known entities in the universe able to reproduce many types of human intellectual labor. That is surprising and striking. Does that mean that they definitively share other traits with humans like consciousness? No, but it does make them one of the most interesting candidates, particularly when we have a very limited understanding of how humans themselves are conscious.