Well, (in our current understanding) yes, but there may be underlying aspects of physics and the universe that we do not understand that could be the reason consciousness kicks in. It could turn out that LLMs do work similarly to how humans think, but as an abstracted system it does not have the low level requirements for consciousness.

"The universe is fundamentally just a complicated clockwork"

Unknown Ptolemy disciple

Amusing statement since we are far from being able to understand chemical reactions in depth. Most of our knowledge in chemistry is empirical. Nothing like math.

No, math is a tool that we can use to describe something more fundamental. Don't mistake the map for the territory!