If something behaves as if it has a goal, that's a functional goal. When I say this chess engine is "trying to take my queen," I'm basically correct, insofar as this is a useful way to understand my computerized opponent. If I give it access to my queen, it'll capture it, because that's what it's trying to do.

If LLMs are "just" statistical models, humans are "just" a bunch of neurons squirting chemicals back and forth. There's no pixie dust in our brains that makes us special.

LLMs are not conscious: They have no analogues for feelings or senses and no construct of selfhood. But if a statistical model had those things—if it did all the mundane, physical bookkeeping our brains do to produce "real" emotions and motivations—then there's no reason it couldn't be as conscious as we are.