You are kind of ignoring the Dennett's idea of different stances: https://en.wikipedia.org/wiki/Intentional_stance

You're looking at humans from intentional stance, but at LLMs from design stance, which would be a form of categorical error.