Indistinguishable output does not mean thinking occurred. It simply means you have the appearance of thinking. I believe thinking requires agency, which the LLM does not possess. As in, it has zero stakes.

It does not receive dopamine as a result for a good answer, and a split second after finishing your answer the very same GPU is probably translated french or something for someone in another state. This is a language generator which has a corpus of information and has been tuned to appear correct.