(EDIT: I, like some others, just noticed this is an older post from 2023. That changes nothing in particular in my response here, but I would be curious to know if the OP's views have shifted at all in light of subsequent developments).

I feel like this post completely misses the point, pretty much across the board. And it does so by repeating the same mistake that everybody keeps making - conflating mechanism and function.

One of the issues in during this research—one that has perplexed me—has been that many people are convinced that language models, or specifically chat-based language models, are intelligent.

That's because they are intelligent.

But there isn’t any mechanism inherent in large language models (LLMs) that would seem to enable this and,

The mechanism is irrelevant to the issue of whether they are intelligent or not. Airplanes fly, despite not flapping their wings. The sign on the marquee says artificial intelligence.

LLMs are not brains and do not meaningfully share any of the mechanisms that animals or people use to reason or think.

Again, irrelevant. Nobody claims that they are brains, and it doesn't matter what mechanism they use. The sign on the marquee says artificial intelligence.

LLMs are a mathematical model of language tokens. You give a LLM text, and it will give you a mathematically plausible response to that text.

That's a bit overly reductionistic. And to the earlier point and, if real, it would be completely unexplained.

I'd probably leave out the word "completely" there, but it is fair to say that not everything about the underlying mechanism is understood. But at the risk of repeating myself, that's orthogonal to the question of whether or not they are intelligent.

There is no reason to believe that it thinks or reasons—indeed, every AI researcher and vendor to date has repeatedly emphasised that these models don’t think.

You mean "There is no reason to believe that it thinks or reasons like a human". Again, this is irrelevant to the question of whether or not they are intelligent. The sign on the marquee says artificial intelligence.

I don't know why people keep obsessing over mechanism in this discussion. It something functions as an intelligence, it is intelligent as far as I'm concerned - at least when the framing is a discussion of artificial intelligence.