"Next-token predictor" is the wrong mental model for LLMs

Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step is doubly wrong:

1. In order to select even the first word of a meaningful sentence, it already has to have structure and meaning of what follows captured somewhere inside, mostly in it's weights/activations or indexed by it's state vector.

2. What you see when you use an LLM is not next-token prediction directly next to the prompt, but instead following a block of varying length of next-token prediction that happened to make progress on the problem in your prompt, and which just summarizes the results.

I'm not going to stop describing things accurately because someone who really wants to stall LLM discussion generated an article that continually undermines its own main point.

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well.

The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into it - but at least I would still call it a "next token predictor"

It's not using just training data, but what it's doing is predicting the next token to get to the solution. As far as my amateur knowledge goes, LLMs still roughly go token by token, deciding which one fits best given the context.

It's just not predicting based on it's training data, but predicting based on RLVR & more, trying to get to the optimal solution ( as much as the solutions CAN be optimal)

And I honestly think keeping this very much in mind is helpful in understanding and dealing with LLMs.

I think the author is arguing against the idea of a next-token predictor as something that simply uses the weights in the neural net which record the probabilities of tokens following other tokens as a valid definition of what an LLM is. Essentially a massive extreme markov chain.

With reinforcement learning and probably attention and other tricks that affect the weights based on things that aren't strictly in the training data, so the argument goes, you can end up with tokens following strings of tokens that would not be possible to be output with the training data and original weights alone. So describing it as solely a next-token predictor is incorrect based on this framing of it.

But that's just my take on this, I'm still trying to wrap my head around it all.

'next token predictor' is a limited mental model but it's actually much better than any others.

'pattern matching' is a better intuition that 'reasoning' even though I think nominally, using the term 'reasoning' is perfectly fine in that context. It's just a loaded word that brings too much to the table.

'It hasn't seen the pattern' is a better description of the limitations of AI because it really just doesn't generalize very well at all. The adaptations described in the article don't change that.

Those are mutations, not expansions of capability.

The errors LLMs typically made for me were looking for "schmutz" as a jiddish word, got "schmuck" on my screen. Thought why the stupid mistake? The next-token predictor model perfectly explains it.

Or starting with "yes". And this early locking in was a total lie, in the discussion that became "yes, might appear that way, but totally no since reasons". So it should have written "No", topmost, but could not self-edit that.

But nice that this gives it a more nuanced view, I might have to update my priors.

> Calling the second system a “next-move predictor” would be strange. It is not trying to predict what move appeared next in a dataset. It is trying to choose a move that wins.

i dont understand the distinction here. does working backwards from a set of win states instead of working forwards from the current state somehow change whether it’s a prediction or not?

It’s written in Claudish, or perhaps a human who has been reading too much Claudish recently. I am starting to become allergic to Claudish. Not there fully yet — but it’s at a tipping point genuinely worth exploring and worth being precise about.

I don’t think next token prediction is a particularly good description of pretraining either.

The intermediate representations at each position are being optimised not only to help predict the next token, but also to help predict all subsequent tokens within the training context.

You can see this directly in backpropagation: the gradient reaching a representation at position i sums contributions from prediction losses for subsequent tokens, not just from the loss for token i+1.

Text renderer, whatever. Doesn't matter how you think of them, they are very interesting technology that is being misused and misconstrued in the name of something that has nothing to do with technology: political economy.

The greatest trick the rich ever pulled was making us think that the economy is about technology, and not politics.

A better statement might be:

    Current agentic systems may be *built* from next-token predictors which are conceptually simple, but because of agentic frameworks, recursive invocation, tool use, and *heavy* investment in reinforcement learning in these contexts and for specific applications, they can no longer be thought of as "Merely" next token predictors.

LLMs are smarter than just "next token predictors".

But their creators are not. They simply told a black box to predict the next token. And kicked it until it did. Unfortunately, they were lucky and it worked. And here we are.

It's a weird article. Despite the title and some of the text, much of the article makes the point that LLMs are next-token predictors, where the predictions are based on both training data and various reinforcement learning techniques.

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities.

Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into complex multi-cellular organisms because individual cells are just "survival and next-mitosis optimizers"

> make_more_likely is, of course, doing a heroic amount of work here.

Indeed it is, and so is even just the inference method. I think it's worth remembering that both involve running the input tokens through a gargantuan neural network with (often) billions of parameters that only gain semantic meaning during the training process itself.

> it is trained to predict next tokens as they occur in its training data.

What I found important to understand is that not even the pretrainig is a deterministic process that only depends on the training data - as you would expect if the model just captured statistical properties of the data.

Gradient descent starts by setting all the parameters of the neural network to some initial values - usually by setting them at random, according to some distribution. Then during training, it gradually nudges them towards values that somehow make them useful to calculate the desired outcome of the network.

This means that by taking the exact same trainset and the exact same model architecture, you can still get models with different internal structure. The result doesn't just depend on the training data, but also on the order of examples, learning rate, the parameter initialization, etc etc.

The biggest problem is the word "predictor". Once you get into post training with RLHF and RLVR, it simply isn't doing that. It is not predicting anything. It's producing tokens, but it isn't predicting them. The chess analogy in the post is a good one - it's closer to searching for a set of moves that give a result than predict. It's search for a set of ideas, represented as locations in very high dimensional space, that when put together in the right order lead to a result.

It's the fitness function: Make a model which is capable of predicting the next token. The next token of what? EVERYTHING.

So what does this lead to? To a generic intelligence which is capable of responding/answering everything.

If overfitted, the model just remembers every possibility in the world but this is not possible anyway so it will start to identify patterns and rules and will use them instead.

Basically 'compressing' every possibility to every question someone could ask -> compression leads to intelligence.

I understand how a computer can know that a chess move is more likely to lead to a win, and therefore “correct”, but I don’t understand how it can know that a token is correct. Can someone explain?

yep "next-embedding" predictor is more correct, and not just at the end but through the layers, and folding back dimensions into that one next token is one small final step, and next-embedding could be named "next-meaning" as well, and we're getting there...

this sentence above would made a longer article if I bothered to so blog as is being blogged here

It's a next-token computer. It computes the probabilities for the next token.

Shrug. My intuition is predicts the new word based on a tensor vector space of patterns using arithmetic and similarity scores.

What’s not intuitive to me is that through pattern matching it’s able to express logic and reasoning.

Calling an LLM a "next-token predictor" is like calling a TomTom a "next-turn predictor." It confuses the serial format of its instructions with the computation producing them, while ignoring the map, the route, the destination, and the goal -- as well as the people, businesses, traffic, and points of interest that make the map a model of an inhabited, changing world.

Better title: Continue thinking of LLMs as Next-Token Predictors

Because no, post training doesn't change that.

slop slop and more slop

Sure, I get the gist of the article. I have never liked the reductionist argument that LLMs are nothing more than next-token predictors. By that rational, the human brain is really not that much different. When I am having a conversation with another person, I do not usually have every word I will respond with stored in my limited working memory. My output is often predicted based on the previous word I spoke.