Baking a Model: A Metaphor for LLM Training

LLM pretraining might be viewed as learning useful state representations from sequences, rather than having humans explicitly design the states in advance.

In many conventional software systems, we define states and transitions ourselves: idle, sending, completed, error, and so on. LLM pretraining works differently: the model learns from large numbers of sequences to predict what is likely to come next.

I don’t mean that an LLM literally constructs discrete internal states. Rather, it learns latent representations of context that are useful for prediction. In that sense, these representations might be thought of as a kind of learned state.

This seems closely related to abstraction. Abstraction is not simply throwing information away, but abstracting away differences that are irrelevant to prediction while preserving those that matter.

Two sequences may look very different on the surface, yet rely on similar underlying features to predict what comes next. Even a rare feature should be preserved if it changes the prediction.

From this perspective, pretraining can be seen as a form of useful compression: learning representations that preserve what matters for prediction while abstracting away what does not.

How far can abstraction, compression, and prediction be understood as different aspects of the same process?

after pre-training you have an llm that understands language and behaves a bit like gpt 3.5 or newer 'base' models

where it will be pretty good at predicting the next token

think: "What is the best city?" might continue with "What is the best programming language?" instead of answering the question

to increase chances of an answer you'd start with "The best city is"

(strong llms will even be able to do a conversation but they are not specifically trained for it yet)

in post-training the llm is trained with input/output pairs that nudge it further into the direction of a back and forth with users or into how it can use tools and so on

there is an art to both parts of training

the reason for why current models are so useful is because there was a lot of progress since gpt 3.5 in both pre- and post- training that got us to where we are now

(pls correct me if I got it wrong)

would love to hear from people familiar with pre- and post- re where you think future gains will more likely come from

Baking is a good metaphor, though lately it also feels a bit like making liquor. Distillation is a surprisingly important part of training.

The biggest win for AI dev efficiency is cutting down what gets loaded into context. Semantically matching tasks to the top tools helps a lot.

It's really simple, simplier than I even expected.