> make_more_likely is, of course, doing a heroic amount of work here.

Indeed it is, and so is even just the inference method. I think it's worth remembering that both involve running the input tokens through a gargantuan neural network with (often) billions of parameters that only gain semantic meaning during the training process itself.

> it is trained to predict next tokens as they occur in its training data.

What I found important to understand is that not even the pretrainig is a deterministic process that only depends on the training data - as you would expect if the model just captured statistical properties of the data.

Gradient descent starts by setting all the parameters of the neural network to some initial values - usually by setting them at random, according to some distribution. Then during training, it gradually nudges them towards values that somehow make them useful to calculate the desired outcome of the network.

This means that by taking the exact same trainset and the exact same model architecture, you can still get models with different internal structure. The result doesn't just depend on the training data, but also on the order of examples, learning rate, the parameter initialization, etc etc.