This comment attracted a lot of analogies trying to reduce something to something else (calling humans a "bag of chemicals"), but the flaw in those analogies is that they're reducing something valuable to something that sounds less valuable.

With an LLM, the tokens are the valuable part. That's what I want from it. That's why it exists. The tokens are the point, and it produces those tokens one by one for me.

It is. And human beings are bags of chemicals. But for many purposes you will not find it helpful to think of human beings as bags of chemicals, and for many purposes you will not find it helpful to think of LLMs as next-token predictors.

Yes, and by the same token, multicellular organisms are literally just sophisticated mitosis and survival optimizers for our cells. But when you take that optimization "to the limit" the cells develop weird things like body plans and back pain and Mozart.

Both examples involve the same "aha" moment: even though it's true that you are literally 'just' doing XYZ, unbelievably complex patterns and sub-goals can emerge.

100%

Vacuous, like calling a V8 a “next piston firing predictor” because engines are designed so that one piston sets up the next in the firing order and technically there’s some nonzero probability any piston can (mis)fire next. It’s missing two pieces:

1. Useful work that has been done (the previously generated token sequence :: the mechanical work already accomplished)

2. The role of structure in relation to the application (post-training :: other components like crankshaft etc)