alt Hacker NewsIt's like you're describing a compression program as "it takes a big file and returns a smaller file by exploiting regularities in the data." Like, you have accurately described what it does, but you have in no way answered the question of how it does that.
If you then explain the function of a CPU and how ELF binaries work (which is the equivalent of trying to answer the question by explaining how neural networks work), you then have still not answered the actually important question! Which is "what are the algorithms that LLMs have learnt that allow them to (apparently) converse and somewhat reason like humans?"
…except we know what every neuron in a neural network is doing. I ask again, what criteria do we need to meet for you to claim we know how LLMs work?
we know the equations, we know the numbers going through a network, we know the universal approximation theorem —- what’re you looking for exactly?
I’ve answered the “what have they learnt” bit; a function that predicts the next token based on data. what more do you need?