alt Hacker News> Further than that, aren't "weights" just another sequence of tokens?
No, not at all. I recommend you learn some of the basic linear algebra about how these things work before engaging in wild and completely nonsensical speculation.
> Further than that, aren't "weights" just another sequence of tokens?
No, not at all. I recommend you learn some of the basic linear algebra about how these things work before engaging in wild and completely nonsensical speculation.
I understand linear algebra. It seems like he probably does, too.
Are you sure you have read and understood his point?
It seem quite possible that the flattened-model-weights ingested into a model have a much greater fidelity of representation in a similar model's KV cache than they do in a human's working memory, and that being able to loop and digest and chunk makes it unsurprising that models can excel at this type of task.
I think that's the point the person is making. Do you disagree? If so, why?