logoalt Hacker News

zergrush • today at 6:47 AM • 1 reply • view on HN

Viewing LLMS as compression is flawed because it shouldn't be possible to solve new novel problems. Zipping every single file in existence and then asking Winzip to produce a novel working zip file clearly doesn't work. Clearly LLMs are learning from data it is fed and a lot of what we do before LLMs also involved "plagirism". Even the greatest writers and artists would copy each other to master their styles and extract the dimensions. I think framing LLMs as compression is not only silly it demonstrates a poor understanding of the inner workings of LLMs


Replies

Joel_Mckay • today at 7:01 AM

Except a statistically salient search context is in proximity to something similar, and manual content curators drop their thumb on the scales weights to boost desirable outcomes in the quantized model.

https://www.youtube.com/watch?v=LPZh9BOjkQs&list=PLZHQObOWTQ...

The encoding of the LLM is almost always lossy compression, but we agree it is not real intelligence.

As a sentient turnip, I likely wouldn't understand such things, but do recognize copyright bleed-through when it includes known obscure works. =3

https://en.wikipedia.org/wiki/Mechanical_Turk