logoalt Hacker News

bluegattyyesterday at 11:46 PM2 repliesview on HN

Making an LLM from raw data is value-add.

Distillation is just value extract.

It's soft, and I'm not sure what the answer should be ... but I think that there is a difference.

I think we start by recognizing that ... and then try to figure it out from there.

'The Internet' may be a public good, maybe we make them pay a tax for that, but that's different than distillation.


Replies

nemomarxtoday at 12:04 AM

What makes the Internet raw data in a different way? wasn't it mostly worked on by people first?

show 1 reply
scotty79yesterday at 11:50 PM

> Making an LLM from raw data is value-add. > Distillation is just value extract.

There is a value-add in selecting the valuable parts out of the garbage. And let's face it. Largest models contain a lot of garbage.

show 1 reply