alt Hacker NewsIt's absolutely mind boggling to see claims of model distillation being theft, a class of attack, and all sorts of claims all the while Meta is in court for copyright violation, anthropic has had to settle a case with authors. With distillation "attacks" at least they paid API fees.
Replies
FergusArgyll • today at 3:14 PM
There are 2 things worth separating.
1) China distills and is therefore morally bad.
As you rightly point out, that's not a great argument.
2) China distills and is therefore possibly not that competent.
I think that makes sense. If they only catch up to the frontier through distillation then 1) Their model will never be as good as the model they are distilling from. 2) They will never reach the frontier - they need someone else to do it first.
➕ show 4 replies
Anthropic had to settle with authors because they literally pirated books! Their behavior regarding distillation is genuinely beyond parody.