alt Hacker NewsSo, unlike what leaders of western labs labs would like you to believe (that Kimi is just the result of distillation attacks), they are introducing new and novel approaches.
Replies
Moreover, Distillation is a misused term here. Distillation means training a smaller student model using a larger teacher model to completely mimic its behavior. As in you take a base model and create its smaller “turbo” version. Like distillation in Chemistry it means it’s %100 purified version of its teacher.
No! It's just not fair!! What about our financial bubble!
"Kimi is largely a byproduct of distillation" and "Kimi is introducing new and novel approaches" are not mutually exclusive, and I'm not sure it's clear from the paper how much of the improvement comes from the new approaches. So I wouldn't take the new approaches to be much evidence about whether the distillation attacks occurred.
"attack"
Even if they are distilling, I don't particularly care. Anthropic and others have been distilling copyrighted material by to build these models, largely without permission.