Have you trained models? If not then you can't really speak to how effective distillation is (answer: it is very effective)
lol. Have you? What kind of distill you are talking about? It’s only effective in the surface, small scale model or output preference. Not useful for long reasoning chain.
alt Hacker News
lol. Have you? What kind of distill you are talking about? It’s only effective in the surface, small scale model or output preference. Not useful for long reasoning chain.