logoalt Hacker News

eldenring • today at 8:00 PM • 1 reply • view on HN

I disagree, I think distillation hides the true technical advantage since you can always get within X% of Claude by ex: training on newly created code on Github, but the underlying intelligence is harder to replicate.


Replies

markasoftware • today at 8:02 PM

based on the behavior of the chinese labs it seems like training on chain-of-thought is extremely important for distillation, and you can't get that by training on llm generated text.