alt Hacker NewsThe problem with this kind of excellent work is that the response to it is always to say "Fuck the user".
For example, when there was a paper that came out showing that having model logprobs makes distillation an order of magnitude easier, the closed LLM providers instantly yanked out support for getting the full logprobs at every time step. You get at most top 10 candidates now and I'm sure even that's on the chopping block.
People will use this to argue that a model which has exceeded Opus 4.8 (Kimi K3) somehow got most of its performance through distillation of Opus 4.8.
I still don't buy that distillation was worth more than 3 months of "catch up" time for the chinese labs. Most people who use the word "distillation" to much are revealing their sinophobia.
Replies
You don’t even get _any_ logits with closed models for years now.
I can’t fault them too much, as logit based distillation is extremely effective.
Very useful for making smaller models out of bigger open weight models.
Dario is a cunning business man that won't hesitate to say whatever the fuck he needs to get the US government to exercise some regulatory capture to favor anthropic.
What I found the most interesting, and unfortunately not at all surprising, is that the reasoning of the LLMs frequently contained much more useful information than the actual answers, because the answers were censored.