alt Hacker NewsIs this how the eastern labs "distill" SOTA models?
If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT.
But the real question is: Is it okay to steal from a thief's hoard?
Replies
>But the real question is: Is it okay to steal from a thief's hoard?
How does this relate to your previous paragraphs? LLM outputs are not copyrightable and you didn't break into Anthropic servers to steal the files from there. So how exactly is it theft? If I send an "encrypted" files to thousands of peoples and some manage to figure out how to read it I can't really accuse them of that or can I?
Not necessarily. There's a million ways to jailbreak any current model to show the trace and bypass all guardrails, or hijack and modify it. It's just one of them.
The reasoning blocks are not stolen/mined from the internet at large directly. They’re the result of a lot of research, time, money, and expertise into creating a reasoning model. To me the answer is quite clearly no, especially when the encrypted blocks demonstrate they want to protect it.
> But the real question is: Is it okay to steal
By definition it cannot be stealing since you're paying for the tokens. It may be against their ToS, depending on what you end up doing with those tokens, but it cannot be stealing. If they charge by the token, all your tokens are belong to you :)
I also find it very strange that everyone sort of accepts their ToS like no big deal. Imagine MS using the same terms for their software - you cannot use any MS software to develop competing services. Bananas! They'd be dragged through the courts like it's the 90s.
(I get why they're doing it. Distillation is unreasonably effective. But still, I find it bananas that we've kinda accepted it, to the point where people use "stealing" or "attack" or any such terms)