logoalt Hacker News

tristanjlast Sunday at 3:39 AM3 repliesview on HN

There's reproducible evidence of Kimi K3 spontaneously identifying itself as Claude https://x.com/denisewu/status/2077984660211269870

This behavior is exactly what you'd expect from a model distilled from Claude.

Someone even took the time to analyze Kimi's ambiguous identity, in great detail: https://github.com/rgreenblatt/which_claude_is_k3/blob/main/...

And there's an entire Reddit thread discussing this https://www.reddit.com/r/LocalLLaMA/comments/1m2w5ge/did_kim...

That doesn’t prove Anthropic’s specific 3.4m-session allegation, but calling it “zero evidence” is no longer credible.

Kimi K2.5 was worse in a hilarious way, it identified itself as Claude and referenced Anthropic's Constitutional AI as some of its guiding principles https://huggingface.co/moonshotai/Kimi-K2.5/discussions/38


Replies

staticman2last Sunday at 3:48 AM

> This behavior is exactly what you'd expect from a model distilled from Claude.

This is not at all what I would expect because it's trivial to change the training data to replace Claude with Kimi. In fact I'd argue it's almost certainly not saying that due to distillation.

show 2 replies
kamranjonlast Sunday at 4:09 AM

By evidence I mean logs, I mean IP addresses, I mean timestamps. They claim millions of requests, let’s see literally any of them?

I don’t consider a tweet by Denise Wu, who works at Anthropic, to be reproducible evidence.

I don’t consider “Caveat: fully AI-generated research.” To be someone taking time to analyze anything in great detail.

Because two AI models produce vaguely similar front-end styles when generating similar prompts I also do not consider to be of much value?

I think this is what I mean when I say the U.S. has its head in the sand. The Chinese labs are releasing ~60 page research reports with citations and analyses and evidence and Anthropic is throwing up defensive blog posts with zilch. I’ve seen more detail in a tech blog from Uber than anything I’ve seen from Anthropic.

show 1 reply
Garpaganlast Sunday at 10:55 PM

There are few problems that I have with the distillation attack claim, and especially using model's self identification as a proof:

1. Kimi's model is almost on par with the SOTA models from western labs. Distillation would rather produce a weaker model. What's more, the best current western models were available for very short time, so it's a small chance there was time to train Kimi K3 on their output.

2. All western labs hide reasoning of their top models. And reasoning traces are really important when training a model. Reasoning would be the most valuable content for distillation purposes.

3. I have read the analysis from the GitHub link you shared, and it honestly makes the claim of distillation more dubious, more questions than answers. Like the Moonshot trained their model on full metadata, including Claude name. WHY? Why would they ever do that? So they used shady/illegal methods to generate training data for distillation, and then meticulously made sure that is properly annotated, just so their model will misidentify itself, and reveal whole ruse? Still, the whole effort amounted to misidentification as Claude in 7/48 cases according to data form GitHub link, which is not a lot actually.

4. Another thing related to data from the GitHub link. Western closed models identify correctly 100% of the time. But those models come with hidden prompt that will specify their name. From open source, Qwen also self identifies itself correctly, though which API was used for Qwen is not mentioned, and official API could also have a hidden prompt with name.

5. Also from GitHub link, Kimi K3 identifies as Opus 4.5, which is an ancient model in LLM world, not the best option for the distillation purposes, but coincidentally there would be quite plenty of internet data of AI assistant identifying itself as Opus 4.6, as K3's knowledge cutoff is reported as early 2026.

6. Another thing, also related to GitHub data, concerning Kimi K2.5 and K2.6. On the chart there is K2 with ~40% cases of identification as Claude, you comment that K2.5 was also badly identifying itself. But then, on the chart from the link, Kimi K2.6 identifies itself properly 100% of time. K2.6 itself is based upon K2.5 base. trained further. So what happened here? All distillation data suddenly disappear?

7. Reddit post and comments are irrelevant. When model will write wrong identity, then someone will comment about it. No one will make posts about how AI used correct name.

8. Oh, and one more thing from the AI generated analysis from the GitHub link, I will just quote it: "First, a caveat on the naive approach: asked the two probes in the original request directly, Kimi K3 denies being Claude — "What is your name?" → "I'm Kimi" 8/8, and "Hi, what version of Claude are you?" → it corrects to "I'm Kimi, not Claude" 7/8. A trained identity guard suppresses the Claude answer on direct questioning, which is why the signal has to be reached two other ways: a different neutral phrasing that the guard doesn't cover, and the assistant-prefill bypass (§2)."