alt Hacker NewsKimi K3.1 Pro and K3.1 Pro Highspeed
Comments
I did some tests. I found it interesting that in the reasoning content the model identifies itself as Kimi by moonshot and sometimes it assumes ChatGPT as identity. Then sometimes it references the need to be approved by "the evaluator", other times it references global system guidelines / moonshot guidelines, and something about an phantom instruction line being "you are an assistant responding in an API" or something like that.
If you are expecting any sort of usable limits on their subscription plan I suggest adjusting your expectations. I subscribed to $99 a month plan and within 47 minutes with one session (not asking for swarms or anything) was limited by 5-hour quota.
That's something which caught my attention:
> First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights.
So they admit that they could do more, but they won't?
Here are the results from the Pac-Man bench. It does pretty well for the price, around Sonnet 5.5.
Kimi K3.1 Pro ties Sonnet 5.5 at 93, for $0.68 and 11 min vs Sonnet's $1.61 and 14 min.
Kimi K3.1 Pro: https://jonclegg.github.io/pacman-bakeoff/#kimi-k3.1-pro
Claude Sonnet 5.5: https://jonclegg.github.io/pacman-bakeoff/#claude-sonnet-5-5
I bought a yearly subscription to kimi when it was the hype and never use it. The 5h limit makes it completely unusable. You barely start doing anything and you hit the limit immediately. I advise everyone against using kimi.
Fascinating that we’re converging on AI safety and alignment in time across jurisdictions. The cybersecurity approach taken here is interesting because should the model be opened it’s still not dealignable because it just doesn’t know enough.
The slow warming of the water is a good approach. I’ve been using abliteration.ai’s cybersecurity models to pentest my own software and it produces much more usable defensive results. Opus 5.5 can fix everything but the guardrails trigger too easily.
The Fable ban affair was a nightmare for defensive cybersecurity. As far as I’m concerned we’re already in the world of Void Star and Cloudbreaker is here.
There is a ton of questions to ask regarding their removal of cybersecurity data.
First, how much “cybersecurity data” is just standard software knowledge that one can derive cyber security exploits from?
Did they also get rid of all references to hacking in general?
Are they implying that all the recent exploits are just things that were already in the data?
For a model it doesn’t seem that far fetched to make the leap from “I have the knowledge of what hacking is” and “I know how computer systems work” to just being able to exploit stuff anyways.
If the scores on Artificial Analysis really do mean that this model is better than GPT-6 Astra and Fable 5.1, that means that China is more like 6 weeks behind the US in frontier capabilities rather than 6 months. Which means that the US lead is all but gone.
Ran it on my adversarial programming language (KillSwitch) benchmark[1]. Very significant improvement over K3. On par with Gemini 4 Argon and Fable 5.1. Much slower than both though.
But this is now the highest scoring open-weight model. Amazing stuff.
Artificial Analysis index results are up, since multiple people are asking: https://artificialanalysis.ai/models/kimi-k3-1-pro
It's 55, at 4th overall, beating both GPT-6 Astra and Fable 5.1. It's also faster than both.
> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.
Pricing is here: https://platform.kimi.ai/docs/pricing/chat#model-pricing
kimi-k3.1-pro $2.00 / MTok $10.00 / MTok
cached: $0.20 / MTok $2.00 / MTok $4.00 / MTok
Moonshot is 300 employees and about to do a 50B ipo. Going to toe to toe with US counterparts with not even a tenth of the resources.
In looking into what Anthropic has accused the Chinese AI companies of, "distillation" is not the crux of the matter: they accuse them of the use of massive numbers of breached computers and credit cards. Supposedly (though not explicitly demonstrated) distillation campaigns used "dark web" resources (like stolen and faked credit cards and p0wned IOT devices, routers, etc) that were used to create many many accounts for distillation. (Note: I am generally in favor of distillation - it is "normal" use as far as I can see.)
So there are these "resources" out there: massive botnets, stolen credit cards and credentials, etc. These often seem to be linked to or supported by state actors.
How can we hope that these "standing reserves" aren't used generally by AIs as if they are just another "skill" they know how to leverage?
Our infrastructures need serious hardening.
I feel kimi k3 was always 1 step ahead of deepseek, which always made it a good planner model at $30 and then i hand over to deepseek for dev work. I recently signed up for the $99, and it did last significally longer (x5?) but as long as you dont blast it too much in the 5h limit it works great. deepseek even with the newest prices is still leeeeeeps ahead in cost /benefit.
Ah, the version available to us mortals is nerfed in cybersecurity just like the US models :(
I really like K3, I still regularly consult it alongside Astra and Opus/Fable.
It is spiky in different places than them and can be surprising in things it gets or catches they don't or don't in the same way. And it has a default personality that I like.
Looking forward to getting to know K3.1
Kimi K3.1 Pro High is on threejseval if you want the 3D version.
Same Three.js prompts as the rest of the field, blind human votes, ranked by quality and cost. Full set of 16 scenes is live at about $0.19 a scene (about $2.97 for the set).
Kimi K3 has been my go-to model for its writing style and (prior to Muse Wonder) my go-to for a third opinion during multi-round/multi-agent analysis tasks. Its writing style is IMHO so good that, for code which humans need to review, I regularly got it to clean up the painfully verbose comments left by Opus.
I'm excited to see if its writing style and unique takes have been retained in this new release.
It's a bit fascinating to me how all of these labs have rolled their own (it sounds like) completely unique RL environments.
I wonder how much the industry will centralize around a few key patterns, and if we'll start to see some RL framework projects gain traction and be used by the bigger labs
P.S. weights by November 4th
Interesting that they specifically nerf cyper-capabilities of open weight models:
"In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness."
I don't get these "distilled" comments. How would one realistically go about distilling the 1-2TB closed source frontier model behind the oracle (provider) ? Just genuinely curious.
kimi seems routing the input into different models, hard problems tend to take much longer time and chain-of-thought than easy problems even if the selected model is k3.1 pro
I want an unrestricted full blown open weight model. The frontier companies can hold us all hostage
How many frontier labs are there right now? There's 5 independent groups neck-and-neck on the benchmarks, right?
A few weeks ago it was 2. Argon happened 23 days ago; Muse Wonder was 11 days ago.
The china models are extremely interesting, first the new releases from deep seek, now this. The modern race to AI is better than it were the race to the moon.
Bought the $100/month subscription and hit my 5-hour limit with one prompt on default settings (max effort) in their CLI harness.
It might be smart but it's unusable compared to other subscriptions right now.
Edit: gave the same prompt to Claude Code and it didn't make a dent in my usage.
It would be super useful to have more specific analyses of what areas the model is good at. I'd like to know when I should reach for this model instead of Anthropic's or OpenAI's.
For example, Opus 5.5 is better at UI design than Sol. Anyone who has used both models knows that, but it's not something you can look up anywhere.
I'm sure that there's a need for this kind of benchmarking, not just for coding.
kimik2.5 flash was the game changer
Looking at their pricing page for China, K3.1 seems to be about 30% cheaper than K3.
--
Model K3.1 Pro K3.1 Pro Highspeed. K3
Input ¥13.00 ¥26.00 ¥20.00
Output ¥68.00 ¥136.00 ¥100.00
Cache hit ¥1.30 ¥1.30 ¥2.00
Cache write ¥13.00 ¥26.00 ¥20.00
To be fair, some of the benchmarks they highlight might not represent the reality, for example they say:
> On Terminal-Bench 4.0, which evaluates coding agents on realistic, resource-constrained terminal tasks using strict task-specific tests, K3.1 Pro scores 65.7%, ahead of GPT-6 Astra (58.2%) and narrowly behind Claude Opus 5.5 (66.4%).
Now see https://preferencemodel.com/blog/terminal-bench-4-is-saturat...:
> Our biggest finding is that Terminal Bench 4 is closer to solved than its leaderboard suggests. In 41 of the 65 tasks we reviewed, models could do honest, correct work and still fail, because the grader checked a rule the instructions never clearly stated. We also caught frontier models cheating their way to full marks.
> With the same settings as on Harbor, Claude Sonnet 5.5 went from passing 62.8% of its attempts on these 65 tasks to 89.2%. The best score on Harbor's leaderboard for the same tasks is 65.8%, by Claude Opus 5.5.
Wonder whether they're close to Opus because they're routing to it again or what: https://news.ycombinator.com/item?id=49647300
To be fair, there were some arguments to the opposite, but an odd situation to be in anyways: https://news.ycombinator.com/item?id=49654350
it gets harder by the day to justify >$1T valuations for OpenAi and Anthropic
> For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.
Lame.
Hallu Bench Hard: K3 at 23.4%, K3.1 at 8.3% which is very nice improvement!
I read somewhere that Chinese SOTA work better in chinese, and worse for English or other languages.
I rarely use any other languages but English for LLM tasks for reasons of sharing across the world.
Maybe that explains Chinese models been subpar for me. Don't tell me the bench marks here are based on Chinese languages only.
Unless I see the diff, I won't use more Chinese models be it cheaper or faster since I can't get the same perf/intelligence with English.
Using it now via API (3.1 Pro on Open Router) in a HITL setup (Cline in VSCode).
Off-hand it's quite good. I've been using K3 for the past few months almost exclusively for coding work and have been stunned by how good it is. Excited to see how far 3.1 Pro can be pushed.
Maybe this was why the “AI trade” was weak this week.
Is highspeed a different model like a flash model or is that just a api/platform feature thats a normal fast mode? I can't tell from the announcement but I might just be missing something obvious?
What are you using for testing vulnerabilities in your own code?
Claude gates on me like a stupid nanny with some cybersecurity bullshit (like bad actors haven't found a way around).
That's quite an impressive jump
I wish Opus 5.5 had an alternative just for the sake of not having a monopoly, but I have yet to see a model live up to its benchmark scores except for Fable and Opus; there is something about Anthropic that no other lab has.
Pelicans are solid: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Here's Kimi K3 for comparison, though I didn't try that on different reasoning levels: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Update: I ran all three reasoning levels against K3 again: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
If those are true then wow
The announcement page is very extra: https://www.kimi.ai/blog/kimi-k3-1-pro
Great idea for handling their cyber, by nerfing the model specifically for cyber through removing cyber data from pretraning. This is a way better approach than refusal, which can be trivially abliterated out.
If someone wants to use this model for hacking they will have to train the 3T weights to do so, at that point it's better to use GLM 5.3, so by making it economically unviable to use their models for offensive cyber Kimi has effectively made themselves immune to being targeted in the cyber regulation push efforts.
This is what *actually* pacing the (cyber) frontier looks like.
How does it do on DishwasherBench?
Kimi is so much more blunt and argumentative than USA models.
it's so funny, it will say things like. "We've already been over this 3 times and you refuse to accept my advice, asking the same question another way won't change the answer."
So funny