Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA

If you haven’t been really running and testing these models yourself, they are all benchmaxxed. No matter how close they score to frontier on whatever metric, they always fall apart in real world tasks and their token efficiency is ridiculously bad. Fireworks has incredible incentive to make this claim in a headline, because Fireworks hosting K3 for you is pure profit for them, unlike when they host closed source models.

Very interesting. They test Kimi K3 and Fable on a set of approx 1000 tasks grouped into 5 areas (SWE, Legal, etc).

They put a router model in front that predicts whether Kimi or Fable is going to give a better cost for a correct result. (They believe that ultimately such a router model should be continuously trained on your own workloads so it makes the best decisions for you).

Their router chose Kimi the majority of the time (72% in one category, all the way to 96% in another category), leading to cost savings in every category (from 1.5x to 50x depending).

I will accept a 5% drop in benchmarks for a model that talks to me like a human.

Anthropic looks like Roman empire fast-farwarded, getting to the other side of the peak even before the IPO.

What's the data governance and privacy controls on using Kimi K3 if I subscribe to their coding plans? I want to migrate away from Anthropic

Genuine question: can these posts be paid to hype the open source models? If yes, what would be the purpose?

On my work tasks, FastAPI Python and Springboot Java on a modern SaaS product, the only open model that can do tasks well and efficiently is Qwen3.7-Max.

In all my experiments, both GLM-5.2 and Kimi are busy grepping around the codebase for ALMOST 70-80K tokens before writing anything and when they do it typically breaks the code… it feels to me that these models are good but only when you write out a super detailed spec of the task just like it was done a year ago… Qwen3.7 just… does it

Is there something specifically with Kimi that's better here? As far as I know Kimi pricing is about the same as Sonnet 5 -- what happens if you use that model and Fable instead? Or Grok 4.5 which is even cheaper?

Hmmm, a company that hosts open models is telling us how good open models are...

I love the Chinese models.

I use DeepSeek exclusively and now Kimi K3 offers a great planning assistant for more advanced coding tasks.

DeepSeek v4 Flash is extremely fast and is able to handle pretty much anything I've thrown at it (I use mostly Rust, PSQL, Angular and Terraform).

I self host Bifrost as my LLM gateway, though I wish LLM vendors would do monthly/daily automatic billing (like VPS providers do) rather than prepaid + auto-top up.

It's annoying maintaining a non-refundable minimum balance across vendors, I would rather be billed for my exact usage.

OpenRouter helps, but I don't really like it as a service and not a fan of the mark up.

Shifted to K3 and it is like a fresh air. While Fable and Sol very good at _generating_ code i even cannot force sol to just read all relevant source files. As result it reinvent existing things or assumes too much about internals of other, which lead to incorrect uses. Even with hard planing mode GPT burned 33% of week tokens for 3 hours producing no result and even cannot find root cause, but K3 fixed it in a minutes. Hilarious that while software not started with empty cache Sol handcrafted empty cache to let it start. Having full procedure right in the MEMORY.md. K3 found this, and downloaded cache correctly. But while using gen5 models i always feeling myself ignored. Any commands, steering anything - just ignoring. At first i added lots of hooks, no "?", expect in rust code, bash hook ban for find|grep|tail, with notice to use ltsp but then it started to ignore strategically. Also whole "thinking" thing is hidden from claude. regenerated thinking summary is incomplete and not useful. While being very verbose kimi k3 doing good job providing whole train of thoughts. So looks like claude degradation started from 4.6 comes to a logical end. Maybe i missing something and changing existing code beyond bug fixes is not a way to go. But there is no currently stable way to generate code on hier of specs and lean models i'd like to.

Nothing makes me happier than seeing AI becoming a commodity rather than the winner takes all bullshit Anthropic and OpenAI have been chasing with hundreds of billions of investor money

Anyone have routing harnesses like this describes with Claude Code? Or other good routing platform recommendations?

(yes, I know this article is about an oracle router)

That's incredible. Hope the chinese keep it up!

Kimi K3 showing competitive performance with Fable while both sitting at the SoTA level on fireworks.ai is a huge milestone. Really interesting to see how the landscape is shifting here.

Mythos/Fable was the state of the art back in March, if not earlier.

I enjoy all the fun of these new models as much as the next guy but I truly don’t see a circumstance in the near future where my $200 a month with the frontier labs doesn’t get me more than enough consumption of what I need. Local models, chinese models, etc are all very fun weekend projects to tinker with but until something changes (entirely possible!) with how much you get with one of the subscriptions I just don’t see why I would move. What am I missing? Is it simply that a subscription is no good for production use cases? I kinda feel the same way with choice of coding harness, openrouter, etc. why would I use anything other than frontier if I don’t have to pay any more pretty much no matter how much I use? pls tell me if I am holding this wrong haha

Your account <...> request reached organization TPD rate limit

SoTA means "State of the art". I wish it didn't take me 5 minutes to figure out what SoTA stands for.

As always, benchmarks rarely paint the whole picture. It also seems like this article is somewhat biased, eg when Fable and Kimi are close but Fable wins it’s “dead heat”, but when Kimi wins it’s “Kimi wins”. GPT 5.6 seems to be missing as well.

I am really eager to give Kimi K3 a try, but I’ll reserve my judgement until I’ve worked with it for at least a few days.

A third the cost, open source, and won't refuse every other request because of some vague possible connection to cybersecurity concerns.

If only you could run K3 locally that would be the magic bullet to make it a true magic bullet!

The article is about the best you could theoretically do with a perfect router. The takeaway is that trying to build a good router is worth doing. But it's unlikely to be a perfect router.

Interesting. So the latest in the technology now is this model routing thing. Cursor estimated Composer + Fable works much better than Fable alone. And here K3 + Fable is supposedly better. Interesting.

The Kimi K3 felt pretty good when I tried it out.

I'm skeptical. According to arena.ai, Fable 5 dominates almost every category: https://arena.ai/leaderboard Kimi K3 has an edge in WebDev but struggles to reach top 10 in many other categories.

Openrouter also features routing. Routing is indeed an option if you don’t need consistent behavior and allow switching models.

The irony is the Chinese are being very democratic with their models, while the USA tries to do central control.

Glad to see centralized control fail on the grandest scale. Maybe we can learn a thing or two.

Was this an out of sample test of the router or was it trained on these specific use cases/eval suites?

For model routers, do they have to retrain the routing model every time a new LLM is released?

we put a router model in front of two other models so the router can decide which model is better at deciding things. next we'll need a router for the router and eventually the entire internet is just routers routing routers to other routers

what are my options if i want to use a router like this ? who provides one ?

They forgot to compare and incorporate GPT-5.6-Sol.

I have several thoughts about this, which I'll just iterate:

1) US export bans have made it so that Chinese companies have to compete using less-than-state-of-the-art hardware. This has forced Chinese companies to build more cost efficient models. Whereas, US companies have moreso tried to be state of the art by spending more money than anyone else on state-of-the-art hardware.

2) Xi Jinping has called for more open AI models (not to be confused with the closed models of OpenAI), and I'm happy to see a powerful world leader advocating for open-weight AI models. Whereas, the US seems likely to just ban models.

3) My impression is that, if China surpasses the US in AI development, there will basically be nothing that the US does better than the rest of the world--except for military spending--we spend a lot, but we don't necessarily spend well (something something Iran). I mean, if the US is no longer a tech leader in the world, like... what are we a leader at? Manufacturing? Healthcare? LOL. Are we a leader in any industry or by any metric? I wonder if China is attempting to remove the last jewel in the USA's crown with these AI releases.

4) It must be refreshing for companies to have access to a new model that isn't going to get pulled because the government bans it 2 days after release. And it's open-weight so it wont go away--amazing--what a shift in the market.

5) If it becomes clear that open-weight models are the future of AI, will that pop a huge bubble in the US economy? Maybe. But, on the other hand, these companies aren't just training AIs, they are also building data centers which will remain valuable no matter what happens.

It is not SOTA. Give me a break. Sure, run it on Cerebras to get speed but that’s pretty much its advantage.

AI written drivel by an inference provider, hyping an open model that isn’t open, otherwise the inference provider would be hosting it, and the inference provider also does not have this router they trumpet available, in any form.

I’d be amenable to it but I’m already aware of them lying about token throughout by 50-100% on AA and on Twitter, even when you get a dedicated server.

Just too much lying piled up to extend courtesy here, even though you’re my favorite open model provider. Please, get your house in order, and if the house is so big that marketing drove this set, ask them to slow down a bit and market things that people can actually use / you actually provide.

Why SoTA (uppercase “T”) instead of SotA (lowercase “T”) ?

“State of [T]he Art” versus “State of [t]he Art”.

If not SotA then at least SOTA, which is more accurate.

[flagged]

[flagged]