logoalt Hacker News

minimaxir • yesterday at 6:08 PM • 8 replies • view on HN

Artificial Analysis index results are up, since multiple people are asking: https://artificialanalysis.ai/models/kimi-k3-1-pro

It's 55, at 4th overall, beating both GPT-6 Astra and Fable 5.1. It's also faster than both.

> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.


Replies

thepasch • yesterday at 6:24 PM

> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.

This is 100M fewer than Opus 5.5, 40M fewer than Fable 5.1, and about a third of Sonnet 5.5. In the top-6, it's 3rd, beaten by Muse Wonder at 140M and Astra at, impressively, just 60M.

➕ show 1 reply
mwigdahl • yesterday at 6:14 PM

Muse Wonder is even higher, and there is no way in hell it's better in practice than either Astra or Fable.

➕ show 1 reply
SkiFire13 • yesterday at 8:24 PM

From that link:

> reasonably priced when comparing to other models of similar price

???

➕ show 1 reply
dom96 • yesterday at 7:59 PM

> beating both GPT-6 Astra and Fable 5.1. It's also faster than both.

Very similar to my adversarial programming language (KillSwitch) benchmark[1], at least on pass rate, it is much slower though.

1 - https://bench.killswitch-lang.org

➕ show 1 reply
saberience • yesterday at 7:29 PM

I don't trust these Artificial Analysis indexes and I'm not sure why anyone considers them authoritative, the site just sprung up a few months ago and now everyone quotes it?

From my own use of the models I find their ranking to be totally inaccurate.

➕ show 4 replies
option • yesterday at 7:42 PM

while this is an awesome model, I have a bridge to sell to folks who think (based on AA leaderboard) that this is better than Opus 5.5

red_hare • yesterday at 6:22 PM

> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.

So their base price is 1/5th of Opus 5.5 per token, but if you want "highspeed" it's 2x that and then I guess it generates 2x the number of tokens as well?

That might suggest that Opus is competitively priced after all.

➕ show 2 replies