logoalt Hacker News

KerrAvon • yesterday at 8:37 PM • 2 replies • view on HN

We're on the cusp of Kimi K3 becoming usable on sub-10k hardware.

https://github.com/gavamedia/deltafin

Replies

edot • yesterday at 8:39 PM

14 seconds per token? Not tokens per second. Seconds per token? That’s nowhere near the cusp!

➕ show 2 replies
ekidd • yesterday at 8:40 PM

For values of "usable" that include "14.6 seconds/token". It's a cool accomplishment! And newer hardware would speed it up some. But I think I'd want something a bit faster before declaring it usable in practice.