Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?

How does this project compare to https://github.com/gavamedia/deltafin ?

Approximate calculation is putting the cost at ~$5 per million tokens (assuming 42W sustained, 20¢/kWh), and that's excluding hardware and other costs.

Neat! But, what do you do with a 0.5tk/s LLM?

Have you tried running it via llamacpp or other software that supports naive SSD offloading to compare speeds?

I couldnt find anything explaining the name of this company on their website but is it okay that they’re riding on the name of an open source tool?

SQLite code itself is public domain but I’m not sure about the name.

This sounds a lot like what the colibri project did for GLM-5.2. I'm a fan so keep at it!

justvugg.github.io/colibri

Dear creator: you didn't ship the first draft of your code - why did you ship the first draft of your README??

So if this Mac uses 30-50W, that's 40-60 tok/Wh...vs maybe 80k for a modern GPU cluster? So that's about 1000-2000x more power for the SSD streaming, unfortunately.

Does it not use Metal, on macOS? Would it be faster if it did?

Where can this 1tb k3.waste be downloaded?

Interesting project. The headline number (29 GB of RAM) is for 4k context.

From what I've read elsewhere, Kimi K3 is quite verbose in its thinking. At the quoted rate, it would generate only a total of 1.8k tokens in 1 hour. Is that enough for it to get any thinking done and produce output on more complicated prompts?