alt Hacker NewsBut why though? Cannot possibly be useful at such slow speeds, and costs a ton to perform that badly
Replies
I guess the point of this whole forum is "Why not?"
I've had the thought that an underexplored use case for llms is treating them as classification machines. Put in a lot of input then ask for a single token classifying the input. Like "does this code have any bugs? reply "yes" or "no""
Doing something like that at 1 token per second may be useful.
The Github README literally has a "But Why?" section
It's not useful for actual work, but the fact it can be run at all shows that we're evolving towards enabling powerful LLMs to run locally.
Because we can. And K3 is frontier-like. Running on a MacBook Pro.
I hate seeing comments like this on HN. We used to upvote “look at this crazy thing I did” work. Not everything has to make sense or be ground breaking.
It is cool that they got it to work at all.
Not useful for chat, agreed — and I wouldn't pretend otherwise. It's useful for the other kind of work: scheduled, unattended jobs where nobody is waiting on the cursor. My use is day/week/month end review — go through the numbers, flag what doesn't reconcile, draft the report — and there the two things that matter are that the model is good enough to trust with the judgement (K3 is, and it's the full 2.8T model, not a cut-down one) and that the data never leaves the machine.
I've never understood why "Hacker" News so frequently gets "But why though?" comments at the top.
The entire history of innovation is filled with people doing something just to see they can get it to work, even if badly, and then people continue to iterate on that until it works better, then works well, and then is so obvious people would never even question it. But it all starts with someone doing it to scratch an itch.
Neural networks, the foundation of our current AI revolution, used to fit well into the "neat, but practically useless" category.
Sure there are countless "but why though?" experiments that don't pan out, but that's just the cost of exploration. There can be no step-function innovation in a world where people only do things that make immediate practical sense.