logoalt Hacker News

younglunaman • today at 4:51 PM • 1 reply • view on HN

I tend to disagree here. Think about how useful it would be to have a nice llm in real-time debugging your applications loaded entirely on your computer. That sounds pretty sweet to me. I don't see how that's not really cool / immediately useful to you?

Maybe your point is the majority of the time that's not needed, fair, then maybe the approach is both?


Replies

herf • today at 6:29 PM

Well, LLMs handle raw data perhaps a million times slower than conventional algorithms, so in "bytes" and "latency" terms, a dialup modem can handle most AI communication today - a rack of cloud GPUs 100ms away is still faster for a lot of things than $10k of hardware in your GPU, Mac, etc.