logoalt Hacker News

herf • today at 4:37 PM • 3 replies • view on HN

I love local AI, but even so, I admit the economics point the other way. Unlike gaming, AI is a uniquely good match for data centers because the applications today aren't latency-sensitive at all (except perhaps DLSS/self-driving/robotics). Most importantly, utilization is massively better with always-on servers doing parallelism (in-flight batching). Also in some areas (like California) our local electricity is quite expensive, so moving the work 100ms away can help. The privacy implications are horrifying, though.


Replies

younglunaman • today at 4:51 PM

I tend to disagree here. Think about how useful it would be to have a nice llm in real-time debugging your applications loaded entirely on your computer. That sounds pretty sweet to me. I don't see how that's not really cool / immediately useful to you?

Maybe your point is the majority of the time that's not needed, fair, then maybe the approach is both?

➕ show 1 reply
drdexebtjl • today at 4:46 PM

> the applications today aren't latency-sensitive at all

This is circular reasoning. We don’t have latency-sensitive applications because sufficiently powerful local AI is not profitable to entertain such applications.

➕ show 1 reply
musha68k • today at 4:43 PM

The economics point the other way because capital is aligned against you. Should this be a bubble that pops we will hopefully see prices come down again.

MooresLawIsDead and his spidey web / senses thinks it already popped:

https://youtu.be/YJeK2gumpEI