I love local AI, but even so, I admit the economics point the other way. Unlike gaming, AI is a uniquely good match for data centers because the applications today aren't latency-sensitive at all (except perhaps DLSS/self-driving/robotics). Most importantly, utilization is massively better with always-on servers doing parallelism (in-flight batching). Also in some areas (like California) our local electricity is quite expensive, so moving the work 100ms away can help. The privacy implications are horrifying, though.
> the applications today aren't latency-sensitive at all
This is circular reasoning. We don’t have latency-sensitive applications because sufficiently powerful local AI is not profitable to entertain such applications.
The economics point the other way because capital is aligned against you. Should this be a bubble that pops we will hopefully see prices come down again.
MooresLawIsDead and his spidey web / senses thinks it already popped:
I tend to disagree here. Think about how useful it would be to have a nice llm in real-time debugging your applications loaded entirely on your computer. That sounds pretty sweet to me. I don't see how that's not really cool / immediately useful to you?
Maybe your point is the majority of the time that's not needed, fair, then maybe the approach is both?