alt Hacker NewsIt seems like there will be a collapse in building data centers, to the point that a large amount of the newly built ones will be reclaimed for other things. There is a very large group of people who are against building new data centers because they hate AI (and any pollution related arguments these people make are in support of their primary concern, being AI). But on the flip side of that, even people that I know that go out of their way to use AI, mostly seem interested in building their own AI rigs and running everything locally. I know that's what I do; I almost never use any hosted AI services these days, but I'm using more tokens than ever. The demand for data centers from people who actually like AI even seems to be plummeting.
Replies
How is anti pollution argument pro ai?
And even if you do inference at home you are not training the models. Moreover, most people certainly are not doing inference locally.
What are you running at home, out of interest?
While I cautiously circle the AI programming concept, one thing close to my mind has definitely been whether the ultimately cloud-tied nature of it is what makes me much more stand offish to it.
We're already at the point where a few months worth of an SMB's token usage from the SaaS LLM providers costs is equivalent to the cost of installing on-prem infrastructure capable of running cutting-edge open-weight models at scale. At my company, we recently installed a server with an array of Gaudi 2 cards on our server rack, and have set up an Open WebUI frontend to expose an LLM connected to all of our internal resources to our staff. The total cost was about $20k, which we'd easily eat up with six months worth of equivalent Claude usage.
We'd originally set this up to be able to locally run larger models, in the 300-400 billion parameter range, but the rate of improvement of open-weight models has been so fast that, coupled with extensive custom skill creation, we're now getting similar results out of Qwen3.8 27B to what we were getting out of Qwen 3.5 297B when we started out with the project, which frees enough memory to allow 20-25 users to have 256K context concurrently. Both the hardware, the software, and the models are improving at an accelerating rate.
Investing in data centers to support SaaS LLM providers today feels a bit like investing in mainframes and minicomputers in the late '70s, with a massive paradigm shift lurking right around the corner.
Actually, it's probably already closer to the early '80s, given that purpose-built local AI workstations are already available at price points lower than the inflation-adjusted initial price of the original IBM PC.