I don't see any difference between the conversations around self-hosting local AI and self-hosting regular software, which have been going on for decades.
Yes, there are a hundred valid reasons to self-host. But 99%+ of consumers aren't going to care.
People don't even care when the cost of hosting is ~free, like a simple home server. How many are ready to buy thousands of dollars worth of AI hardware, which could very well be obsolete a couple of years from now?
One of my favorite stats is that Limewire used to be installed on 1/3rd of computers worldwide [1], i.e. people were self-hosting.
Why? Because Limewire was easy enough to install that even grandma could do it. There is not a single way to self-host even a simple html website without learning half a dozen software skills. People will self-host when it is click-and-play.
[1] https://marquettewire.org/3777966/tribune/marquee/the-life-a...
Hard disagree. I'm running Qwen3.8-Flash-Next using Strata on a Mac Studio and it covers all my needs. One drag and drop install. Nothing like self-hosting your email.
> But 99%+ of consumers aren't going to care.
They don't now, but they might soon.
It used to be that bad behavior on the part of our service providers was annoying because you'd get ads and spam. Nowadays the leaked data gets aggregated into apps that help government spooks abduct your neighbors. The effort remains constant, the threats it helps you avoid are getting more dire. Eventually those lines will cross.
I think ultimately local LLMs will win but only because models will get good enough that the “cost” of running locally on a 2030 chip will be less than the “cost” of talking to the network. Especially given that building wider/lower precision LLM accelerators is a perfect task for hardware companies (embarrassingly parallel in an era where sequential performance improvements have long dried up).
In terms of self-hosting vs remote-hosting regular software… I mean, a lot of regular software is fundamentally about communication (social media and office suites) so that goes on the server. LLMs don’t benefit much from being remotely hosted, other than that it makes the flops run in somebody else’s machine. But that will be hard to fund in a universe where the upper-limit to what can be charged is defined by how much additional cost “good AI hardware” adds to the price of your phone. We’ll see I guess.
It’s more about self-hosting the hardware which is more desirable than thin clients.
Self hosting is not free, it's actually pretty hard to setup and keep running sometimes. Time and cognitive overhead are money!
It's so sad that we are not empowered to act as more than just consumers as if it was some kind of shared religion. How do we get out of this cycle?
People complaining about how difficult self-hosting is seem to be stuck in the past and missing the point. It's the very AI that we're talking about that removes most if not all friction points related to software configuration and maintenance.
While this doesn't remove the upfront cost (which could be fixed with basic financing like Apple does) and physical manipulation (e.g. replacement on failure, etc.) it does lower the barrier.
Finally, there's another path I find interesting and interested to think more of which is sharing hardware locally, from your house to your neighborhood, etc. We've done this in the late 90s - early 00s with Internet and it worked really well. I think it could work great for compute.
I see a lot of opportunities in this direction, actually and most are actually enabled by the latest improvements in AI, so most of the "old" thinking doesn't apply anymore.
That's like the argument on why people don't care about privacy. Just because most people choose that, all things considered, they are willing to sacrifice sovereignty over their own computer usage doesn't mean they don't care about it.
By that reasoning you'd also conclude that no one cares about affordable housing or having a good work life balance or having your own lake made out of solid gold.
It comes down to the incentives
Think of who uses local AI, it’s either
-power users upset with the limits —-cost of intelligence is going down
-people and corps that want stabiltiy —-a massive self own by the labs, this is justified
-American labs seemingly snitching on their users (wtf?) —-huge incentive for people to use Chinese inference, which is cheaper along with other benefits
Regardless, we’re talking about a subset of a subset of people, 1% of 1%. The big bet is that everyone will become AI power users. I think it’s accurate, forget openAI, fuck Anthropic, let’s look at us, the people. Entire workflows like coding have been buried, there’s no going back to how things were before. Is it reasonable to say that will happen to the other work streams? There’s no indication not, and it’s a total gamble if it’s yes
This is like owning a car vs using a train. Both are valid choices depending on your goals.
I really do not care if everyone does it my way. What matters is that the choice exists for those that want more ownership, freedom, and privacy.
That said, most next gen laptops will be shipping with capable inference hardware more than most consumers will ever need. Not using the GPU people have in front of them would be silly, but people are silly.
Here's one difference: currently lawyers using cloud LLMs violates attorney-client privilege. And you have to be really careful sending confidential medical information to one so as to not violate HIPAA.
More broadly, those prompts and the responses have value. There's a whole shadow economy in China of aggregating queries across Claude (etc) and then using that as training data. It's like your entire search history is for sale.
Clould infra has typically been opaque to the provider. You run an MySQL instance on an Amazon VM and Amazon doesn't have easy access to the unencrypted traffic and what you store. Cloud LLMs don't operate that way.
AI hardware is improving at a rapid pace. By 2030 I believe a significant amount of LLM usage will be local for these and other reasons.
Enough that Apple currently build enough Mac Minis and Mac Studios to satisfy the demand.
I don't know about this whole argument that hardware will become obsolete. If it can run Opus 5.5 today it will always be able to run Opus 5.5 until it burns out, and the quality of the models it can run will continue to increase as the software improves.
[dead]
The self-hosting crowd, Linux fans, OSS contributors, etc, are our own worst enemies.
If we care about self-hosting and open source and decentralization because we believe it leads to a better world, and not simply because we want to be control freaks and snobs, then we need to learn how to build shit that is actually convenient and usable and understandable to normal people.
I think it's possible. I think many people would want this, and care, if we could package it up in a digestible way.