alt Hacker NewsWe're talking miraculous level of improvement for a SOA LLM to run on a phone without crushing battery life this decade.
People are missing the forest for the trees here. Being the go to consumer Gen AI is a trillion+ dollar business. How many 10s of billions you waste on building unnecessary data centers is a rounding error. The important number is your odds of becoming that default provider in the minds of consumers.
Replies
It’s extremely easy to switch.
I used ChatGPT for every day stuff, but in my experience their responses got worse and I had to wait much longer to get them. I switched to Gemini and their answers were better and were much faster.
I don’t have any loyalty to Gemini though. If it gets slow or another provider gives better answers, I’ll change. They all have the same UI and they all work the same (from a user’s perspective).
There is no moat for consumer genAI. And did I mention I’m not paying for any of it?
It’s like quick commerce, sure it’s easy to get users by offering them something expensive off of VC money. The second they raise prices or offer degraded experience to make the service profitable, the users will leave for another alternative.
In fact it's apparently $5.2 trillion by 2030 [0] (out of $6.7T total data center spend; meaning all of "traditional IT needs" are less than a quarter of the total). That's the total if you add up all of the firms chasing this opportunity.
I do wonder, if you (and the commenter you replied to) think this is a good thing, will you be OK with a data center springing up in your neighbourhood, driving up water or power prices, emitting CO2? Then if SOTA LLMs become efficient enough to run on a smartphone will you be OK with a data center bailout coming from your tax dollars?
[0]: https://www.mckinsey.com/industries/technology-media-and-tel...
Just build more tulip fields/railways/websites no matter the cost - the golden age is inevitable! ;-)
How big does an LLM need to be to support natural language queries with RAG?
> The important number is your odds of becoming that default provider in the minds of consumers.
I haven't seen any evidence that any Gen AI provider will be able to build a moat that allows for this.
Some are better than others at certain things over certain time periods, but they are all relatively interchangeable for most practical uses and the small differences are becoming less pronounced, not more.
I use LLMs fairly frequently now and I just bounce around between them to stay within their free tiers. Short of some actual large breakthrough I never need to commit to one, and I can take advantage of their own massive spends and wait it out a couple of years until I'm running a local model self-hosted with a cloudflare tunnel if I need to access it on my phone.
And yes, most people won't do that, but there will be a lot of opportunity for cheap providers to offer that as a service with some data center spend, but nowhere near the massive amounts OpenAI, Google, Meta, et al are burning now.