People said the same things about mobile gaming [1] and mainframes. Technology keeps pushing forward. Neural coprocessors will get more efficient. Small LLMs will get smarter. New use-cases will emerge that don't need 160IQ super-intellects (most use-cases even today do not)

The problem for other companies is not necessarily that data center-borne GPUs aren't technically better; its that the financials might never make sense, much like how the financials behind Stadia never did, or at least need Google-levels of scale to bring in advertising and ultra-enterprise revenue.

[1] https://apps.apple.com/us/app/resident-evil-3/id1640630077

> I don't think the throughput of a general purpose device will make a competitive offering; so being local is a joke.

Until the first Cambridge Analytica-sized privacy story hits a major cloud LLM provider, maybe.

> All the fun stuff is running on servers at the moment.

With "Apple Intelligence" it looks like Apple is setting themselves up (again) to be the gatekeeper for these kind of services, "allow" their users to participate and earn a revenue share for this, all while collecting data on what types of tasks are actually in high-demand, ready to in-source something whenever it makes economic sense for them...

Outside of fun stuff there is potential to just make chat another UI technology that is coupled with a specific API. Surely smaller models could do that, particularly as improvements happen. If that was good enough what would be the benefit of an app developer using an extra API? Particularly if Apple can offer an experience that can be familiar across apps.

Also why would you want it sucking your battery or heating your room when a data center is only 20 milliseconds away and it's nothing more than a few kilobytes of text. It makes no sense for the large majority of users' preferences which downweight privacy and the ability to tinker.