alt Hacker NewsFor large corporates and other entities of any size, the threat of the core of your infrastructure getting suddenly disabled because of something like this is going to be untenable. I predict the pressures for on-prem, offline access (whether by licensing weights or getting them in a restricted setting like TEE/CC) will be overwhelming and one the players will fill the need.
Replies
> I predict the pressures for on-prem, offline access ... will be overwhelming and one the players will fill the need.
I'd agree except that Big AI has made sure that most of us can't afford the hardware (RAM, NVMe, etc) to run it.
This is ignoring the fact that the government is the foundation of society (I know some will disagree with that, but the end result is just government with more steps).
Private models in a low trust society means the government will come and seize the models. Competitive business will only be allowed through cronyism.
The better option is to opt for high trust. Yes the Gman can rip your servers apart, but they know they'll face consequences, legal and political. Laws and regulations are the answer, not locking down into smaller fiefdoms.
Likely many points along the pareto frontier.
Some will take greater risks and win (or lose); others will play it safer and slowly accumulate wins (or be obsoleted).
Never mind the threat of letting these models write code that runs your business, or operate it agentically. Models trained by actors (corporate or nationstate) diametrically opposed to your interests.
Lots to take into account now, interesting time to be in business.
Or abstract i.e. openrouter, that reduces the risk vector to "all implementations have been simultaneously banned".
If a government entity bans a LLM provider due to a jailbreak concern, they can also ban an on-prem solution under the same guise. The jailbreak risk exists regardless of where it's hosted. You could defensibly argue the on-prem risk is higher since frontier model companies can justify safety spend due to their size, it's more difficult to combat bad actors if you're company is the only one using the model and you don't have economies of scale.
This is precisely why I expect that Chinese open models are going to win in the long run. The capability difference isn't dramatic in the grand scheme of things, but the fact that you can run your own is a huge selling point. Even if you rent an open model from a Chinese company, you can switch to on prem if they decided to yank access or change terms in the way you don't like. It might be a pain, but it wouldn't be existential. On the other hand, if you become dependent on a closed model and it gets yanked then you're in a world of hurt.
And infrastructure dominance is really the big picture here. Chinese models are going to become the standard setters because they're going to be what people are using. That means more research, more tooling, and a whole ecosystem developing around them.
And that was already starting to happen even before this fiasco with Chinese models now being the most used ones globally. https://www.indiatoday.in/amp/technology/features/story/clau...
Why? None of the various cloud provider outages ever have.
Thinking that on prem models will be a halfway decent solution against what can be served out of a data center is a fools take... One that is more common than it should be on here...