logoalt Hacker News

MSFT_Edging • yesterday at 3:43 PM • 2 replies • view on HN

This is such an interesting part of the whole AI equation.

The main benefit of the closed models is the regulatory control to say, insert guard rails against hacking things. That's also the main drawback as you now have handed control over this technology to a handful of companies, who will reap all the rewards of this captured sector.

It would be like the google/apple duopoly on steroids. Everyone forced to buy in to one of two ecosystems, with limited to no ability to undermine the duopoly via control of the device hardware. One can't make a real competitor because it requires access guarded by existing relationships and money. For the AI houses, it means you need approval from these groups to create in a competitive manner. That, IMO is too much power for an unelected organization to carry.

AI will likely only provide real benefits to humanity if it's open. But if it's open, it will also enable extreme harm in the form of cyber attacks, scams, and destruction of IP.

By keeping AI closed, it will bring a new form of imperialism. Instead of the core maintaining IP and the periphery suffering the secondary effects of industry, the core will maintain the capability and the periphery will be locked out of competition due to silly nation-state rivalries. This won't benefit humanity, it will lock us into the same cycle of misery for marginal gains measured in stock values.

So there's two ways AI can reasonably go: Cyber punk corporate overlords or cyber anarchy due to the inability to put a cork on the genie's bottle.

Both seem terrible to me. What good is space travel without freedom? What good is automation if there are incentives to maintain profit over stability? What good is the technology if any usage of it exposes you to fraud, violence, and blackmail at any moment?

Big fans of LLMs never ask "to what end?", they repeat shallow near term "wows" without considering the second and third order effects. "wow think about how good LLMs can spot cancer in scans!", great! but who will afford the scan without work? Will the overlords commit to luxury gay space communism? Or will they simply say "tough luck" and let that cancer fester?

Real AGI is incompatible with our current power structures, yet those current power structures desire the AGI. This cannot end well. Stagnation and stability would be preferable to the level of misery that would follow.


Replies

seiferteric • yesterday at 3:53 PM

I generally agree, but the main reason I am pro AI (or at least feel it is necessary) is because I do think it will end up helping us solve other problems that I think are existential in themselves like climate change. Maybe it won't happen, but personally I think it will and don't want to go into more detail because some of it will probably seem far fetched but I do think it will happen now. And previously these problems seemed intractable or were only being solved too slowly for us to escape the worst outcomes. So now I think we have this situation where to solve some of our worst problems, we are now creating other problems...

➕ show 3 replies
YZF • today at 6:17 AM

I don't know but here's a counter-view. AI right now isn't really smarter than people. It is smarter in some niches but not others. It scales some things by allowing cheaper intelligence to work more broadly. We already have 8.3B of "intelligent units" on this planet. The "intelligence bandwidth" of AI is still a lot less than that. We already experienced increases in "intelligence bandwidth" by increase in number of people, better education etc. The point that I'm making is that an increase in that bandwidth on its own isn't necessarily a problem. If we look at AI "smarter" than us then we don't know but we do know that we have people who are "genius" (e.g. much smarter than the average person) but that on its own doesn't unlock any magic. If AI has an "IQ" of 200 or 300 or 400 we don't really know what that means in how far it can push on any given topic.

Our economy adapts and find equilibrium when things change. We don't need AI to get oppressive societies. We already have those. Does it make it easier to have one? Does it make it harder? I donno. But a lot of this depends on governance and policies. If AI is used for "good" it could mean a better life for everyone. That's not necessarily opposing the interests of AI companies or their owners. What sort of jobs will those people have and how different will those be than the job market today- I donno. Maybe we'll need more software engineers because we'll be building x10,000 the amount of software or maybe we'll be part of a service economy or planting trees. But the economy isn't as simple as "AI can now write code therefore this is the end of software engineering" just like making software engineers a lot more productive (and we have) didn't reduce the number of software engineering jobs. It created more.

In terms of AGI's compatibility with power structures. I don't think there's one answer there. Does it fundamentally change the different forces driving us? If AGI is smarter than us then what does it want? Clearly a lot of what AI offers today is still tied to existing systems. Can spot cancer in scans feels like a good thing. Somebody has to pay for it otherwise why would anyone build it? Like anything else there will be supply and demand and if supply is high and demand low the price will go down. Why should it necessarily drift more towards anarchy or totalitarianism? Why can't it be "free/capitalist + AGI"?