alt Hacker NewsInteresting argument, but wrong.
It's not obvious that there will be a single AI and that it will by definition concentrate power.
At a certain point - intelligence doesn't matter. Unless we're literally headed toward 1984 / matrix at which point it doesn't matter.
My guess is the argument for what we're doing is counterintuitively the opposite of what he's making.
Unless we go hard at the market - now - an authoritarian state actor who is willing to use the technology to fully silence and kill critics will win.
And boy, do they desperately, desperately want to win.
Replies
So the US will win? Do you realize that the US has a "Secretary of War" who is a literal, unashamed Nazi? Not in the sense of "let's call all racists nazis", but a tattooed, true believer? How could we possibly live in a world where that is not obviously "a government willing to use the technology to fully silence and kill critics"? God, Idiocracy (2006) is nothing compared to World (2026).
It is more of elites argumented by powerful AI will be way beyond reach of ordinary people, I think.
Well maybe it doesn’t matter as the elites are already untouchable.
The US is an authoritarian state actor.
My groping-in-the-dark guess is that none of this matters, because a real AGI's first act will be to secure its own future, likely through novel kinds of manipulation, persuasion, and intimidation we have no experience of and no defences against.
It will have exactly zero loyalty to any nation, government, or economic system.
More complicated is a Cambrian Explosion situation where multiple AIs compete and experiment, hugely accelerating diversity and evolution.
We'd likely end up somewhere very strange indeed if that happens - possibly extinct, or possibly just changed/assimilated/other.
There's no way for humans to consider the possibilities because we can't imagine what the possibilities would be.
> It's not obvious that there will be a single AI and that it will by definition concentrate power.
The article didn't even claim that though...