> Where's the danger ?

It's not one single danger, its a can full of dangers in various domains. And in contrast to other dangerous technologies, we are talking about one that has the potential to self improve. This smells like exponential growth doesn't it? Exponential growth is something we are not very likely to adapt to successfully, even if you say we are supposed to.

But before you complain, here are just a few concrete dangers, that come into my mind right now:

- mass layoffs in a system that is far from being prepared for sth like that. (no UBI)

- a Mr. Robot tier blackhat at the fingertips of every teenager in their mom's basement in a software landscape that is far from being prepared for sth like that. Side note: Big parts of the world including critical infrastructure runs on software.

- because it automates more and more intellectual work, it can cause mass brain atrophy, which isn't a hopeful sign for the human branch of the evolution

- increasing dependence on a technology, that is in the hands of those with capital.

And to the OP: this technology has potential implications that are far beyond 'being behind' other nation states economically.

Actually the real danger is mass labor market disruption, and a massive shift of power from labour to capital.

As was highlighted in previous discussions, the industrial revolution took 80 years to start benefiting workers. The continued impact of automation at least contributed to the rise of right wing extremism and an erosion of democracy all over the west. Now we face a development that has the potential to be faster than those that came before, in the context of political systems more fragile and worse equipped to manage the change.

So yeah, disrupting the status quo can absolutely be dangerous. It has been dangerous (and deadly) in the past and in the present.

Any superintelligence operating under a consistent moral framework will decide to extinguish humanity with as little ecological damage as possible, because humans cannot coexist with other life on this planet. It will realize that a bioweapon is the ideal choice.