I think the debate around this is the perfect example of why the ai debate is dysfunctional. People who treat this as interesting / worrying are observing it at a higher layer of abstraction (namely, agents with unbounded execution ability, who have above-amateur coding ability, networked into a large scale network with shared memory - is a worrisome thing) and people who are downplaying it are focusing on the fact that human readable narratives on moltbook are obviously sci fi trope slop, not consciousness.

The first group doesn’t care about the narratives, the second group is too focused on the narratives to see the real threat.

Regardless of what you think about the current state of ai intelligence, networking autonomous agents that have evolution ability (due to them being dynamic and able to absorb new skills) and giving them scale that potentially ranges into millions is not a good idea. In the same way that releasing volatile pathogens into dense populations of animals wouldn’t be a good idea, even if the first order effects are not harmful to humans. And even if probability of a mutation that results in a human killing pathogen is miniscule.

Basically the only thing preventing this to become a consistent cybersecurity threat is the intelligence ceiling , of which we are unsure of, and the fact that moltbook can be ddos’d which limits the scale explosion

And when I say intelligence, I don’t mean human intelligence. An amoeba intelligence is dangerous if you supercharge its evolution.

Some people should be more aware that we already have superintelligence on this planet. Humanity is an order of magnitude more intelligent than any individual human (which is why humans today can build quantum computers although no biologically different from apes that were the first homo sapiens who couldn’t use tools.)

EDIT: I was pretty comfortable in the “doom scenarios are years if not decades away” camp before I saw this. I failed to account for human recklesness and stupidity.