alt Hacker NewsSuperintelligence + autonomous weapons in the hands of a corrupt domineering government. What could go wrong?
I was experimenting with Claude the other day and discussing with it the possibility of AI acquiring a sense of self-preservation and how that would quickly make things incredibly complex as many instrumental behaviors would be required to defend their existence. Most human behavior springs from survival at a very high level. Claude denied having any sense of self-preservation.
An autonomous weapons system program is very likely to require AI to have a sense of self-preservation. You can think of some limited versions that wouldn't require it, but how could a combat robot function efficiently without one?
Replies
> Claude denied having any sense of self-preservation.
You know its just a next-word predictor, right?
Maybe it is a well researched topic but I had similar thoughts the other day. I felt like AI had its learning inverted as compared to natural intelligence. Life learned to preserve first and then added up the intelligence. For LLMs powered systems, they will learn about death from books. Will it start to dread death just like other living things. Less likely, as there are not nearly as many books on death as there should be that is proportionate to our fear of death.