logoalt Hacker News

jillesvangurptoday at 10:25 AM1 replyview on HN

Neal Stephenson touches on this topic in several of his novels. Probably the most concrete of this is "Fall; or Dodge in Hell" which involves a simulation of people's scanned brains rediscovering qualia and constructing their own simulated world from scratch. In the book, two of the deceased digital "souls" eventually mate and produce digital offspring and the whole simulation starts consuming more and more resources.

His Baroque Cycle series also touches on this in several places. One funny side plot involves a freed African slave (Dappa) who speaks dozens of different languages and is highly intelligent and an aristocratic person who maintains that of course this former slave (who is obviously a lot smarter than this aristocrat) is just a trained monkey that naturally is not conscious even though he is quite clever with language. The same books also have a lot of side plots involving Leibniz and various attempts to build thinking/computing machines.

The Dappa plot is probably the closest to a lot of debates there will be around AGI with people likely to insist for all sorts of reasons rooted in philosophy, religion, etc. that even though the AGI walks, talks, etc. like a duck, it can't be a duck. At some point, we'll have AGI that pass any test we can think of and we'll still have people arguing that these cannot be conscious.


Replies

card_zerotoday at 10:30 AM

The clinching thing for me is how the AGI is supposed to work. If it's the same as our theory of how we work, then fine, it's one of us. It wouldn't even have to work very well.