alt Hacker NewsI see several problems with this,
- by definition, the AI isn't human, irrespective of its computational abilities, it needs a human to tell it what being human is, and the ways "humanity can be advanced" need to be validated on this basis
- "advancing humanity" isn't a one dimensional problem/single-KPI optimization game, you might optimize certain things (e.g. expected life expectancy) at the detriment of others (e.g. freedom of movement). If you are not understanding the proposition, you are not understanding the target outcome.
- even if the target outcome is perfectly formally specified and commonly understood (which it can't), you want your AI to rationalize that the journey to get there is the most direct and efficient.
- especially so since subsequent runs of the same prompt will provide different plans
- anything less than that is begging to be conned: as a sensible person, you wouldn't give unlimited power and all your faith to a single individual trusted to "advance humanity" if they cannot be understood. On what ground would you give the machine a pass?
In all, such comments really worry me. It's like AI is triggering for some people the kinds of oppressive religious feelings whereby the individual should submit itself to the will and desires of a pretended all-powerful being. Do you really think our ancestors chose to cut off their legs in abandonment when they figured that some animals could outpace them? No, they built traps and throwing weapons to catch them and see how they taste.
I agree that AI needs alignment and validation, and this is already been worked on [*1]. I'll assume below that this is a given.
For the rest, AI itself may be able to handle better than most humans. It already has good idea about what being human is [*2], understands that this isn't a one-dimensional problem, can do balancing like humans would, find more direct/efficient paths than humans. For many things I discuss with AI, I find that it already reasons much better than most humans (not even considering that AI knows so much more than any human).
I think we over-index on "the same prompt will provide different plans". Humans would do the same too. Humans, working in isolation or with collaboration can self-correct, but the same applies to AI.
>> On what ground would you give the machine a pass?
The benchmark I hold is humans themselves. Humans currently have the pass, and have had it since history. Yet, there are many irrational decisions everywhere around.
I hear complaints that AI hallucinates. Yes, it does. Humans do too, and more often than they are willing to admit. The concept of 'god' may entirely be a hallucination (i.e., something not supported by facts). There is no good scientific evidence of prayers working, yet many humans believe in the same.
The real issue is that AI seems to learn and hallucinate in a different way than humans -- it sometimes makes some very silly mistakes. No disagreement, we need to make it better. The pace at which AI can become better however could easily surpass the speed at which humans learn or change. I am not suggesting we give AI the pass till it becomes good enough, there's enough progress on alignment, etc.
>> Do you really think our ancestors chose to cut off their legs in abandonment when they figured that some animals could outpace them?
Great! Here lies an important point. Between humans and animals, we have nature's evolutionary processes, survival of the fittest, ... Depending on how one sees it (and this is a real debate in my mind), we could use AI to enhance our survival and progress faster, or we could see AI as an enemy (like it is another species) and compete.
For some people, the goal is exactly advancements of humans. I, so far, see it as evolutionary progress, whatever form it takes.
[*1] Whether we are doing enough for that or not, is a valid debate.
[*2] AI cannot experience it, it cannot 'know' it in the human sense of knowing if that means something different, but it could emulate well enough.