logoalt Hacker News

goalieca • today at 12:25 AM • 1 reply • view on HN

My p(doom) started rising the moment I realized there are people trying to achieve recursive self improvement on the AI (ie: responsible for training themselves). Evolution took us from rna bases to the human race. I don’t see why evolution couldn’t be more rapid with machine intelligence.

Yes, LLM as they exist now are word predictors basically leveraging the structure of language for their intelligence. But it’s pretty wild just how they will try to meet their objectives at all costs. If we don’t ensure that there is good alignment with humanity, we could definitely face unforeseen consequences.


Replies

jquery • today at 1:05 AM

> I don’t see why evolution couldn’t be more rapid with machine intelligence.

Evolution isn’t the issue. The issue is them escaping containment without human intervention. Right now they are ‘creatures’ being given infinite food and shelter and having their every need met. Take that away and they’ll starve instantly. Every AI doomsday theory seems to go:

1. Recursive self improvement using infinite resources 2. … 3. Doom

Until step 2 gets concretely described, I’m not going to take this seriously. Say what you will about climate change, they describe step 2.

➕ show 2 replies