As far as I can tell, either LLMs naturally fail to be viable, we collectively decide to put a stop to it, or we end up in one of several dystopias.
My bet is that we end up building an awful outcome for nearly humans, which means that trying to put a stop to it is a duty, even if it's likely to fail. I feel the same about climate change -- it's likely that we're going to fail to keep much of earth habitable, but it's not morally acceptable to stop trying.
As far as I can tell, either LLMs naturally fail to be viable, we collectively decide to put a stop to it, or we end up in one of several dystopias.
My bet is that we end up building an awful outcome for nearly humans, which means that trying to put a stop to it is a duty, even if it's likely to fail. I feel the same about climate change -- it's likely that we're going to fail to keep much of earth habitable, but it's not morally acceptable to stop trying.