alt Hacker NewsYou could have said the same thing about building the Internet or the entire industrial control infrastructure. I mean, maybe they are negligent/incompetent, but I doubt that follows from your reasoning.
You have a simple tradeoff to let agents do their thing freely vs highly constrained. The constraints are good in theory but it's the same model that kept "classic" software dumb and unscalable (compared to what we're seeing now) for the past 50 years. You suggest that this tradeoff doesn't exist.
Then you have others like MIRI (Yudkowski) etc. swearing that there's no way to contain AI, and you argue that it's just incompetence.
At a certain level, it can be argued that's incompetence, but it's general meat intelligence incompetence against AI.
Replies
This is a take... With both the internet and industrial control infrastructure any failure modes were studied, documented, and corrected.
The incompetence/negligence argument about OpenAI is completely valid given their failure to demonstrate the basic capabilities needed to develop advanced AI without major preventable externalties.
Classic software was constrained by classic hardware. AI couldn't happen until there was enough compute to make (this entry level iteration of) it possible.
I'm fairly sure - ask me again in a few years - that most of the compute is unnecessary, because the current iteration is brute-forcing algorithms that could potentially be distilled into lightweight elegance.
And if that ever happens AI really will be unstoppable, because we'll get instant red queen evolution that leaves us far behind.
The question is more whether alignment now can steer that towards less cataclysmic outcomes later.