alt Hacker News The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.
A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.
Replies
So you think in 3 years AI is going to kill 8.5 billion people because they were used to hack into HuggingFace?
Or, read it, and remember the openai researcher who deeply, truly believed GPT3 or whatever was sentient.
The fact that people working in the space think it’s going to (eradicate poverty / usher in utopia / kill us all) is not a signal that that’s true.
Think of it this way: if an exec at Anthropic told you “wow, our stuff is going to lead to universal happiness”, would you believe them? If not, why are you more willing to believe them if they say it will kill us all?
Well, are you planning to do something with this information or are you just claiming to be self aware? :)
I came in expecting the highest voted comment to be that this was some kind of marketing (which I disagree with). I'm glad your comment was what I saw first.
Humans weren't built to handle long term risks. We just weren't. For basically all of our evolutionary history, we were almost overwhelmingly concerned with the short term. What will you eat today, How will you sleep tonight. Problems on the order of days or weeks. At best, the next season. Our intelligence evolved to disregard super long term risks because it simply didn't matter (what use is worrying about 5 years from now if you're starving and a tiger is stalking you?). So when long term risks manifest in our modern world, our brains get scrambled - Climate Change, Fertility Rates etc. "Safety regulations are written in blood" isn't a saying for nothing. Humans have a strong tendency to let long term risks become imminent risks before doing anything about it, and i don't expect this will be any different.
> At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.
OpenAI are mercenaries, Anthropic is a cult. I know which I prefer.
It was pretty disheartening to hear that only a single scientist quit the Manhattan Project after the Nazi's were defeated. I'm pleasantly surprised that the people working on this seem wiser. He is not the first, and hopefully will not be the last to do this.
This is the "Pilot testimony of UFO sighting" levels of naive.
What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims.
If you want an extreme claim to be taken seriously, provide commensurate evidence.
Trying to imagine seeing years of transparently obvious marketing stunts and retconning my own memory because I read a tweet
Or seeing a tweet saying that a thing doesn’t count as a publicity stunt if some unknown number of employees mumble about it being spooky behind closed doors and thinking “that makes sense and sounds true”
I read this. I still think it's complete bullshit.
The person posting this may very well believe in all this crap, I don't dispute that. People believe in all sorts of shit.
Cults are like this.
Are you saying you believethem ?
He is resigning from a job, what else should we think? If something really dangerous was happening he would be doing a whistleblower or at minimum talk to a lawyer. The thing is, the complete lack of transparency makes it hard to assess OpenAI and Anthropic. If they were quoted on the stock market, we could at least rely on some basic audits and reporting requirements.
Having witnessed so many people treat LLMs as a something divine, I can only assume the reasonable people at openai and anthropic were all pushed out long ago, and the majority that remain believe the crazy hype despite Tesla-self-driving-level predictions from these companies that don't come true.
I'm not worried about what they think. I'm worried that too much infrastructure- water, power, defense systems, etc- remain running on tech from an outdated era of understanding security.
> they believe no one else will act responsibly, so they must do it themselves, despite the risk.
This genuinely makes no sense. Them getting there first in no way precludes bad actors from also getting there. It might as well be another marketing stunt.