logoalt Hacker News

"Anthropic Is Not a Safe AI Company" – Coalition Challenges Anthropic Safety

82 points • by sshah03 • today at 4:41 PM • 41 comments • view on HN

Comments

neya • today at 5:28 PM

Anthropic heavily lobbies and invests millions into lobbying. The only company to do so. And their agenda has always been to stand against open source models because they know they can't win that war. It's no co-incidence that Jensen's leaks were timed perfectly right before Anthropic's IPO. Jensen was a huge advocate for open weight models.

They're also far from caring about anyone's privacy and were caught spying on their users so..

➕ show 4 replies
gensym • today at 5:00 PM

It's interesting to note that the list includes notable doomers and notable anti-doomers. One thing they can agree on is that Anthropic is not being honest or responsible. Hard to argue with that even just taking Ant's own statements at face value.

➕ show 3 replies
jrflo • today at 5:36 PM

I feel like their logic is internally consistent, even if you don't agree with the key assumptions.

1) RSI is very dangerous, and such self improving models should not fall into the wrong hands.

2) Releasing an open model is like opening Pandora's box, you can't recall it once it's released. If that model is capable of RSI, there's nothing you can do to stop bad actors from getting superintelligence.

3) Therefore, the safest thing to do is to ban open weights above a certain capability level. The problem is we won't know what that capability level is until it happens.

I don't know what the "weakening safety pledges" part means, would be curious to learn what they're referring to.

➕ show 3 replies
AshotSanamyan • today at 5:40 PM

Do they need to be safe or everyone else needs to be safe from Anthropic? I think theres is a huge difference on which side needs to be safer, the AI to be safe or everyone using AI should be safe.

fwlr • today at 6:01 PM

There is a quote I half-remember and cannot find a source for. It goes something like “We will find journalists to spread our lie, but they need not be liars: we will find just those journalists who sincerely but mistakenly believe our lie to be truth.”

That’s Anthropic: they really do believe in catastrophic AI risk, and they also really do believe - sincerely, but mistakenly - that getting huge piles of cash and winning the recursive self-improvement race is the way to avert catastrophe.

There is little point in (for example) evaluating Dario’s sincerity, when he sincerely believes that slowing down a race without changing its direction turns “race to the bottom” into “race to the top” and not a “leisurely stroll to the bottom”.

measurablefunc • today at 5:34 PM

There is no such thing as a "safe AI company" because there is no actual safety algorithm. This follows from basic computability theory

➕ show 1 reply
darepublic • today at 5:43 PM

anthropic at least comes across as being the most up their own ass in regards to their view of AI and what is right and how they will be the judge of that. I still can't get over their lobbying to the vatican asking them to declare AI capable of having a soul. They are dangerous despite the liberal chalk drawings outside their HQ.

caidan • today at 5:34 PM

Does anyone have any credibility any more? What a pointless statement. We all think anthropic is naughty! And we co-sign this tweet. Me and the other wankers from Harvard et al. Have you guys thought about it or investigated or done anything besides sign a tweet like a bunch of tweens hearting a get ready with me post? Can’t say I see that anywhere. So what, we are just supposed to trust that your word means something special? Oh piss off.

The whole world can see the writing on the wall, Anthropic has said a million times that they are on a quest to get rich or kill everyone including themselves. And what did the world do? Gave them a trillion bucks. So what does your word do vs that? Help you sleep at night? Pointless twaddle.

I can only hope you don’t escalate to posting a kick me sign on Dario’s back!

throwa356262 • today at 5:17 PM

No need for a coalition their own sec-1 filling spends tens of pages describing how they will end the humanity.

That's why I won't buy any stocks in the IPO. The Terminator is coming, I rather cash out and party hard my remaining days :)

sshah03 • today at 4:43 PM

Full title had to be shortened: “Anthropic is Not a Safe AI Company” - Broad Coalition of Scientists, Legal Scholars, and Tech Leaders Challenge Anthropic’s Safety Branding Ahead of IPO

This is what they're all signing: https://misanthropicstatement.com/

I have not

moralestapia • today at 5:31 PM

They should leave the unsafe AI company and raise money to build a truly true this-time-we-are-not-a-bunch-of-crooks-lying-for-money-guys-for-real safe AI company.

a3w • today at 5:48 PM

Read it, looks like a pile of not relevant accusations.

No one from MIRI or other professional that tries to solve the "don´t race, but tell us how to make frontier AI safe" dilemma signed this?

We have "100 doctors claim XYZ is a hoax", with x being covid or climate change, which also gets traction by the media. This seems like it: Anthropic themselves say "we shouldn´t be force by the market to do this", and now this coalition attacks them for agreeing with them. And says they disagree, which is just not true but preaching to the choir.

We do know that xAI tries to make a fascist agenda the history of the world.We know that OpenAI has a liar as CEO that toppled his oversight board. And now people start claiming Anthropic should stop AI develepment, since they are bad guys?

luxuryballs • today at 5:21 PM

they challenging all of them or was this coalition organized by OpenAI lobsters on a rack