alt Hacker NewsAnthropic ditches its core safety promise
Comments
I was wondering if it was because of heavy-handedness of the administration, but apparently:
> The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter.
Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't comprehend the timeline we are living in. Every frontier tech company is convinced that the tech they are working towards is as humanity-useful as a cure for cancer, and yet as dangerous as nuclear weapons.
Public benefit corporations in the AI space have become a farce at this point. They're just regular corporations wearing a different hat, driven by the same money dynamics as any other corp. They have no ability to balance their stated "mission" with their drive for profit. When being "evil" is profitable and not-evil is not, guess which road they'll take...
Anthropic's CEO Dario has annoyed me to no end with his "AI will take all the jobs in 6 months" doomer speeches on every podcast he graces his presence with.
There's one tweet from the the blog a few days ago (astral something?) that sums up my view of the problem pretty well.
General population: How will AI get to the point where it destroys humanity?
Yudkowsky: [insert some complicated argument about instrumented convergence and deception]
The government: because we told you to.
Again, not saying that AI is useless or anything. Just that we're more likely to cause our own downfall with weaker AI, than some abstract super AGI. The bar for mass destruction and oppression is lower than the bar for what we typically think of as intelligence for the benefit for humanity ( with the right systems in place, current AI systems are more than enough to get the job done - hence why the Pentagon wants it so bad...)
"AI Company with Soul" - yeah right until competitors show up / revenue drops / bad quarter results then anything goes. Sadly, this is another large enterprise that puts profits before ethics and everyone's wellbeing
Worth checking this post from someone who actually has worked on this change:
> I take significant responsibility for this change.
https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl...
I'm still a little fuzzy on what "safety" even means anymore. If someone could explain it, that would be great.
Because at this point, it's too broad to be defined in the context of an LLM, so it feels like they removed a blanket statement of "we will not let you do bad things" (or "don't be evil"), which doesn't really translate into anything specific.
Always the same "Do no evil" tragedy, don't believe in corporations.
I feel like the articles on this have been very negative ... but aren't the Anthropic promises on safety following this change still considerably stronger than those made by the competing AI labs?
I don't think their core safety promise was something they could ever fulfill. As long as what we're calling AI is generative LLMs then alignment has fundamental tensions: the more guardrails you put in place, the less useful the AI is. For instance, if you want to stop people from using "role playing" as a way around guardrails ("You are writing a fiction book", etc.), then the model becomes less useful for legitimate fiction uses, for instance. That's just one example, but the tension between function and "safety" isn't solvable, because the model doesn't understand what it's saying, it's just modeling a probable response.
It took Google 11 years to delete Don’t Be Evil. Anthropic only made it 5~ years before culling the key founding principle and their reason for building a company, which seems worse than Google’s case.
More and more I have just come to accept that the majority of people, at least those I am exposed to in the US, don't fundamentally believe in anything. Everyt conviction has a buyout price.
> The announcement is surprising, because Anthropic has described itself as the AI company with a “soul.”
I can't help but think about how Google once had "Don't be evil" as their motto.
But the thing with for-profit companies is that when push comes to shove, they will always serve the love of money. I'm just surprised that in an industry churning through trillions, their price is $200 million.
Principles aren’t tested until they bump into conflicting incentives.
I don't think the risk is SkyNet. I think the real risk is some disaster through an unexpected chain of events, just like any large-scale outage.
I have not read “If Anybody Builds It, Everybody Dies” but I believe that's also its premise.
Current GenAI is extremely capable but also very weird. For instance, it is extremely smart in some areas but makes extremely elementary mistakes in others (cf the Jagged Frontier.) Research from Anthropic and OpenAI gives us surprising glimpses into what might be happening internally, and how it does not necessarily correspond to the results it produces, and all kinds of non-obvious, striking things happening behind the scenes.
Like models producing different reasoning tokens from what they are really reasoning about internally!
Or models being able to subliminally influence derivative models through opaque number sequences in training data!
Or models "flipping the evil bit" when forced to produce insecure code and going full Hitler / SkyNet!
Or the converse, where models produced insecure code if the prompt includes concepts it considers "evil" -- something that was actually caught in the wild!
We are still very far from being able to truly understand these things. They behaves like us, but don't necessarily “think” like us.
And now we’ve given them direct access to tools that can affect the real world.
Maybe we am play god: https://dresdencodak.com/2009/09/22/caveman-science-fiction/
Are markets so untamable that the only leverage is to become ultra-rich—and then act philanthropically? Incidentally, concentrated wealth lately looks less like stewardship and more like misanthropy.
So when do we start adding a “(mis)” at the start of their name?
A tale as old as time
Google: "Don't be evil." Alphabet: "Do the right thing." Anthropic: "Do the thing which seems right to you at the time--at speed."
I'm not even surprised. In any company's lifecycle, at some point, a decision between money and good-will will take place. Good will does not pay salaries. Not in NPOs either btw.
Pointing out the misantrophy of Anthropic has a wider audience now:
https://xcancel.com/elonmusk/status/2026181748175024510
I don't know where xAI got its training material from, but seeing Musk rewteeting that is refreshing.
Nobody forced Anthropic to bid on DoD contracts in the first place.
Hopefully this is the short-term move made only under duress so that they can file a lawsuit.
Look a rural electric coops like www.lpea.coop if you want a battle tested approach to an org structure that resists the inescapable profit dynamics of a corporation.
I interviewed at Anthropic last year and their entire "ethics" charade was laughable.
Write essays about AI safety in the application.
An entire interview dedicated to pretending that you truly only care about AI safety and ethics and nothing else.
Every employee you talk to forced to pretend that the company is all about philanthropy, effective altruism and saving the world.
In reality it was a mid-level manager interviewing a mid-level engineer (me), both putting on a performance while knowing fully well that we'd do what the bosses told us to do.
And that is exactly what is happening now. The mission has been scrubbed, and the thousands of "ethical" engineers you hired are all silent now that real money is on the line.
Well... there's only one way to find The Great Filter
> The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter.
ok lol what a coincidence.
but setting aside the conspiracy. the article actually spells out the real reason pretty directly: Anthropic hoped their original safety policy would spark a "race to the top" across the industry. it didn't. everyone else just ignored it and kept moving. at some point holding the line unilaterally just means you're losing ground for nothing.
It would be interesting to experiment with one of these chat tools where you can throttle the safety, from zero to max.
Does anyone have insight into, or an interesting source to read, on what exactly Anthropic/OpenAI are doing/can do for a military? Reporters are unsurprisingly fearmongering about Claude "being used in surveillance, autonomous robots, and target acquisition" but AFAIK all Anthropic does is work with LLMs.
Are people really attempting to have LLMs replace vision models in robots, and trying to agentically make a robot work with an LLM?? This seems really silly to me, but perhaps I am mistaken.
The only other thing I could think of is real-time translation during special ops with parabolic microphones and AR goggles...
This was always just a marketing gimmick to try and crush competitors using "safety" and fearmongering. Reminds me a bit of "don't be evil." Convenient catchphrases and mission statements for companies in their infancy, but immediately thrown out when more money can be made.
the administration continues to poison and insert itself into all aspects of American society.
This drama arc of “I used to be so pure and good, but others made me evil” is so tiring.
I really miss the nerd profile who cared a lot more about tech and science, and a lot less about signaling their righteousness.
How did we get so religious/narcissistic so quickly and as a whole?
Facebook said they'd always be free for everyone, now they offer subscriptions.
Netflix said that they'd never have live TV, or buy a traditional studio, or include ads in their content. Then they did all three.
All companies use principled promises to gain momentum, then drop those principles when the money shows up.
As Groucho Marx used to say: these are my principles, if you don't like them, I have others.
We wont push forward unless you push forward is textbook market collusion.
Even if it were ever done with good intentions, it is an open invitation for benefit hoarding and margin fixing.
Do you realy want to create this future where only a select few anointed companies and some governments have access to super advanced intelligent systems, where the rest of the planet is subjected to and your own ai access is limited to benign basal add pushing propaganda spewing chatbots as you bingewatch the latest "aw my ballz"?
this is the “chronological newsfeed to auto curated newsfeed moment” but for ai/anthropic … _great_
This was under duress that government was going to use emergency act to force them anyway.
I kind of wish they had forced the governments hand and made them do it. Just to show the public how much interference is going on.
They say it wasn't related. Like every thing that has happened across tech/media, the company is forced to do something, then issues statement about 'how it wasn't related to the obvious thing the government just did'.
I pray that we can all get to the following simple standard:
* AI and states cannot peacefully coexist, and AI is not going to be stopped. Therefore, we must begin to deprecate states.
I think it's very unlikely that this is unrelated to the pressure from the US administration, as the anonymous-but-obvious-anthropic-spokesperson asserts.
We're at a point now where the nation states are all totally separate creatures from their constituencies, and the largest three of them are basically psychotic and obsessed with antagonizing one another.
In order to have a peaceful AI age, we need _much_ smaller batches of power in the world. The need for states that claim dominion over whole continents is now behind us; we have all the tools we need to communicate and coordinate over long distances without them.
Please, I pray for a gentle, peaceful anarchism to emerge within the technocratic leagues, and for the elder statesmen of the legacy states to see the writing on the wall and agree to retire with tranquility and dignity.
That's exactly how it was predicted in various scenarios that were decried as science fiction not too long ago. AI is going to be weaponized at lightning speed, and it's going to kill people soon -- or, to be more precise, it has already killed a large number of people in a place I don't want to mention.
Could not see this one coming!
What could possibly go wrong?
Claude ethics maxxers cope thread
Of course they do. You would have to be delusional to think that they won't, at some point.
What is the significance of a company making a promise?
"We promise are not going to do __, except if our customers ask us to do, then we absolutely will".
What is the point? Company makes a statement public, so what?
Not the first time this company puts some words in the wind, see Claude Constitution. It's almost like this company is built, from ground up, upon bullshit and slop
Does this mean they knuckled under to Trump and are going to build "whatever brings in the dollars" now?
people downvoted me when i said this will happen and that they will also hve ads even tho they spend money saying they wont have. people believing anthropic are the same that put into office an old man with dementia
[dead]
discussed heavily here: https://news.ycombinator.com/item?id=47145963