alt Hacker NewsOur position on open-weights models
Comments
Frankly, why would I put much weight into what Anthropic say about open weight models?
I’m tired of Anthropic. They’re scared of everything.
Release open weight models, no guard rails, no censors, straight to the public. Let everything else sort itself out. There is nothing more powerful than an idea whose time has come.
"All sufficiently capable models, open and closed, should go through mandatory safety testing."
I love how they invoke fear of "terrorism" to justify their oppressive position.
Anthropic would love the US to do everything in this list under the guise of "safety testing":
This is more terrible PR for Anthropic.
Pretty laughable. Dario seems to be missing one fundamental point with all this gibberish - if CCP is so bad why would they release the model open-weights for everyone to examine? Actually, the letter would make much more sense if what's actually happening is reversed, i.e. they are releasing open-weights model for the public good and China is distilling their model for its evil purpose.
Nobody will ever see this but if you're harping on responsibility and your product was created using the world's generational output, your responsibility is to make it as free as the training was as available to protect the world from impact. Greedy schmucks.
Anthropic (and some others) should or will soon learn how to do less pontificating and more engineering. They are in no position to be an arbiter of things, although for sure they can and should voice an opinion. If we collectively decide AI is a dangerous tool, company making it is not the arbiter. Governments are.
It's like hearing Smith & Wesson opine on the policies.. oh, wait.
It's refreshing to see how there's almost no person in this thread who can't see the BS. All the goodwill that Anthropic could have had is basically gone. Anthropic is likely on the path of becoming the most hated company in the world.
So my question is: is this by design (they know nobody's buying this), or is Dario simply so out of touch with reality?
If it's the former, then why publish this?
If you gate models for safety you aren't safe. There will always be actors with ungated models. Better that we use ungated models to improve our security. It's an arms race.
Imagine chinese regime getting access to powerful models who bombed school students in Iran amusing AI !
"China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips"
source: Trust me bro.
There are hundreds of articles showing that China have developed their own chips and have a massive manufacturing capacity. This blog post feels like is pondering to the brain dead Fox News audience.
I’m done reading anthropic propaganda
I think I lost some brain cells reading that copium. LLMs providing a “Permanent military advantage”.. dumb!!
This is nothing but a post made by the scared CEO of a company that is now facing fierce competition from Chinese labs and losing its competitive moat, nothing more, nothing less.
If decades of fighting have failed to stop piracy, I’m sure nobody can stop China from sourcing high end chips. Unlike piracy, I’m happy that Chinese labs are releasing open-source models, so people in developing countries are no longer at the mercy of this capitalist bullshit.
China is simply undercutting again, by copying and subsidising, to ensure another industry in the west fails.
Just like batteries, the automotive industry and in fact most machining.
We really ought to boycott Chinese products from stolen tech and patents from entering our markets
What nakedly self-serving bullshit.
The US doesn't have some magic wand that prevents "incredibly deep repression of their own people."
Insurrection, wars of choice, ICE, Palantir, Flock -- keep up man, we're the baddies.
This man sees himself as your master.
Do you accept?
I disagree with 100% of everything said in this article.
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP)....
This is why open weights win. See Linux and how it's taken over the world. Your business model will need to change eventually. Instead you're advocating trying to exterminate competition via regulation and fear mongering.
> My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks
Yawn... this is getting old.
> We should not sell powerful chips or chipmaking equipment to China
For as someone as smart as you guys, you sure lack common sense. China is just going to develop these technologies organically then and you lose 100% of control. It's already happened in reverse with things like Solar, rare earth minerals, etc. China flooded our market, destroyed our ability to produce things, now holds the keys. One thing they DIDNT do was stop trading to the US. They killed us with cheap goods.
> We should crack down on industrial-scale distillation operations.
Thats your problem, not my problem. Also, irony meter here hitting 11 about all those pirated books you stole...
> All sufficiently capable models, open and closed, should go through mandatory safety testing
Oh, fuck, no. This is a crackdown on free speech and rights of people to do whatever they want. My right to free speech means I'm allowed to write whatever computer program I want, no matter what its size is or how "sufficiently advanced" it is. Individual rights always win.
I really hope people don't believe this garbage. For a company with a great product, this is absolute nonsense.
Makes sense since ICE, Israel are so much nicer & kinder than the Chinese
I don’t feel the need to rebate any of his points, lots of people here have done it already pretty well. I’m just baffled he thought releasing this letter was a good idea smh.
Reads like fearmongering about oppressive and violent applications of AI that the Chinese government hypothetically could deploy, which the US government is already actively working on in the meantime.
> Anthropic has never advocated for a ban on open-weights models.
Shut the fuck up
1. We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.
Regulate others, but not us, please. And f.u. Jensen for your tweet.
2. We should crack down on industrial-scale distillation operations.
Boogeyman to still not allow Chinese models but pretend to support open-weights. Also, please ignore our distillation of research, illegally. That's different!
3. All sufficiently capable models, open and closed, should go through mandatory safety testing.
...That we author. Oh, and please ignore our own easing-of-guardrails when it comes to money: https://x.com/NoahLebovic/status/2081277517709922501
I guess they have to make some statement about this, but we don't have to care. This is like the zoo making a statement about the employment of clowns. Clowns a a circus thing, nobody asked the zoo's opinion.
As an American I’d gladly take payments from China to feed them my Claude transcripts for distillation. I’m surprised I haven’t heard of such an initiative.
It's so ironic to me the way people will say "china should not have these chips" meanwhile they manufacture like 99% of all the electronics we have in the united states.
Ironic to oppose giving AI tech to “authoritarian governments” while approvingly quoting the authoritarian-wannabe government of the US and framing that government as the good guys.
Boris here. This post does not give us good publicity so I will refrain from commenting. Please wait for another demo thread of a new feature of ours or AI appraisal blog post and I will gladly go into as much detail as I can to give us as much hype as possible!
Yikes
First thing I'm going to do with my uncensored Kimi K3 release is to engineer a virus that only targets gatekeepers in the AI field.
(obviously this is a joke)
> For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials,
If he had just left that bit out it wouldn't be so obvious that he's just clutching at straws at this point. In some twisted sense it's almost sad to see.
> where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials
if someone figures out a way to give an LLM full operational control over a virus lab, we've got a whole different set of problems than the ones Dario is describing
> Open-weights models that don’t have dangerous capabilities are a public good
This statement (and the entire post) couldn't possibly be more two-faced.
Open-weights models by definition have "dangerous capabilities" (according to Anthropic's own definitions of "dangerous", not mine), you can't bake in guardrails that can't be finetuned out.
FUD used to pump value
Microsoft Corporation, October 31, 1998
Our position on Linux
Fud, fuddly, fuddy-duddy fud ...
Wow, so much cynicism and distrust in the comments, to the level of conspiracy theory, in my opinion. Yeah, this is the company that fairly recently refused to allow the government to use its models for autonomous weapons or mass surveillance, and refused to remove its guardrails.
Am not saying we should take what Dario is saying at face value, but he already has shown by his actual actions that he can be well intentioned. There might be elements of truth to what he’s saying.
Knew Anthropic were the bad guys.
that is so long
At the end of the day it hurts U.S. consumers to not have Chinese cars mainstream in the market. We lose out on features and stagnate on innovation because we are not pushed to compete.
I can’t imagine this would be any different — banning open weight models would hurt us in the long run. The point is to beat the competition, not suppress it.
There should be no limits on open or custom models. Too often safety is a synonym for surveillance and control. It’s a natural consequence, intended or not.
I am curious… why can’t distillation be stopped?
As a side not Im not against protectionism, but it has to be across the board and the same in all industries with no excrptions. We’ve let all these industries die on the vine due to cheap cost in foreign countries. It could very well happen to ai.
I will be so happy if/when the AI bubble bursts and we don’t have to deal with Dario’s god complex anymore.
"Our position is that we want moats, just not that moat."
What Dario said is half true. The untrue half is that the ship to stop China from overtaking the US in AI has sailed long ago. While China is busy getting their frontier models trained on smuggled chips using distilled data from millions of shell accounts, the US is busy debating whether to check voter IDs, or secure borders. It is just delusional to think the US is actually capable to stop China from getting the chips they want.
I always find it interesting when people choose to so carefully stress and spell out the words "Chinese Communist Party".
It's a bit like spelling out "Barack Hussein Obama". It's a dogwhistle.
Yes yes, it's still called the Chinese Communist Party, I know.
But since we are talking about a one-party authoritarian state with a hybrid economy that underwrites much of western prosperity (including by producing a large percentage of the components of the data centres Anthropic is dependent on), that has long-since abandoned many of the salient principles that mark it out as conceptually communist rather than totalitarian, and since we're talking about a man who runs a debt-ridden business in a country where the president is seemingly shaking down a 10% share of everything profitable for the state while running an entirely arbitrary tariff regime and suddenly calling anyone remotely left-winga Communist, it's a deliberate and telling choice to spell out "Chinese Communist Party (CCP)" when he could just as easily and arguably more usefully and appropriately have written "Chinese government" or "Chinese state".
This is some ham-fisted Republican-fishing. He must really be worried Sam is Donald's favourite.
The only real surprise is he didn't illustrate it with a Silmarillion analogy.
Is Dario Amodei another version of Trump? https://www.youtube.com/watch?v=RDrfE9I8_hs
As an European I really dislike this consistent anti-Chinese narrative.
It's not China starting a war every few years, now causing a global economic fallout in Iran, it's not China threatening to annex Greenland/Canada/Panama, it's not China attacking foreign countries and kidnapping their leaders, it's not China who has been found to spy and intercept the communications and movements of its citizens and its allies and their leaders for the longest time, it's not China bombing civilians or stopping countries from obtaining basics like food, gas or oil.
I'm not saying that China is a paradise and US is bad, nor the contrary. We could make similar lists about most of the biggest countries out there.
I'm simply stating that this never ending US exceptionalism "US has to be the first and at the frontier of military, technology and this and that, but does not need to comply with the rules of the institutions it itself created" was already sickening and annoying before, but increasingly malign in the last decade and strongly accelerating as of recently.
I miss the time US CEOs were globalists and used their influence to advocate for a simpler world.
This is just another case of a business insisting their people should be in charge of the evaluation of the safety. Worked really well for Boeing right?
Also I find it incredibly difficult to hear any company in the US worry about repression of people when financial repression is happening here. I’ve been to China and seen what the services to the public are like. Meanwhile the US funds and employs AI weapons in an ongoing genocide.
These are all valid points but not strong enough to warrant banning open models IMO. This is a new reality we have to live with, with new dangers we cannot avoid. Best solution is improving the social system, lowering the incentive for people to do harm, restoring social trust, etc... This is the only way I see the US getting out of it. We really need a high trust society now. This is the only protection. We need neighbors looking after each other. There needs to be an incentive for people to do the right thing. Everyone should want to protect the system. Coercing people into not doing harm is never going to work long term IMO.
When people say "What will be left for us to work on once AI takes over", I say "Ourselves" - We have to stop looking at humans as commodities and strategic pieces and start paying attention to people as individuals, based on the content of their character. We need a society which is attuned to this, which has enough resources and time to pay attention to this. Now we are blind to people's character because it is masked by money, power and status; all of which currently have higher priority. This order of priority is determined by scarcity, which is largely artificial.
We have a dishonest system which tries to control people's behaviors through scarcity-based coercion instead of straight forward laws with clear explanations.
"Bottled-water vendor position on tap water drinking".