alt Hacker NewsOur position on open-weights models
Comments
> Anthropic has never advocated for a ban on open-weights models.
What are the legal ramifications of this statement if it turns out Anthropic have lobbied for this? Does it just get swept under the rug? I can't say this is bullshit (that would be defamatory) but I am intensely skeptical.
> China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips.
This is playing to readers' biases; isn't DeepSeek V4 Pro deployed on Huawei Ascend already? The old "Chinese can only copy" meme is getting pretty tired these days.
> All sufficiently capable models, open and closed, should go through mandatory safety testing
Applying such standards in the US means that US defenders are blocked from using the models, but attackers from other countries aren't. That is clearly counterproductive.
It's already been pointed out quite eloquently elsewhere that there is no such thing as a safety filter because the LLM and external filters can't actually identify malicious use. They can only identify the weaker implication "if the user is malicious, this is bad."
>Open-weights models that don’t have dangerous capabilities are a public good
For convenient definitions of "dangerous"
We need a seti@home for open weight models
“By virtue of being the good people everything we do is good, and if it happens to be in our best personal and financial interest then that is good too because it enables us to do more good. And if it’s to the detriment to others then it’s because you don’t understand the good behind it. Which is fine, because we’re good and you can trust that what we do is good because what we do is good by virtue of us being the good ones.”
A lot of people have a lot of concerns with AI, its capabilities and with how it is used now and what it will lead to in the future. For good reason. But the question is, do we have a strategy that is actually useful? If so, what is it? I think nature shows us some answers in ecosystems.
Diverse ecosystems can absorb shocks. Diverse ecosystems are a sign of health of that ecosystem. When an invasive species comes into a healthy, diverse, ecosystem it doesn't mean that it isn't disrupted, but it does mean that it is far more likely to emerge with a lot of its diversity intact. In fact, it is likely to emerge even stronger because it can absorb that new shock and incorporate it, adding to its diversity. The balance may be changed, but the ecosystem survives or even thrives.
Nature also likes to show us that artificial barriers rarely last. You want to control a river? Good luck. It take constant maintenance to hold that flow in place and even then you are likely to get extremes that are made worse by your efforts because, eventually, somewhere in the system fails in a way you didn't anticipate. Then the water comes rushing in. Artificial barriers often have a way of building up tension over time, not reducing it, so that when a failure eventually happens it can be catastrophic. In other words, you had better really understand the system you are trying to control or else you can make things actively worse.
Relating this to the world now means, I think, that our best chance to minimize long term shock and maximize the chance that the diversity we have around us survives is to try to grow as healthy of an ecosystem as we can as quickly as possible. Lots of models large and small in lots of different hands is, I think, a better solution than artificial barriers restricting the variety and diversity of models and users. I think this is closer to an ecosystem solution and has a shot at working. Basically, I highly doubt we understand this situation enough to do a good job of controlling it with artificial barriers. Instead I think we are more likely to build catastrophic imbalances than we are to create the healthy ecosystem we really need.
Anthropic starting to sound like Microsoft from the 1990s.
This reads almost dishonestly.
> Anthropic has never advocated for a ban on open-weights models.
This is not an unqualified never. The very next sentence makes a qualified statement: "Open-weights models that don’t have dangerous capabilities are a public good". That prompts the question, what about ones which do have "dangerous capabilities"? Are they not a public good? If not, then should they be banned? Who gets to decide on the definitions of these terms?
If nothing else, Anthropic has explicitly acknowledged the threat to its moat and that the margin is thin between its models and open-weight models. I'd take that as a win for open-weight models.
Good luck preventing distillation and limiting the supply of accelerators to China when Jensen himself is a strong opponent of any such barriers [1].
Talk about a giant whiff.
I was hoping they would announce their first open weights model, perhaps an older model they don’t offer anymore, but no. Instead he get this bs statement that reeks of “dam it I’m so close to being a billionaire” desperation. Not even acknowledgement of how much data they stole from others yet he whines about distilling.
It’s like his goal in life is to be a Scooby-Doo villain.
"needs safety eval"
also anthropic
"we're upset were not being considered for military contracts"
come on, which is it? Is it all about saftey or is it that only US/Israeli ai is allowed to kill? Seems to me that the only real threat is to the techno fudalism OAi, Anthropic & co are trying to build.
I think he asked LLM to rephrase "I want open models to be banned, but sound like I don't want it but mean it somehow".
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
That is not an argument against open weight models. That's just a generic protectionist argument against any Other lab.
Open weights are the future.
Anthropic’s second biggest issue is they increasingly rent when others buy. Even if model value compresses, owning your vertical stack protects your margins upstream, and potentially downstream in applications.
But the biggest issue is this. Many hate Dario because he’s smug, he caps usage, and because OpenAI effectively ran a counter-positioning campaign to paint Anthropic as undemocratic.
Who is he really and what are his motives? None of you know, really.
How about we don't let the leading model makers make the rules? How about we make AI models that were trained on public data public goods? Let's circle back on that, kthxbye.
He wants to restrict China from access to chips. All that will do will accelerate their development of competing, sovereign hardware.
As an example take North Korea. Sanctions didn't stop them from developing nukes and delivery systems.
"My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people."
Look in the mirror.
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people
But what if it's the US that becomes authoritarian and uses AI models to perpetrate incredibly deep repression of their own people?
Woof, talk about self aggrandizing.
> We should not sell powerful chips or chipmaking equipment to China
Yes, please. We don't know whether we'd have open weight models today, had the chip-prohibition not been in place. Nor would we see the more optimized models such as DeepSeek or qwen.
We also would not see new players entering RAM market after you and your pals in Silicon Valley hoarded the entire world's hardware.
So by all means, double, no, triple down on this.
> We should crack down on industrial-scale distillation operations
And let's apply this retroactively to Anthropic too. You industrial-scale-operation-distilled all of humanity's knowledge. Let's have some of that crack down on you too.
I'm cynical enough that I would suspect all of his statements are duplicitous anyway, but my recent experiences with Claude Fable give weight to it.
I asked a question about a series of tokens - bam, denied and downgraded. There's no cyber security or public risk here, but Fable doesn't want me to learn how things work.
I asked a question about quantization in models - bam, denied and downgraded. I edit my question to make it clear I'm talking about Google's Gemma QAT models. Oh, that's fine then, and it answered the question helpfully.
Anti-competitive bullshit. I hope they fail.
Their number one concern is about the the commies perpetrating deep repression of their own people! How noble! This whole time I thought it was because they wanted more money and power. I guess we should probably ban these open weight models.
“We care about safety so powerful open models are bad” - and what of OpenAI? What if Mythos got into the wrong hands? It’s a pretty weak argument
I’m tired of Anthropic. They’re scared of everything.
Release open weight models, no guard rails, no censors, straight to the public. Let everything else sort itself out. There is nothing more powerful than an idea whose time has come.
Frankly, why would I put much weight into what Anthropic say about open weight models?
Nobody will ever see this but if you're harping on responsibility and your product was created using the world's generational output, your responsibility is to make it as free as the training was as available to protect the world from impact. Greedy schmucks.
"All sufficiently capable models, open and closed, should go through mandatory safety testing."
I love how they invoke fear of "terrorism" to justify their oppressive position.
Anthropic would love the US to do everything in this list under the guise of "safety testing":
Pretty laughable. Dario seems to be missing one fundamental point with all this gibberish - if CCP is so bad why would they release the model open-weights for everyone to examine? Actually, the letter would make much more sense if what's actually happening is reversed, i.e. they are releasing open-weights model for the public good and China is distilling their model for its evil purpose.
What Dario said is half true. The untrue half is that the ship to stop China from overtaking the US in AI has sailed long ago. While China is busy getting their frontier models trained on smuggled chips using distilled data from millions of shell accounts, the US is busy debating whether to check voter IDs, or secure borders. It is just delusional to think the US is actually capable to stop China from getting the chips they want.
This is more terrible PR for Anthropic.
Imagine chinese regime getting access to powerful models who bombed school students in Iran amusing AI !
If you gate models for safety you aren't safe. There will always be actors with ungated models. Better that we use ungated models to improve our security. It's an arms race.
Anthropic (and some others) should or will soon learn how to do less pontificating and more engineering. They are in no position to be an arbiter of things, although for sure they can and should voice an opinion. If we collectively decide AI is a dangerous tool, company making it is not the arbiter. Governments are.
It's like hearing Smith & Wesson opine on the policies.. oh, wait.
It's refreshing to see how there's almost no person in this thread who can't see the BS. All the goodwill that Anthropic could have had is basically gone. Anthropic is likely on the path of becoming the most hated company in the world.
So my question is: is this by design (they know nobody's buying this), or is Dario simply so out of touch with reality?
If it's the former, then why publish this?
I’m done reading anthropic propaganda
This is nothing but a post made by the scared CEO of a company that is now facing fierce competition from Chinese labs and losing its competitive moat, nothing more, nothing less.
If decades of fighting have failed to stop piracy, I’m sure nobody can stop China from sourcing high end chips. Unlike piracy, I’m happy that Chinese labs are releasing open-source models, so people in developing countries are no longer at the mercy of this capitalist bullshit.
"China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips"
source: Trust me bro.
There are hundreds of articles showing that China have developed their own chips and have a massive manufacturing capacity. This blog post feels like is pondering to the brain dead Fox News audience.
I think I lost some brain cells reading that copium. LLMs providing a “Permanent military advantage”.. dumb!!
FUD used to pump value
> Anthropic has never advocated for a ban on open-weights models.
Shut the fuck up
This man sees himself as your master.
Do you accept?
What nakedly self-serving bullshit.
The US doesn't have some magic wand that prevents "incredibly deep repression of their own people."
Insurrection, wars of choice, ICE, Palantir, Flock -- keep up man, we're the baddies.
I disagree with 100% of everything said in this article.
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP)....
This is why open weights win. See Linux and how it's taken over the world. Your business model will need to change eventually. Instead you're advocating trying to exterminate competition via regulation and fear mongering.
> My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks
Yawn... this is getting old.
> We should not sell powerful chips or chipmaking equipment to China
For as someone as smart as you guys, you sure lack common sense. China is just going to develop these technologies organically then and you lose 100% of control. It's already happened in reverse with things like Solar, rare earth minerals, etc. China flooded our market, destroyed our ability to produce things, now holds the keys. One thing they DIDNT do was stop trading to the US. They killed us with cheap goods.
> We should crack down on industrial-scale distillation operations.
Thats your problem, not my problem. Also, irony meter here hitting 11 about all those pirated books you stole...
> All sufficiently capable models, open and closed, should go through mandatory safety testing
Oh, fuck, no. This is a crackdown on free speech and rights of people to do whatever they want. My right to free speech means I'm allowed to write whatever computer program I want, no matter what its size is or how "sufficiently advanced" it is. Individual rights always win.
I really hope people don't believe this garbage. For a company with a great product, this is absolute nonsense.
Makes sense since ICE, Israel are so much nicer & kinder than the Chinese
I don’t feel the need to rebate any of his points, lots of people here have done it already pretty well. I’m just baffled he thought releasing this letter was a good idea smh.
Reads like fearmongering about oppressive and violent applications of AI that the Chinese government hypothetically could deploy, which the US government is already actively working on in the meantime.
Jesus Dario we get it man, you want clout for the IPO.
This constant whining from anthropic about distillation attacks continues to be rich given the amount of stolen data that went into any Claude variant.