alt Hacker NewsOur position on open-weights models
Comments
Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!)
The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line.
Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips so obviously they want to restrict what models are out there and more importantly who can produce new ones. Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag.
Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country.
No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".
In the first paragraph,
> Anyone who has read my past writing should know that I don’t regard such bans as a useful measure,
Later (on banning chip sales to china)
> we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.
If you truly believe that bans don't work, the same applies to hardware too.
Furthermore, Dario says later "To address these concerns, I do support the following three measures...": 1. ban chip sales to China 2. crack down on distillation 3. all capable models should go through mandatory safety testing
Just so happens that all these moves commercially benefit Anthropic. If Dario really wanted to make a point, it would land a lot better had Anthropic released a single open-weights model
I can't remember the last time (if ever) a company managed to go from golden goose to.. whatever this is.. so quickly. The permanent defensiveness in his presentation is really hard to swallow, it actively puts me off wanting to believe in or rely on their product line with Dario at the helm. I don't even understand the logic leading up to this post. Who was it even hoping to convince. Is it possible Anthropic is due an oil change?
To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be able to pull off attacks at this scale. Do you seriously want this level of capabilities to be generally available with no guardrails?
The open weight issue has a lot of difficult nuance. Biasing toward supporting openness makes sense and is a good instinct, but it's incredibly naive to be absolutely in favor of it in every circumstance without seriously thinking about its implications.
So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology.
Quis custodiet ipsos custodes?
Oh no! The evil CCP is a huge threat to world peace and goodness! Give all your money and input token data to Palantir to support a rules based world order where the good guys thrive and cleanse the earth from crooked turtle biologists.
https://www.theguardian.com/world/2026/jun/20/mona-khalil-tu...
> All sufficiently capable models, open and closed, should go through mandatory safety testing
What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now?
"Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if he won't say it.
Every single risk he identifies as a concern regarding China is exactly my concerns with the US having absolute control. Literally the exact same concerns
What Dario misses time and time again, is that people don't trust the US to create aligned AI anymore. His entire strategy rests on the assumption that the US (and their government) are exceptional.
This is clearly false to the rest of the world.
Dario, as always, is so deeply in the middle of a morass that he helped to create that he doesn't seem to understand how geopolitics currently operates. He also assumes that just because he's from the US that he is somehow automatically more trustworthy than <insert "evil" country here> is. This blog post is a political document geared towards further regulatory capture and the furtherance of major sources of revenue for his company.
I'm not convinced that he is at all interested in the social or existential effects that AI causes. He is a greedy bastard who has taken more VC money than god to do this with. He has zero moral leg to stand on, IMO. He gave that away ages ago and I wish this technique didn't work as well as it does.
Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem.
If he had wanted a weak open-weight ecosystem, he should have had Anthropic cater better to those needs. And now he's trying to ban them.
The strong momentum behind open-weight models from Chinese labs is now an unstoppable force. Instead of trying to ban it, Dario should consider a different approach: here are our cyber and bio alignment datasets and here are our RL recipes for making that alignment training work well. By openly sharing its data and code, Anthropic could help influence and shape these models before they are released, rather than treating the entire ecosystem as an enemy.
Cyber and bio alignment aren't Anthropic's competitive advantage, they are forms of risk management. There should therefore be little reason to keep this work private. If Anthropic genuinely believes these capabilities pose serious global risks, the more productive approach would be to welcome collaboration and help the broader ecosystem manage those risks better.
On refusals, the irony is that a company like Hugging Face had to use a Chinese open-weight model to fend off an illegal hacking of its platform (done by no other than OpenAI). If a company like Hugging Face can't get past the refusal gates, then everyone else doesn't stand a chance.
You can put lipstick on a pig, it'll still be a pig
"Anthropic has never advocated for a ban on open-weights models."
---
"We should crack down on industrial-scale distillation operations"
"All sufficiently capable models, open and closed, should go through mandatory safety testing"
These are in tension with advocating for open weight models. Not direct but enough that it calls into question the first statement. What is the testing criterion? How do you pass it? Is it a government body that approves a pass fail or a global body? If it is government, and boy does it seem to be, how do you disambiguate MASSIVE corporate lobbying to set up the safety testing in such a way that the boys in blue are let through and all others are barred out of safety concerns?
My concerns aside, much of the soft-points being made are non-historic
"But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true."
It doesn't mater what his opinion is. The fact is that an advanced, closed, American AI model hacked another company. The only defense was open-source AI from China. We aren't in a vacuum, we have real world examples now and these statements are counter-factual.
> We should not sell powerful chips or chipmaking equipment to China
This is so short-sighted given that the US needs China equipment for.. everything. They are part of the supply chain needed for building the machines that build these very chips.
I understand the arguments for Anthropic barrelling ahead while simultaneously advocating for pauses and regulation. I also understand how individuals can desire a pause but have good reasons to keep working at an AI org.
But if everyone thinks this way then things continue to escalate and nothing changes, waiting on a consensus that may never come. And always there is the economic incentive that pushes all players to rationalise continuing.
I wish there was more real action from the inside. When decisions get too hard to calculate you can always fall back on basic principles. If you think AI is developing too fast, stop developing it. Now you're no longer contributing. If an AI company wants a pause, pause. Set a good example. Maybe others will even follow suit, and they'll look irresponsible if they don't. Let he who chooses not to sin put his stone down first.
Seems like the maximal position he could take compatible with his expressed principles. There’s no way to allow for bioweapon and cyberweapon grade models being open weight if one doesn’t want widespread human damage.
So I cannot disagree with him on the idea. It’s only a matter of degree and whether we’re already there or not. I have $50k in GPUs that incentivizes me to believe we are not.
The concerns are legitimate but the proposals are nothing more than a stopgap solution.
If US wants to maintain engineering superiority, we needs to invest in it -- education, research and infrastructure. Bring in top researchers across the globe and not make it harder.
China is building infrastructure for the future generations and investing in growth sectors while the US is cutting of university grants and spending billions on a war without clear path to resolution.
Surprisingly incoherent for Anthropic and Dario (cue peanut gallery — “always has been!” No, I don’t think so. I think this is new).
It seems to me like there is just no good answer to how one could possibly stop open weight models from being used for nefarious purposes. How are you going to enforce guardrails on open source? The only way is to turn the USA into a 1984-type totalitarian surveillance state (even more so than it is). Unable to say that, we just get this floundering instead. How long is not giving them chips going to slow them down? Until we RSI? Then what? Just because RSI runs off the exponential doesn’t mean that the eventual open-weight Moonshot Mythos won’t be able to make bioweapons. Genuinely what is the endgame.
Hate to break it to you Dario but the way things stand right now the world at large trusts the Chinese establishment a lot more than the American one.
"open weight" models are not open source. They are still deeply proprietary. It is not possible to know what they do or what they are capable of without interrogating them since we have no access to their source materials.
The only difference is the Chinese labs have allowed 3rd party inference providers run the proprietary models for them since they cannot do it themselves due to domestic GPU compute constraints.
Is there a market for distillation as a service? I see Google just added one for Gemini: https://docs.cloud.google.com/gemini-enterprise-agent-platfo...
The distillation commentary is really crazy coming from a company that stole all it's training material.
Wow, this comment thread clearly shows that at least Anthropic has not been a great communicator.
If one reads this with a charitable lens, Dario is simply saying that 1) Nation state actors are a threat which needs to be combatted by chip bans and distillation prevention and 2) open-weight models can pose biological risk.
One may or may not agree with item 1 but item 2 above should have broad support given the unknown unknowns in play?
Of course, the CCP with access to extremely powerful AI models would be a tremendous risk.
The NSA and the CIA with the same models, on the other hand, would use them exclusively for the good of the common man.
“Anthropic has never advocated for a ban on open-weights models.
Open-weights models that don’t have dangerous capabilities are a public good…”
A bit confused on this part, what model doesn’t have dangerous capabilities?
> In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression.
Aren't Anthropic models used in project maven: https://en.wikipedia.org/wiki/Project_Maven ?
I know it's unpopular, or unfashionable, but I agree with this letter.
LLMs are becoming so powerful that they are dangerous. We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle.
It is absolutely ok to have open weight models at the level of GPT-OSS-100B. That one was released one year ago, and I think it's still a strong one. GLM 5.2 is a whole new level, but it appears to still be safe. Maybe Kimi K3 will be ok too. But beyond that, things will start being dicey.
It's easy to dismiss this and claim that Dario Amodei is just looking to fatten his pockets. And, sure, if Anthropic manages to put the brakes on open weight models, that reduces the competitive pressure it feels. But that does not make what Amodei's argument incorrect.
Demand #1 is standard political nonsense
Demand #2 is hypocritical ladder pulling
Demand #3 is contrary to freedom of speech
so they can clarify however they like, their position is still a stinker
"bad actors are unlikely to be legitimate US businesses"
I think he lives in a different word than me
I'm tired of being strung along on these silly narratives. I can't wait for open-weight models to be deployed around the world just so people like Dario will shut up about the mystical levels of power these models have.
Distillation has to be way more energy efficient and beneficial for the planet. But if they can figure out a way to ban it, go ahead, that’s not a regulation problem, it’s an Anthropic problem.
The danger of an authoritarian government having some AI is muted by everyone else having that same capable open model. The only authoritarians to fear are those that keep models private. What kind of chance did Estonia have it having their own AI model at the level of Fable without China donating Kimi to the world?
Crack down on distillation, just for Chinese companies or is it ok for Chinese/US companies to distil? I find it hard to take Anthropic/OpenAI on distillation, because the way see it they started with "distillation" of another kind. They used all the content out there without consent of the creators and its still happening. Model distillation is just a different layer of abstraction, but same thing more or less.
> the risk that authoritarian governments [...] build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
This is rich coming from a guy who signed deals with an authoritarian government that's in the midst of launching an unprecedented surveillance apparatus (hello flock, hi p4l4nt1r), having already deployed, nation-wide, an exorbitantly funded army of unaccountable shock troops under the guise of immigration enforcement.
The call is coming from inside the house, at 130dB, and your ears should be bleeding at this point.
What this document suggests is a way to fast-track Chinese development of advanced silicon, as far as I can see. Does Anthropic really believe all the silicon is made in the USA? I thought Taiwan and Korea did most of the really heavy lifting.
Finally, some sense. This is the only argument I have seen that genuinely engages with the problem and approaches it with humility, rather than charging ahead on the basis of assumptions and without a shred of evidence. OpenAI should have been the one making it.
“Questions like this should be answered empirically through rigorous pre-release testing, not assumed in advance.”
Exactly.
> Anthropic has never advocated for a ban on open-weights models.
Not even Anthropic's own Claude believes that.
If this is about safety, am I being too naive & idealistic to think that a "Kamar-Taj" rule would solve some safety issues?
The "Kamar-Taj" rule is, no knowledge is forbidden, only certain practices. If a model gives you detailed instructions on how to kill all humans, the knowledge itself isn't the problem. The problem is the person who acts on it.
Not that I expected him to, but I note no acknowledgement that it took a non-locked-down open weight model (GLM) to stop the Hugging Face attack.
I'm less concerned that the attack was caused by a closed model, than I am that no closed model was willing to stop it.
The worst part is I'm confident Fable would have done a better job stopping the attack, but their 'guardrails' made it decide not to want to.
Unless of course, you pay up: "Anthropic GTM people used large comitted spend contracts as a prereq for lowering safeguards"
-Noah Lebovic, former Anthropic staff
Guys. Guys, you got it all wrong. We don't want to ban open-weight models!
We just want to ban the competition guys! Very different.
--
The ridiculous anthropic/openai strategy of selling shovels at a loss in a gold rush isn't going to play out, and the hilarious thing is that these AI companies are going to create tons of value and _capture none of it_.
Their only path to profitability is if they get to capture it and they're going to do everything to do so. Put it this way: *all the blog posts that Anthropic and OpenAI are putting out are DESIGNED to scare you so that you let them capture the market*.
...and "distillation attacks" (hilarious framing of "saving the output of our models")... Whatever.
So your concern is safety, and you claim you are the only one that can give us safety but do so by keeping your product closed? Then how about you release the weights?
I think it's only fair to introduce this if you're willing to have a real skin in the game, otherwise that's just weakness disguised as principle.
> We should crack down on industrial-scale distillation operations.
Also Anthropic:
AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo
> > All sufficiently capable models, open and closed, should go through mandatory safety testing.
The problem with this is the cycles required to abliterate a model is significantly less than the cycles required to train a model.
This is the biggest reason why I'm against locking these models down / preventing their use. It's just delaying things by ~3-6mo, while in the process preventing legitimate use and adding red tape overhead.
I don't really see the link between use of AI and military superiority.
My current understanding is a lot of current US military problems are due to rare earths supply chains.
I don't see how AI would either help or hurt with that.
> Open-weights models that don’t have dangerous capabilities are a public good.
Who decides what is dangerous and what isn’t? Lawmakers usually have the say but Anthropic can easily bribe… I mean lobby them to favor your viewpoint.
China doesn't seem to think that powerful open weight models are a serious threat to them. Otherwise they wouldn't release them. Those models could also be used by their enemies against China.
I'm not a fan of the Chinese political system, but they usually think things through, and do smart things for their benefit.
This is nothing but a post made by the scared CEO of a company that is now facing fierce competition from Chinese labs and losing its competitive moat, nothing more, nothing less.
If decades of fighting have failed to stop piracy, I’m sure nobody can stop China from sourcing high end chips. Unlike piracy, I’m happy that Chinese labs are releasing open-source models, so people in developing countries are no longer at the mercy of this capitalist bullshit.
I wish we had a good LLM developer toolset that is not Anthropic or OpenAI with sensible business and ethical practices and good performance.
Can't wait for local on machine LLMs that are on par with Opus/Fable.
It's so convenient that the US government already classified China as "authoritarian". If they hadn't, Dario would have to say “it’s a risk that other people build models more powerful than us”. I have to wonder what his response would be if for instance a lab in France or Germany came out with an open-weight model this good.
> Anthropic has never advocated for a ban on open-weights models.
> All sufficiently capable models, open and closed, should go through mandatory safety testing.
Yeah, this is anthropic advocating for a ban on open weight models.
Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate.
This is exactly how the US has banned goods in the past, by requiring a stamp and then refusing to issue it.