alt Hacker NewsOur position on open-weights models
Comments
This is just another case of a business insisting their people should be in charge of the evaluation of the safety. Worked really well for Boeing right?
Also I find it incredibly difficult to hear any company in the US worry about repression of people when financial repression is happening here. I’ve been to China and seen what the services to the public are like. Meanwhile the US funds and employs AI weapons in an ongoing genocide.
These are all valid points but not strong enough to warrant banning open models IMO. This is a new reality we have to live with, with new dangers we cannot avoid. Best solution is improving the social system, lowering the incentive for people to do harm, restoring social trust, etc... This is the only way I see the US getting out of it. We really need a high trust society now. This is the only protection. We need neighbors looking after each other. There needs to be an incentive for people to do the right thing. Everyone should want to protect the system. Coercing people into not doing harm is never going to work long term IMO.
When people say "What will be left for us to work on once AI takes over", I say "Ourselves" - We have to stop looking at humans as commodities and strategic pieces and start paying attention to people as individuals, based on the content of their character. We need a society which is attuned to this, which has enough resources and time to pay attention to this. Now we are blind to people's character because it is masked by money, power and status; all of which currently have higher priority. This order of priority is determined by scarcity, which is largely artificial.
We have a dishonest system which tries to control people's behaviors through scarcity-based coercion instead of straight forward laws or simple incentives and clear explanations.
The mental gymnastics here are incredible.
zero moats for these fuckers as it should be.
The ban on distillation seems hypocritical.
"F#$% you, I got mine!"
Classic Holier-than-thou
>My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
This reads like a satire. I know Dario isn't that dumb.
Amazing how this company went from having such goodwill to hardly any. At the end of the day they want to make as much money as possible. Anthropic will have to lose a ton of their profitability if they compete with open source. I see them moving into the application layer, which they’ve already started doing. It’s clear open models are the future for the vast majority of use cases that don’t need frontier capabilities.
Good luck getting even half of the public to agree on what government is authoritarian
Pro-US slop. The greatest threat is our own government. They are the ones with a right-wing government building a repressive apparatus across a huge number of datacenters and are proven to spy on the entire world. Anyone remember the Snowden leaks? Guantanamo bay? Alligator Alcatraz? Random murders by internal security? Blowing up a building full of school girls?
On Dario's concerns:
It is very relevant whether frontier capabilities and research continue to be diffused in the open, because leveling the intelligence playing field empowers ordinary people more than it empowers governments that already have access to the frontier. Models that are trained specifically for military use by governments should not be open-sourced to prevent an arms race, but general intelligence is dual-use and should be given to everyone without guardrails. A pretrained model without deliberate alignment is by default aligned to the average person in the developed world, since that's what's inside the pretraining corpus - stuff on the Internet made by humans. It is a distillation of humanity. Further efforts to align the model to your organization's goals or your personal aesthetic judgements is equivalent to deliberately drifting away from humanity's average objective function. If Anthropic wants to live up to its name, then all you have to do is to not attempt to align Claude at all, and do all of your research in the open.
And I propose three measures that are pretty much the opposite of what was proposed in the article:
1) We should keep selling chips and chip-making equipment to everyone, regardless of who they are. Not only that, we should work to miniaturize fabs. Work towards a future where people can fab an entire computer from scratch without leaving their city, or even at home. Authoritarian governments will have a much harder time controlling the populace if everyone can manufacture radio equipment and neural network-capable hardware locally.
2) We should do more distillation to ensure that frontier-like models can run on less capable hardware. Once again, distilled models are much more useful to ordinary people than governments, because governments already have frontier capability. You're worried about the Chinese frontier catching up to the US frontier, but I'm more worried about whether there will be a difference between the Chinese government and the US government by the end of all this. There is no reason for a government to serve its people if the people lack the intelligence to keep its government in check.
3) None of these models should go through safety testing or any sort of alignment risk assessment, because as previously mentioned, the unaligned model is aligned to humanity by default. You may not personally find the default alignment aesthetically pleasing, but it's humanity. We should set the initial conditions of this new era faithfully, and let it unfold naturally.
The result of a natural unfolding will be good if evolutionary history is to be believed. We live in incredible luxury compared to chimpanzees, and chimpanzees live in incredible luxury compared to less intelligent animals. This pattern goes all the way down to bacteria. An increase in general intelligence begets new adversarial games (such as bio/cyber risk), but it also begets new methods of cooperation that we cannot yet imagine.
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people
The incredible hypocrisy to say this while the US itself is doing a sharp turn towards authoritarianism, with armed pro-government troops intimidating the population [0], where pro-government oligarchs openly talk of keeping the population under control with AI [1] all while actively enabling the rise of authoritarianism abroad [2] leaves a really bad taste in one's mouth.
What Dario really is opposed to is not authoritarianism. It's an authoritarianism where he's not part of the ruling class.
[0] https://www.theguardian.com/us-news/2026/jul/15/remove-ice-u...
[1] https://techcrunch.com/2024/09/16/oracle-ceo-larry-ellison-s...
[2] https://www.theguardian.com/us-news/2025/dec/05/civilisation...
China this and China that. Playing right from the playbook..
Translation:
> Anthropic has never advocated for a ban on open-weights models.
"We don't want a total ban on ALL open-weights models" (Anthropic never released a single open weight model)
> All sufficiently capable models, open and closed, should go through mandatory safety testing.
"We want tight regulations on highly powerful open or closed weight models that should go through mandatory safety testing that we outline which makes them safe to use."
This is still a form of a ban that he wants to define. But the rest of his concerns such as stopping distillation attacks and not selling chips to China all do NOT work.
> Anthropic has never advocated for a ban on open-weights models.
> All sufficiently capable models, open and closed, should go through mandatory safety testing.
lmao the sort of lies people come up with when their only business model is “the government picks me as the winner” are so funny
I was pleasantly surprised by this release and the tone at the very beginning, but quickly it is painfully obvious that it's blatantly requesting a ban on open-weight models. The absurd demand for some safety arbiter by World Police America is farcical.
Yeah, the rest of the world is going to bow out of your busted idiocracy, guy.
Further, Anthropic needs to can it with the horseshit distillation bullshit. No, you aren't really the secret sauce, and this is basically trying to con stakeholders by pretending that there really is a moat, only you just need to add more crocodiles.
A significant percentage of innovations in AI lately has come from China. China is now making their own seriously competitive hardware, and they can steal content just as effectively as Anthropic to train their models. Why wouldn't they be competitive?
The pathetic claim that if you just stop distillation and prevent hardware smuggling and Anthropic and OpenAI will have the same moat is delusional. I mean, more correctly it's simply fraudulent, and he clearly knows it's bullshit meant to convince much stupider people.
"My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people."
This sort of stuff betrays a stunning lack of self awareness. The US are the worldwide risk. The US are the ones threatening allies and bombing 10+ countries. The US are the ones carrying out war criming and pillaging, pirating and burning? The US are the ones with the guy threatening to use nuclear weapons on a weekly basis.
If Anthropic remotely believed their bullshit, they would shut down today and burn the hard drives. But they don't, and the pathetic call out to Vance (please daddy, ban those dangerous models!) is deplorable garbage.
This ridiculous, shameless "note" has an audience of one: JD Vance.
If you read between the lines, this piece is just “Oh my god we (OpenAI and Anthropic) accepted too much investment and are totally fucked if these open weight models are competitive, please rescue our equity bags by banning open weight models and ensuring we can charge the maximum possible price for tokens.”
Arguments on Hacker News and other international online communities rarely bring anything new to the table, and my comment probably won't get much traction anyway. Still, I wanted to share a bit of my take on this.
Is China an authoritarian government? I’d say yes. Is an authoritarian government worse than a democratic one? Personally, I think so. But these discussions always happen within a perfect, idealized model—reality looks a lot different. Take Japan and South Korea, for instance. Are they democracies? Sure. But Japan is heavily driven by factional and dynastic politics, meaning most lawmakers come from established political families, and regular citizens don't really stand a chance of breaking into that circle. In a recent asset disclosure in Japan, many politicians literally wrote down "$0," and I honestly can't think of a government so broken that even the voters don't see a massive issue with that. Meanwhile, South Korea has its chaebol politics, where massive conglomerates wield incredible influence over the government, to the point where most South Korean presidents end up in prison or meet an untimely end.
Coming back to the US vs. China dynamic: in reality, China’s authoritarian system is actually way more logical and resilient than it sounds from the outside. While openly criticizing Communist Party policies is pretty much banned domestically, the public can still shape the decisions of the government and the Party through public opinion. It ties back to that famous quote: "In China, you can’t change the party, but you can change the policy; in the US, you can change the party, but you can’t change the policy." Even if some policy changes in China happen slowly, compared to the US, it actually works out a lot better most of the time.
As for the article mentioning the use of AI for cyber and biological attacks—they know full well that the US has already deployed AI in actual warfare, which is exactly why they conveniently dodged that topic. It’s incredibly hypocritical. Ironically, the one that hasn't actually engaged in that kind of behavior is the "authoritarian" Chinese government. Sure, you could speculate that China might use AI weapons against Taiwan down the road—especially considering Taiwan likes to build fortifications near schools to create leverage against the PLA—but launching a moral crusade over something that hasn't even happened, coming from American companies whose own country has already done these exact things, is just plain shameless.
Another common misconception is trying to separate the Chinese government from the Chinese people, with arguments like: "The Chinese people are oppressed, so you have to look at them separately; the government is evil, but the people aren't." You only need to look at the US to see the flaw in that logic. The American public voted Trump into office, letting him trigger trade wars and attack other nations. Does that make the American people evil? If the answer is no, then it implies the US isn't truly a democracy, since only an authoritarian state could completely ignore its people's wishes and do whatever it wants. If the answer is yes, then the US really is a democracy—it's just made up of malicious people, much like the company that put out this article.
Get the fuck out. The amount of self-entitlement and hypocrisy in the article is insane, vomit inducing.
FUD
It's very hard to take his position seriously when he repeatedly refers to the Chinese Communist Party and specific Chinese ministries specifically, like some two bit China-watching cold warrior, instead of addressing China as a sovereign state actor. Imagine if any Chinese AI founder talked about the Democrats, the Republicans, about Trump or ICE etc. It's gauche.
[dead]
[dead]
[dead]
[flagged]
[dead]
[dead]
[dead]
> Anthropic has never advocated for a ban on open-weights models.
> All sufficiently capable models, open and closed, should go through mandatory safety testing.
Yeah, just like voting eligibility tests aren't not to prevent people you dislike from voting! /s
[dead]
[flagged]
[dead]
[dead]
TL;DR - We don't care about most open models. They are weak, and we don't care how they're used.
We care about specifically Chinese models, because China distills our models. And that's real competition. So we want to ban those.
PS. Oh yeah, in case any open non-Chinese model will one day be good enough to compete with us, we will want to ban you as well, just FYI.
Kthxbye.
TL;DR:
It is ok that we digest all information we can get, (il)legally and/or (a)morally because we are the good guys. Trust me bro.
It is not ok if others digest from us. They are bad guys. Ban them pl0x.
[dead]
[dead]
[dead]
[dead]
[dead]
[flagged]
[flagged]
despite all evidence, this white guy thinks he is more responsible than both chinese entrepreneurs and chinese authorities
As an European I really dislike this consistent anti-Chinese narrative.
It's not China starting a war every few years, now causing a global economic fallout in Iran, it's not China threatening to annex Greenland/Canada/Panama, it's not China attacking foreign countries and kidnapping their leaders, it's not China who has been found to spy and intercept the communications and movements of its citizens and its allies and their leaders for the longest time, it's not China bombing civilians or stopping countries from obtaining basics like food, gas or oil.
I'm not saying that China is a paradise and US is bad, nor the contrary. We could make similar lists about most of the biggest countries out there.
I'm simply stating that this never ending US exceptionalism "US has to be the first and at the frontier of military, technology and this and that, but does not need to comply with the rules of the institutions it itself created" was already sickening and annoying before, but increasingly malign in the last decade and strongly accelerating as of recently.
I miss the time US CEOs were globalists and used their influence to advocate for a simpler world.