alt Hacker NewsMaybe this will be simpler for Anthropic to understand if they take their own high-minded philosophical nonsense and ego out of it and consider it the way a neutral party would.
Suppose a company calls themselves The Doomsday Device Company. They make and sell excellent-quality doomsday devices. They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated.
The Doomsday Device Company then says they have the world's best doomsday device. (They don't, but they claim they do.)
The US Government hates the Doosmday Device Company for various political reasons, but also has a vested interest in there not being a massive proliferation of doomsday devices.
The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!" (though it probably isn't) and "Everyone can use it!" (for a lot of money)
It is completely logical, then, for the US Government to say: No, everyone cannot use your doomsday device, because doomsday is bad. (While also meaning: Only we should be able to use it, and you shouldn't be able to tell us how.)
If you do not want to be in the business of having your doomsday devices shut down by the government, well, it would help if you didn't so loudly and aggressively proclaim how doomy they are. It doesn't matter how trustworthy you claim to be, given that your business is making evil doosmday devices. You still won't be trusted!
Replies
When I read this comment I don't get "Neutral Party" from it. I'm finding it hard to decide whether to focus on the loaded characterisations or (my best guess at) the underlying substance.
To address the point I think you (and the article) are making; there's a difference between advocating that a scoped, due-process-protected power exist and endorsing any given exercise of it. This point is made in Anthropic's original statement, but it's seemingly been missed by everyone taking a position against them.
>As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles. [1]
But with respect to your specific post:
>high-minded philosophical nonsense
Taking a sophisticated approach to a problem will generally improve the outcome. I suppose your point is that it can be difficult to deliniate sophisticated & good-faith reasoning from self-serving rationalisation. Much of what I've seen from Anthropic supports the former. Perhaps others could do more to build up the case for the latter.
You can look at things like the circumstances that the company was founded in, the lack of political cozying with the current admin, the lack of controversies or disgruntled ex-employees, them taking a stand against the DoD (against their business interests), positions they've taken on various issues including data centres in gulf states.
>Suppose a company calls themselves The Doomsday Device Company
Anthropic hasn't. What's the explanatory purpose of adding this to the analogy?
>They make and sell excellent-quality doomsday devices | They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated.
Anthropic invests a reasonable amount into assuring the safety of the products they sell. No-one is claiming they sell doomsday devices. Concerns of doom relate to future iterations of the underlying technology.
>The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!"
They have what's essentially an automated vulnerability-discovering machine. They've done their best to be open about the very real implications of general availability of a system like this. Using the word "doomy" positions them as childish and unsophisticated for doing this, when it's actually the reasonable and responsible thing to do.
It's quite a bit more complicated than that. The popular narrative is not exactly on Anthropic, as the general public is far more aware of OpenAI than Anthropic. The narrative is on AI and whether everything we know about society is going to change.
Also, as far as priorities and worries go, for most people cybersecurity is way down the list.
Well, regulating doomsday devices is a reasonable thing to want. A reasonable regulation of such devices would call for proper safeguards and safety testing. I think Anthropic would have been fine with that.
Instead what happened is a one-off nationalist decree that solves none of the two concerns.
Yes this is also why if you're building a startup in the nuclear power space (empirically demonstrated Doomsday Device), then you can expect USG to come in and apply arbitrary, opaque, and unexplained rules on you, steal your assets, and destroy your business. And also probably not do that to any of your competitors who are doing the exact same thing as you.
Okay, but hasn't OpenAI been doing the same thing for years? They seem to be on slightly better terms though...
David Sacks corroborates this: https://twitter.com/DavidSacks/status/2065853007619588171
Shame on a company for sticking to their values, I guess.
The dichotomy between Anthropic and OpenAI's treatment honestly couldn't be more obvious. OpenAI has also asked for increased AI regulation, and they've also released GPT 5.5 Cyber which is claimed to have the same vulnerability-finding abilities as Mythos. OpenAI received no such notices like Anthropic. OpenAI also received a government contract, while Anthropic was banned from DoD use.
Regardless of your thoughts about Dario or his company, this treatment is obviously not based in any rational principle, and pretending it is would be stupid.
It's only a matter of months before the open source models achieve this same capability. What is the US government going to do then? Ban all people in the world from accessing the Chinese models? If you think about these arguments for more than five minutes they really do fall flat.
How does this look if it's a perpetual motion machine instead of a doomsday device?
>(They don't, but they claim they do.)
I hate to shill them, but wasn't mythos/Fable SOTA?
Your main point still stands without this aside
>It is completely logical, then, for the US Government to say: No
Not sure about that one given that the US government just reversed a ban on the exports on the very chips to China that enable said technologies, you don't hear so much about the chip wars any more.
I think entrepreneurs largely approached this administration with the attitude that if you're running Doomsday Incorporated they aren't going to say, "no, don't export that", but "hell yeah baby, how do we get a 25% cut on every sale?" because that's quite literally what they did on the hardware front.
I mean I have no strong opinion on whether Antrophics statement are true or smart, but the idea that it was regulated because someone in this administration thought it posed incalculable risks is a bit funny, i"m pretty sure they wear that as a logo printed on their t-shirts, it seems to be the sole guiding principle of their foreign policies. Palantir has successfully used doom-mongering as an advertisement strategy for a decade or so
Another neutral party might not believe it's really a doomsday device and that what currently looks like exponential growth in capability could be an s-curve that plateaus in a year or two. After that, it will be diminishing returns to invest heavily into a tech that won't get much better.
So what are the current leaders in the field supposed to do to stave off competition? They should convince the public that they do have a doomsday device, claim it must be regulated, and then they can profit from their duopoly because it's exceedingly expensive to break into the high end of the market. The government has its own nefarious incentives, not limited to collecting fees and using the unrestricted versions for surveillance or black hat stuff.