> The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one.

But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems like some do, most don't, Anthropic and Claude remains very popular among the people who use LLMs, there is no denying that.

I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.

Try running the latest OS models on a normal Mac or PC. Claude Fable and Mythos are systems not just pure models.

And of course marketing. Don't believe the hype.

I think Claude is often times underwhelming. Security concerns are also a concern companies have a blond spot for. The really toughest pro security (Yes, pro! Totally different framing!) company I know is Google after all.

What I can companies advise to do is, really having more than just bug bounties but a professional hacker team that does nothing else but attacking them the whole day and night 24/7. This needs to be coordinated with the government otherwise you might sound an alarm and will be SWATed for doing good. And I would pay them huge sums since the risk and fallout warrant such a treatment, not the standard wage.

Hackers are the real deal, not AI. Proof: Hackers using AI.

"Distillation" from APIs is not a thing, it cannot replicate a model's deep reasoning and behavior.

> no nationality controls in place

Not for now, but how long before we have KYC regulations concerning LLMs?

Do you think token completion endpoints are the final form for AI APIs?

That thesis is not about what Anthropic will achieve, but about what power they think they ought to have.

That's a different problem that what you're arguing against.

To this point, I've never understood the supposed "alignment" between the EA/AI Safety crowd and Anthropic's mission that the author comments on. Be the stewards of the Machine God, but responsibly? I think the Manhattan project, which AI development is commonly analogized to, had a lot more intrinsic properties to gate against uncontrolled proliferation (which still happened to some extent). Also this is a company that is expected to go public this year, at which point there will be a slew of new voices pushing the company to increase its value, mission be damned.

People like Yud at least have a clear consistency in their advocacy that we shouldn't be developing this at all. Anyone who thinks they can reconcile Anthropic's work with the AI safety mission is in total fantasyland, if it's not just a public persona they've adopted strategically.

The distilled versions miss the spark of the model. Its like they land in the uncanny valley of models.