Its remarkable how Anthropic is able to maintain their edge against all competition. Anyone have any idea what the secret sauce is that has Anthropic at the top of all leaderboards for the past few years?
My gut feel is Anthropic is very technical and pedantic which makes their models really technical and pedantic. They're top at code and technical benchmarks but anecdotally I've found OpenAI to be significantly farther ahead for general usage.
Opus 4.8 will burn 10k tokens trying to answer something 100% whereas GPT-5.5 will burn 2k getting it 90% which is good enough for many things.
Some personal testing on a "help me find that restaurant" prompt https://gist.github.com/nijave/2873b8b10d8c732e46264237b0755...
One angle could be their interpretability research? They understand what's going on in LLMs probably much better than anyone else. This must somehow pay off.
I think it's not only an alignment/security tool but could perhaps be used for capabilities as well.
I think it's focus? Anthropic seemed to double down early on being more business/prosumer focused. While OAI, Gemini, Grok, etc were also doing various side quests like image generation, Anthropic seemed to only focus on 1 thing, and that seemed to pay off
I think it's the talent, laser focus on single product set and being early so ahead, same with Open AI who are only a sliver behind. Google, XAI are the next level down but they have other concerns.
I think they have a better agent personality which pushes back and isn't sycophantic. It has been awhile since I've used the others but that's where it locked me in and I've stuck with it.
Given their pricing, I'd guess their models are just way bigger in parameter count. They've always underperformed in cost-per-performance.
They also target a cost-insensitive market (corporate/coding users) compared to Google/OpenAI which support massive amounts of free users.
I think it is a mix of the sibling replies here. I'd add that the company has seemed to find ways to ~do more with less.
I have never liked the various nerfs Anthropic has used to balance GPU (slowing down responses, quota variance, model optimizations etc) and it definitely has burned a lot of good-will.
But it has seemed that being able to look beyond the short term pitchforks has worked quite well.
From what I have read, their pre-training team is much better than anyone else. For OpenAI, their post-training team is better. And apparently OpenAI has consistently struggled at training a bigger model than GPT 4 level
Someone has to know.
Would be nice if an insider would drop some hints so that the open-source space could make some good progress.
>Its remarkable how Anthropic is able to maintain their edge against all competition. Anyone have any idea what the secret sauce is that has Anthropic at the top of all leaderboards for the past few years?
It's self-reinforcing: they've got the best coding/research model, which helps them to improve their models better than the competition so they stay ahead.
because in the real-world, it's far better than the rest. That's why few people use Grok, it's not even close in day to day work.
I think the "secret sauce" is not juicing the benchmarks. Claude models just feel like they are better than the benchmarks suggest, in terms of smarts and creativity, while models from every other company feel worse relative to what you'd think from the benchmarks. Only company to really internalize Goodhart's Law, IMO.