logoalt Hacker News

czhu12today at 6:17 PM12 repliesview on HN

Its remarkable how Anthropic is able to maintain their edge against all competition. Anyone have any idea what the secret sauce is that has Anthropic at the top of all leaderboards for the past few years?


Replies

levocardiatoday at 7:32 PM

I think the "secret sauce" is not juicing the benchmarks. Claude models just feel like they are better than the benchmarks suggest, in terms of smarts and creativity, while models from every other company feel worse relative to what you'd think from the benchmarks. Only company to really internalize Goodhart's Law, IMO.

show 2 replies
nijavetoday at 6:33 PM

My gut feel is Anthropic is very technical and pedantic which makes their models really technical and pedantic. They're top at code and technical benchmarks but anecdotally I've found OpenAI to be significantly farther ahead for general usage.

Opus 4.8 will burn 10k tokens trying to answer something 100% whereas GPT-5.5 will burn 2k getting it 90% which is good enough for many things.

Some personal testing on a "help me find that restaurant" prompt https://gist.github.com/nijave/2873b8b10d8c732e46264237b0755...

show 1 reply
mnickytoday at 8:19 PM

One angle could be their interpretability research? They understand what's going on in LLMs probably much better than anyone else. This must somehow pay off.

I think it's not only an alignment/security tool but could perhaps be used for capabilities as well.

hello_newmantoday at 6:27 PM

I think it's focus? Anthropic seemed to double down early on being more business/prosumer focused. While OAI, Gemini, Grok, etc were also doing various side quests like image generation, Anthropic seemed to only focus on 1 thing, and that seemed to pay off

small_modeltoday at 6:36 PM

I think it's the talent, laser focus on single product set and being early so ahead, same with Open AI who are only a sliver behind. Google, XAI are the next level down but they have other concerns.

hectdevtoday at 7:16 PM

I think they have a better agent personality which pushes back and isn't sycophantic. It has been awhile since I've used the others but that's where it locked me in and I've stuck with it.

show 1 reply
x312today at 6:25 PM

Given their pricing, I'd guess their models are just way bigger in parameter count. They've always underperformed in cost-per-performance.

They also target a cost-insensitive market (corporate/coding users) compared to Google/OpenAI which support massive amounts of free users.

bredrentoday at 6:51 PM

I think it is a mix of the sibling replies here. I'd add that the company has seemed to find ways to ~do more with less.

I have never liked the various nerfs Anthropic has used to balance GPU (slowing down responses, quota variance, model optimizations etc) and it definitely has burned a lot of good-will.

But it has seemed that being able to look beyond the short term pitchforks has worked quite well.

Handy-Mantoday at 6:27 PM

From what I have read, their pre-training team is much better than anyone else. For OpenAI, their post-training team is better. And apparently OpenAI has consistently struggled at training a bigger model than GPT 4 level

show 1 reply
nullbiotoday at 6:29 PM

Someone has to know.

Would be nice if an insider would drop some hints so that the open-source space could make some good progress.

show 1 reply
logicchainstoday at 8:32 PM

>Its remarkable how Anthropic is able to maintain their edge against all competition. Anyone have any idea what the secret sauce is that has Anthropic at the top of all leaderboards for the past few years?

It's self-reinforcing: they've got the best coding/research model, which helps them to improve their models better than the competition so they stay ahead.

hn1986today at 6:27 PM

because in the real-world, it's far better than the rest. That's why few people use Grok, it's not even close in day to day work.