Anthropic spent a lot more time editing their pre-training data with higher-quality material. For example, they used actual books for pre-training, because books are generally run through human editors for higher quality. (this is from the copyright lawsuits)
Other companies would use completely raw internet data for pre-training, which is basically trash. Anthropic's secret sauce is that they filtered and edited down their data for only the highest-quality material. You really don't need quadrillions of tokens to pre-train, as that's just random noise. A smaller-but-higher-quality data set is better in every way.
Meanwhile its obvious that companies like Google would use trash data like email and chat logs or Internet forums for pre-training. Just complete trash data.
This is a classic case of "less is more" and "garbage-in-equals-garbage-out"
I don't think it pre-training, because surely these labs all have similar data and Google should be very good at ingesting, cleaning, and filtering data.
I honestly think it just their post-training is fantastic and I think it has something to do with them being very opinionated on how they want the model to act that it basically the best at roleplaying a human. Humans are very good at completely and figuring out things that there aren't "trained" on if given basic set of intelligence. So when it working on these long tasks that might not be directly in it training data it still reason through it very well.
They got made fun of for this in the pass because they do a lot of research on this including hiring a philosopher.
This is just my best guess, because you don't see any other labs talk about how they made the model act and Anthropic is basically obsessed with it and talks about it every model release.
I think they just have better "taste" and that taste is their secret sauce.
Where do you get that information from? Or is it just speculation?
Microsoft (MSAI) showed that quality >> quantity with their Phi model a couple of years ago. It was a small model, but trained on very high quality data, and did better than much larger models.
This is incorrect.
Everyone uses books as well as internet data.
Anthropics difference is they have a huge data budget for buying RL environments for real world tasks.
I can't say what precisely, but other big companies are also doing what they can to get human data of high quality, just look for all the companies that popped like mushrooms that do ai reinforcement tasks, micro1, ethos, mercor ecc
That doesn't really make sense because as good as Opus 5.5 is it writes awful terrible, disgusting slop.
I think by now everyone training trillion parameter models is using almost every book, magazine, research paper, etc they can get.
Purely outside looking in, but yes, I'd argue you are right and higher-quality is a major part of it, both for pre- and post-training. There appears to be a far greater willingness to adopt specifically tailored training sets from third-parties, consistently update and build on them.
From simple frontend work to strategically adding virtual rulers when working with LIDAR scans to extract more accurate data, there are very clear points when Anthropic introduces a new external data set into their mix and while I have seen the same from competitors such as Google, there simply appears to be a greater tendency to stick with these once they have them.
As an example, at release it was quite apparent that Gemini 3 Pro, 3 Flash and Opus 4.5 had a similar, possible even the same frontend-focused training base bought in from a third party. But while Anthropic has had both smaller and larger improvements across each subsequent Opus release, I have not noticed such a consistent improvement with Gemini models over the last year. Maybe Anthropic simply buys updates more often, maybe they create more artificial PRs on top, who knows.
Same with the LIDAR scans, there was a point a few months ago when models started to exhibit a very systematic way of handling visual references in some few leather-crafting tasks, enabling whole new ways of working more efficiently combining existing measurements with images. Step-by-step pixel measurements, compensation for angles and poor captures, etc. Have since seen this in other models, again very much feels like a bought in data set that they are pushing as far as it goes.
Then there is 3D capability with models like Muse Wonder Full. I have not used it myself, if it ain't on API, it doesn't exist, but have seen PRs and do find the model output very parsable, despite it being terrible at any modelling task because these capabilities do not emerge from just training on vast amounts of data, they require high-quality, well tailored, step-by-step reference data of the entire process.