It sounds impressive at first but after you think it becomes mundane.
OpenAI, Anthropic, etc are locked in a race to create better models. They all have interpretability teams. Of course they train their models on every technique under the sun to understand and improve their models, including raw weights. The primary function of these models is to build their successor.
Of course they try train their models on these techniques. Finding success is still news.
> every technique under the sun to understand and improve their models, including raw weights.
What does this even mean?
Why would they train the models they release with this though?
That's like saying the moon landing wasn't impressive because the US were in a space race with the soviets.