I have a suspicion that they are just trained on puzzles by now
It's like we've come full circle:
First people practiced L33t3cod3 problems for interviews
Then people built AIs to build software
And now the AIs are studying L33t3cod3 problems
It is RLVR, Not a puzzle, Not leetcode
There are private datasets, and 3rd party providers of these models. Fable doesn’t have a datapoint here because of its particular data retention policy. Even if you don’t trust AWS, do you think Opus on AWS is also sending the data to Anthropic? Do you have any evidence?