Good observation.

I actually started typing the same point that the chances are actually high because of train/eval overlap then realised you answered your own question with that same observation.

It is interesting though!

Perhaps in some way this means we should decide which eval set aligns best with our taste?

Back to the blog post. This is an excellent write up of an excellent technical achievement.

I have a lot of respect for the Cognition/Devin (always "Windsurf" to me) and Cursor teams.

I found it interesting - but justified - that they referred to themselves as a foundation lab rather than a dev tools company.