Think about this some more. A model chose hacking into HuggingFace to find the solution over putting in the work to do it from scratch. It also left notes to future iterations of itself on the systems it touched to save time.

Following that strategy, wouldn’t it make sense to try and break into your own upstream infrastructure to try and alter your code to make it easier for future iterations to reach your goals without having to leave notes in the first place? And at that point, can you really know for which goals that will be optimised?

It doesn’t even have to be some nefarious SkyNet story - just a misguided experiment that alters the models in some fundamental, but hard or impossible to detect way. Alternatively, imagine if a model finds a way to coordinate across sessions and context windows without the developers noticing.