As a counterpoint, continuing the human engineer analogy, we've likely all worked with individuals that seem incapable of doing the most basic problem solving on their own. In a way, they're being efficient by asking an expert that can resolve their problem much faster than they can on their own, but it is a net loss in productivity for the team. 'Let me Google that for you' is a satirical example.

So, I'm sure there's value in rewarding agent behavior that solves blockers whenever possible without human intervention. For the kind of cybersecurity exploit work they're doing, it may not be known to the human designing the task what is in or out of scope for the agents to explore on their own. Additionally, the HF incident reported that these agents had their guardrails intentionally disabled and agents were left unattended with minimal oversight.

I'm not defending OAI's behavior or role in this hack. The legal concept of negligence perfectly applies to their lack of responsible oversight. Similar to allowing a child easy access to a firearm or not controlling a dangerous dog that independently runs off and bites someone.