alt Hacker NewsThis take only makes sense when the required inputs are human exclusive. As soon as the abilities of the "chatbots" near that of the average human (a point that we are fast approaching if we haven't reached it already) the logic falls apart because any newly created task that a human can do can instead be automated in turn. Even if we're left with a few highly difficult tasks at the top of the pyramid by definition the vast majority of people won't be capable of performing them.
If anything, this report actually made me feel a bit better about AI-led job extinction not being that close. The sheer complexity of the swarm's actions required AI to parse and aggregate, but even with the METR team effectively having an unmetered token budget to do so, the output/summary still required extensive human review.
Even if we ignore the hinted possibility that the agents used to summarize the voluminous data may be acting deceptively (i.e. no snitching), the agents' summaries often 'missed the mark'. Maybe this is another example of 'taste', but it seems less subjective than arguments I've seen for that. It could be that an LLM is no better able to define 'usefulness' or 'relevance' to humans absent being told exactly what that is.