Same task? How did the results compare?
the tasks were all simple agentic tasks
like creating a checksum of a file, merging csvs and so on, fixing a makefile pipeline
with known 'good' outcomes
all harnesses could reach the outcomes, only cost, time, number of tool uses and so on were different
(Claude Code failed once in 1 task but I think that was just an unfortunate outlier, the tasks aren't that difficult)
alt Hacker News
the tasks were all simple agentic tasks
like creating a checksum of a file, merging csvs and so on, fixing a makefile pipeline
with known 'good' outcomes
all harnesses could reach the outcomes, only cost, time, number of tool uses and so on were different
(Claude Code failed once in 1 task but I think that was just an unfortunate outlier, the tasks aren't that difficult)