alt Hacker NewsI had actually been thinking more about all the non-LLM functionality that go into the harnesses. I'm not going to name names and I haven't done any rigorous testing, but my general impression is that choice of harness matters more than choice of model. In terms of basic task completion success specifically, not code aesthetics.
A perfect harness will not extract gold from a dumb model. It's a system that builds on each other, though we've not probed that frontier much to have a good intuition on what effects what.