So using this harness five qwen3.8-27b models roughly matched Fable 5 on a saturated coding benchmark from early 2025. No comparison to astra/6.1 sol/fable 5.1/opus 5.5. No comparison on a benchmark that reflects actual agentic work (and agentic behavior is what modern models optimize for anyway - probably makes qwen look worse than it is?). And also obviously using a very old benchmark like this:

> The updated release of the dataset with problems released between May 2023 and Apr 2025 containing 1055 problems

means essentially guarenteed contamination.

I have no idea if this harness is any good. Agressive orchestration could be interesting given how hard models are RLed on orchestration? But the methods used to evaluate this harness are not informative