> - Why are they stacking Fable xHigh against Carbon Med?

Because on that specific benchmark (FrontierCode 2.0), Medium is Carbon's best effort level and xhigh is Fable 5.2's best effort level. You can see that on the graph your next point is about.

And it's just that one benchmark that's at medium. "All scores show each model’s highest-available effort unless otherwise noted".

> - How did they produce such a bizarre effort-intel curve for Carbon that goes against every available benchmark?

The graph is labeled as being about FrontierCode 2.0. Those appear to be Carbon's scores on the official FrontierCode 2.0 leaderboard: https://cognition.com/blog/frontier-code-2.0

> - Why did they leave Carbon-- positioned primarily as a cyber model-- off of the single cyber benchmark comparison?

> - Why do they switch to comparing against Argon (rather than carbon) for robodojo?

Because no numbers are available for Carbon for those benchmarks, and the model isn't publicly available so they couldn't run the benchmarks themselves?

And from your followup:

> where's Astra on that CVE chart

To run Astra on a vulnerability finding eval, you'd need to be part of OpenAI's verified cybersecurity program. Otherwise the results would be tainted by refusals. Do you expect that OpenAI would grant such access in this case?

Because Carbon is not out, nobody can run tests on it.

yes. this.

Just to follow-up-- artificial Analysis shows no justification for that Carbon effort curve, but it does show Carbon on medium smashing Fable on things like Terminal-Bench and AA-Omniscience v1.1 Non-Hallucination Rate.

But I suppose the reality is rather obvious when you note the funny games they're playing with the graphs, and the things they won't talk about (where's Astra on that CVE chart?)