> The simplest explanation is that 'Fable-level' doesn't mean anything; it's just hype, and there's not much difference in capability.

Couldn't be further from the truth. The models can be tested and statistically evaluated.

I ran a massive Fable max code review on my lone lisp codebase. Now that I have switched to OpenAI, I decided to run an equivalent review using Sol max and compare them. I'm keeping all data so I can thoroughly evaluate their performance in multiple areas such as correctness, rigor, performance, security, maintainability, consistency, among others.

Fable pass is 100% done and I'm around 70% done with the Sol pass. Preliminary results are already becoming clear: Sol is capable of reproducing around 70% to 90% of Fable's performance. Haven't tested open weight models but I'd wager they have the same performance as Sol if not lower.

It seems Fable is still king, I'm afraid. It's undeniable that OpenAI is providing huge value here: up to 90% Fable performance at multiple times the usage on a subscription than what Anthropic offers us is a phenomenal deal. However, if one desires the best model, to me it looks like Fable is still it.