alt Hacker NewsGPT-5.6
Comments
GPT-5.6 Sol sets a new SOTA on ARC-AGI-3: 7.8%
Sol is the first verified frontier model to ever beat an ARC-AGI-3 game
Ok long time Claude Code user here; lately I've started to realize there's other great models out there I should be trying, but I'm hesitant to leave Claude Code behind for something new.
What's the consensus today on codex vs claude code, does it really matter anymore?
Funny to see that they did not include Fable 5 in their GeneBench and LifeSciBench comparisons because "it does not answer advanced biology questions and refuses the majority of questions in this eval".
Winner by default!
I love testing the new models by asking them to code a toy RTS game. Here's what Terra did: https://senko.net/vibecode-bench/2026/rts-gpt-5.6-terra.html (one try, in codex app, xhigh effort)
Comparing this to other models, I find it similar to GPT-5.5 and a bit behind Sonnet 5. You can see how other models fared here: https://senko.net/vibecode-bench/ (you can also fetch the prompt and the the 5.6 Terra resulting code on from that page).
I don't have access to Sol yet (on a Plus sub, which should get it according to what I've read), so can't do the more interesting test. I'll update the above page as soon as I get access - hopefully soon.
I really appreciate the focus on intelligence WITH token efficiency. I'd like to see that become the trend. Smartest per token metrics. Least tokens to accomplish the task above a certain success level. Most of my tasks would benefit from efficiency / token, but switching models constantly, and trying to guess the right model and effort level takes up too much of my processing.
I really wish there was just an easy guide on when to use Sol vs Terra vs Luna, and it just moves further into confusing territory when it comes to naming.
The naming convention is especially difficult to decipher depending on what your native language is. Of course a latin language speaker might be able to easily determine oh yeah each one is slightly bigger than the other but I still think it borderlines too confusing.
That aside all the numbers look amazing, and I'll be happy to probably main this alongside grok-4.5 for a while comparing the two on price and efficiency.
I vastly prefer the direction that OpenAI seems to be going with token efficiency and performance compared to Anthropic who seems to be moving towards a world where you just token-max as much as possible ignoring any and all costs.
We Openly hate OpenAI because they’re not very Open but we secretly hope they win against not-open-at-all Anthropic.
The frontier graph on all these benchmark are extremely in favor of 5.6 Sol over Fable, more than the best model comparisons in previous iterations.
I'd like to know how cherry-picked this is, and what tests it performed less overwhelmingly in, but I suppose that info is not going to be on this post.
If it pans out to be as good as it says, that's great. On the other hand, if this model is not overwhelmingly impressive over Fable, I will lose what remaining trust I had in these announcements.
"We've extended usage of Claude Fable" message incoming any day now.
Anyone else noticed the "Extended: Fable 5 is included in your weekly limit through July 12 blablabla" disappeared from claude code? Did they panic-delete the july 12th deadline ?
Here are 18 pelicans - six each for Luna, Terra and Sol at the six different reasoning effort levels (plus the price to generate each one): https://static.simonwillison.net/static/2026/gpt-5.6-pelican...
Or if you want to see some in 3D, OpenAI featured a pelican riding a tricycle, bicycle, pony and another pelican in their livestream this morning: https://www.youtube.com/live/Wq45rvPGNHs?t=1070s
I find it interesting that no one here has mentioned the increased (usable) context window 258k -> 353k. That's huge, but I wonder if it means we pay long context (2x) for the ones past 272k still.
5.6 Terra (mid tier model) as good as Fable on DeepSWE while cheaper than Opus API pricing. Seems like a homerun.
"GPT‑5.6 delivers a step change in design judgment. With only high-level direction, GPT‑5.6 creates tasteful, ergonomic, and functional interfaces. Its stronger computer-use capabilities let it inspect and refine the rendered result—not just generate the underlying code or content—so it can catch visual and functional issues and apply finishing touches before handing the work back."
This one is really promising, as it may allow to close major gap with Claude in design/UI skills
>> approximately 700,000 A100e GPU hours of black-box automated red teaming
Amusing that they use A100e as the reference point to sound impressive. Different ways you could make that conversion, but based on FP4 FLOPs (yes it's disadvantageous to A100, that's the point), that's something like 200hr on a GB300 NVL72 rack.
Not nothing either, but far less astounding sounding than 700k hrs.
Where is Gemini in all this? Lately it's not even been in the running. Sir Demis asleep at the wheel? Or Google too scared to release a SOTA model?
Or ... maybe Gemini 4 is too good and the NSA is using it to break into systems worldwide ...?
GPT-5.6 is a really good model, and quite cheap. I can finally replace GPT-5.3-Codex for my Tool Calling in n8n.
Here's my benchmark results for GPT-5.6:
https://aibenchy.com/?q=gpt-5.6
(the high reasoning variants are still running, uploading them soon too)
EDIT: The high variants are there too, enjoy the hamsters[0].
The most impressive part is the token efficiency/cost per task of 5.6 Sol, it makes Opus 4.8 and Fable look extremely bad ($1.04 vs $1.80 vs $2.75)[0].
And 5.6 Luna ($0.21) is also impressive, cheaper than GLM 5.2 ($0.37) with higher intelligence.
Things I have been struggling with Fable over and GPT 5.5, were just solved handily by SOL in a real "thank you, next problem" kind of way. Overall, something that just works is way less wasteful for your usage than struggling back and forth for hours.
Unfortunately, I'm finding that in long-form agentic use, when I'm trying to use Sol, I keep tripping guardrails – moreso than even Fable, somehow.
I don't know exactly what part of my codebase is triggering it, so I'm going to have to keep poking, but apparently the guardrails are not that gentle despite the phrasing. :(
Just my two cents. I'm on the Plus plan, I ask gpt-5.6 sol / high to analyze a vibe-coded codebase (~50k LoC) and write a plan to make it production ready. It wasn't a great prompt, I just wanted to test it quickly. It ran for ~15min and consumed 95% of my 5h quota (I thought it was gonna crash). The output is excellent but just a heads up that it consumes a lot of quota!
Wow the video is much better.. the PR spend clearly went up a lot. Mainly just showing "real people" doing "real stuff".
I flip back and forth between whoever currently has the more powerful frontier model that isn't cost prohibitive - subscriptions only, API pricing a non-starter. Today that's Fable 5 which has been excellent, as soon as it's Sol I'll switch to that. The OAI/Anthropic harness behavior has mostly stabilized for me with consistent AGENTS.md that I sync with CLAUDE.md - I like pi (pi.dev) and have tried to build it up to get performance comparable to the two "first-party" harnesses, I'm just not there yet.
One major sticking criteria for not going with OpenCode / pi for all of my coding is I want access to the tier-1 frontier model of the day without API pricing - e.g. afaik I can't use Fable 5 via pi harness even though I have a subscription, so for this week I'm on Claude Code. It's not the need to Fable 5 for everything, but even if I just want the marginal intelligence benefit to stress test an architecture decision, it's a safety blanket to know there isn't a ~smarter~ model I could have used. And for my use cases, the doggedness and capability of these frontier models has been insanely effective.
My feeling is we're still in the Uber era subsidy period - the moment the subscriptions either try to lock me in longer than a month or stop OAI/Anthropic stop delivering frontier models in the subscriptions, I'm out - switching fully over to pi.dev or another OS harness and routing my token spend via OpenRouter or offloading to Qwen locally. Then I'll have to put an accurate dollar amount on frontier intelligence.
I haven't tried an OpenAI model for a long time, but with Fable going to API pricing soon this might be enough to get me to try codex.
Huh, a good alternative just as anthropic's 50% weekly subscription subsidy is ending this weekend. Time to see if it's benchmaxxed or actually a strong leap over GPT5.5.
They also seem to really not care about alignment, or care about it in the wrong way. It's entirely missing in the blogpost and there are some concerning bits in the model card, seemingly treating CoT controllability as something to be "investigated" rather than the warning sign it's supposed to be.
Benchmarks look really promising. Suspiciously good, even. I guess we’ll see soon enough.
My question to previewers: how are the guardrails for random joe that wasn’t personally blessed by the ai pope to access the non-nerfed model? Fable is a nightmare in this regard, but I’m not sure whether 5.6 also gets a critical side-eye from the gubmint when you ask it to fix bugs in your code (you filthy hacker, you).
In the introduction video they say 5.6 Sol autonomously post-trained 5.6 Luna. Curious what this means.
I use 5.5 a ton. It's immediately apparent that 5.6 is truly a better model. Hope they don't lobotomize it later.
Very interesting: I wonder if the RL approach is diverging between Anthropic and OAI?
I noticed that Fable uses shell tools almost exclusively (even to search and edit files), compared to previous Anthropic models.
Having run some experiments with 5.6, I notice that it uses built-in file systems and provider native tools much more (not shell tools), compared to previous OAI models.
I’m interested in knowing how each of GPT 5.6’s variants fare in non-English writing/translation tasks.
GPT 5.5 has a tendency to write English calques and non-idiomatic prose in other languages. Although that can be somewhat tamed with detailed instructions and a corpus of confusing terms, the model’s output often reads like a literal translation rather than native prose. Since I notice these issues most clearly in languages I know well, it makes me reluctant to trust the model’s output in languages in which I’m less proficient.
Ironically, ChatGPT began as a simple text-generation tool, but much of its offerings and benchmarks now focus on coding and agentic workflows, while leaving behind what made it notable in the first place.
Based on the Intelligence vs. Cost graph, not clear to me why anyone would use Terra? Luna looks quite interesting though, happy to see OpenAI still serving the more budget-oriented side of the market (seems like Anthropic and Google have lost interest there).
I can't try it since it hasn't appeared in my Codex yet, but this is is necessary from OpenAI in my opinion. Fable is just so much better at understanding broad context. I only use GPT 5.5 for straight forward easy to describe tasks, and it does crush those. But I spend a lot more time steering Codex towards good design on broad concept type tasks, ones that Fable shows sometimes surprising clarity.
I look forward to seeing how it compares once I have access. Not getting tripped by spurious safe guard flags could be an advantage.
Is any of those comparisons about Pro vs non-Pro (Pro is only available in $100+ plans)? I am curious about that but I think Sol, Terra, Luna are different sizes of it without the Pro part, and I want to know how much worse do I have it on the $20 plan compared to if I upgrade.
Just used terra ultra for exactly one prompt in codex and it ate through my full 5h window in about 10mns (20$ plan). The results look pretty good though. Luckily I have had my chatGPT subscription for a while and have a bunch of resets available (nice compared to anthropic).
Assuming I take the 5x plan it would give me about an hour of active sessions with terra ultra (maybe ultra is not good value regarding tokens?), not even using Sol yet. Does everyone using codex use the 200$ plan?
I normally use the 100$ anthropic plan and barely ever reach the usage limit.
8% on ARC-AGI-3, they actually got some traction going...
Not sure what everyone's experience is but I find 5.6 Sol to be a great liar. Reported success on a half done job and left things in a broken state after having quite a few back & forth followups on the initial prompt to clarify the plan. Didn't experience this with 5.5. Opus 4.7 and below sometimes did it but they fixed it in Opus 4.8. So, overall, the initial experience has made me think that this model will be a lot more stressful to work with just because the level of trust that it actually completes the task is now much much lower.
I use both Claude and Codex, but mostly Claude for planning and coding, and Codex to review Claude’s work.
I follow a sort of waterfall workflow which is verbose but fully transparent.
Anthropic’s $100 subscription works fine for me, but whatever subscription my company has with OpenAI reaches the 5hr limit ridiculously quickly.
Not specific to OpenAI / Codex, but I'm curious what people are doing to protect themselves from any destructive actions by their coding agents? Just install and pray? Explicity approve all actions? Reconfigure for safety? Run in a sandbox (Docker) ?
Dirac (https://github.com/dirac-run/dirac, https://dirac.run/) now supports gpt-5.6. This thing does now seem to be on the chatGPT/codex accounts yet.
UPDATE: it is now available in chatGPT account also, they rolled it out
I am seeing some bugginess in testing:
Parameter: reasoning_effort
Function tools with reasoning_effort are not supported for gpt-5.6-sol in /v1/chat/completions.
To use function tools, use /v1/responses or set reasoning_effort to 'none'.'
I hope we aren't trying to push customers off the chat completion endpoint... Responses endpoint looks great on paper, but the business wants more visibility and control over the reasoning process than this product currently offers.
On the tiny voids demo: does your Firefox js thread lock up as well, when you try to interact with it?
Oh man, I love capitalism spoiling us here. I was just enjoying my extra Fable credits, now I'll switch to using 5.6 this weekend. I was planning to ration my Anthropic credits, I guess now I do not have to. And I was half wondering if exactly this would happen: right when Fable usage credits were starting to kick in for people, OAI swoops in and takes the puck. As much the AI craze is crazy, this play by play part is pretty fun.
One of my best use cases for the short duration I have fable is to use it to create the plan and acceptance test files then use GPT 5.5 Pro to do an adversarial review on the plan then feed that feedback into fable to fix the plan.
There is an issue on the page that causes the benchmark tables to get cut off. If you highlight and drag right you can see a few more models like Gemini and Claude Opus. It's also interesting that they introduced explicit caching, which is something that only Anthropic had for a long time.
> Instead of requiring developers to script every step or passing every tool response back through the model, Programmatic Tool Calling in the Responses API can filter large amounts of intermediate data, retain only what matters, and adapt its workflow along the way.
this seems very interesting
So with this release do they kill the 5.5-Pro model with super long thinking and reasoning? 5.6-Sol-Ultra is not the equivalent, right?
they update these shits too much.
The claims are pretty bold. I think 5.6 may exceed Fable.
The developer's guide (https://developers.openai.com/api/docs/guides/latest-model) has some interesting semantic tips for using the model:
> Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly.
> Original image detail: GPT-5.6 preserves the original dimensions of images sent with original or auto detail instead of resizing them to a patch budget or pixel-dimension limit.
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
> Control warmth: GPT-5.6 does not become meaningfully better when prompted to be broadly friendlier or more empathetic.