logoalt Hacker News

simonw • today at 6:16 PM • 20 replies • view on HN

Pelicans for 6.2 Sol: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

And for 6.2 Astra: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

Here's an updated comparison grid for the entire GPT-6 family: https://static.simonwillison.net/static/2026/openai-pelicans...

UPDATE: And if you're bored of pelicans, here's what I got for:

> Make a three js animation where yonkers stands up as a big mech (cool stand up motion) and stomps around manhattan and queens and Brooklyn, use openstreetmap mapping data

6.2 Astra (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...

6.2 Sol (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...

For comparison, here's Fable 5.2 Medium: https://tools.simonwillison.net/yonkers-rises and Gemini 4 Carbon: https://static.simonwillison.net/static/2026/yonkers-mech-st...


Replies

vopi • today at 6:23 PM

This might be in other pelicans, but 6.2 Sol/Astra is the first time I'm noticing a model shade the leg further in the frame darker to mimic depth (or a shadow).

It's also interesting you can clearly tell 6.2 Astra does a much better job of the above. 6.2 Sol attempts it at higher reasoning levels, but it's crude in comparison.

I wonder if it comes from the (seemingly) recent inclusion of 3D environments in training data.

glenjamin • today at 7:15 PM

Interestingly, on the mech thing it seems like only Fable really nailed the brief - the others might look cooler, but Yonkers and the mech aren't directly connected in the same way.

ryanschaefer • today at 7:11 PM

I think this is the first time I’d call the pelican benchmark solved. All levels of effort look good.

➕ show 3 replies
RGS1811 • today at 6:35 PM

The emergence of the beach setting and the scarf is interesting.

➕ show 1 reply
fekunde • today at 7:35 PM

Wow, surprisingly Gemini 4 Carbon really nailed the Yonkers prompt. I like how the sections of Yonkers held shape in the mech. Fable tried but gemini is more detailed.

➕ show 1 reply
okrad • today at 8:04 PM

Looks like high/xhigh both get the back leg z-index incorrect relative to the back wheel of the bike and frame.

boutell • today at 7:19 PM

Fable gets at the mech Yonkers idea in a much more entertaining way than the others.

➕ show 1 reply
globular-toast • today at 6:47 PM

Do we know why they always end with a scarf?

➕ show 3 replies
boutell • today at 7:15 PM

Interesting that pelican quality does not go up monotonically within Astra. Honestly, I think low was the best one.

adverbly • today at 6:26 PM

What's with everyone saying these Pelicans are great?

These are some of the worst regressions that I've seen in a while.

Pay close attention to the stacking layers around the feet, pedals (sometimes there aren't even two pedals), wheel, and two legs. Some pretty obvious mistakes in most if not all of these...

Edit: And what's up with the single arm on the two highest Astra images?

edit2: really? I'm getting downvoted because people don't like that I pointed out a mistake in a Pelican svg? Is that where we're at?

➕ show 2 replies
clzerv • today at 8:05 PM

The Fable5.2 version is the only one that allows user interaction with the model, it's also very clean and minimal. I've been using Sol-5.1-high for a dumb web project and it tends to create really baroque code and UIs. I haven't used Claude in a while and this makes me want to try it again.

It seems like the models have a kind of built in aesthetic sense and I wonder what part of the training influences this. Maybe if more RL time was spent on artistic tasks the models could not only make better visuals but also code.

Currently they all seem to care little about code quality and maintainability (by humans), just functionality, which is fine I guess if you can expect to vibe code forever. But what if there is a critical bug and the models are temporarily unavailable for some reason? It would be hopeless to expect a human to wade through the giant bowl of clanker soup and fix it anytime soon.

dllu • today at 7:30 PM

Pelicans still not saturated. No chain or front fork offset, and Astra added a rather disconnected detail on top of the pelican's head. Still, we've come a long way since the early days. Very exciting for the future of pelican cyclists!

BrokenCogs • today at 7:21 PM

No idea what a yonkers is but that fable 5.2 animation blows the gpt 6.2 models out of the water

tintor • today at 6:21 PM

Sol 6.2 max is still failing on pelican benchmark. Left leg is jammed in chain.

Astra 6.2 max did a similar mistake on the left bicycle pedal.

➕ show 1 reply
BoldColdHold • today at 6:23 PM

Wow the yellow is leaking into their llms too.

firemelt • today at 6:21 PM

any index link for all ur pelicans i want to see fable and opus

➕ show 1 reply
I_Write_It • today at 6:24 PM

They all seem very good, with a lot of attention in detail ! Thx for the Pelicans!

p11p • today at 6:19 PM

Are you getting any signal out of this anymore? All I can see is "good pelican" and "dogshit pelican". These ones look pretty good.

➕ show 2 replies
Hacktrick • today at 6:19 PM

I'm not sure if this is a useful benchmark any longer

➕ show 1 reply
EugeneOZ • today at 6:20 PM

still the best benchmark! thanks