alt Hacker NewsSix effort levels (minimal, low, medium, high, xhigh, max), so here are six pelicans (actually five, "high" didn't return a complete SVG):
https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
There's usually a reasonably strong correlation between thinking level and visual quality of the pelican. This is the first time I've seen that correlation appear to break - more reasoning tokens doesn't seem to produce an obviously better SVG.
Replies
Wouldn't that be a sign that those models were trained to optimise that test case specifically, as in, by exhibiting equal or higher performance while using less power?
Wow, minimal seems to be the best here interestingly. Only 10 reasoning tokens so basically none?
Did minimal just hand it off to something else? It seems completely out of line with the rest.
Is there a problem if I like the minimal more than the others?
Any idea why there's so much convergence on the fish in a basket idea?
Fidelity.
I like that from xhigh on up its wearing a helmet.
Incredible. How do you get these out so fast?
Pelican is the new teapot
birds at minimal and return at max
Not sure this is a good way to benchmark LLM's, no offence, but a single SVG is hardly a measure of a models capability.
[dead]
It does seem pretty insane to me that only on max setting (and after waiting 27 seconds) did you actually manage to receive an SVG where it appears that the pelican's legs are straddling the bike correctly.
Edited to make clear what it looks like to me and what I was looking for.