alt Hacker NewsGenuine question, what's the goal of posting this on almost every single new model thread here on HN? I may be old and grumpy but to me it got old a while ago, and is closer to a low effort Reddit comment
Replies
Maybe this can help
https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/
This isn't even a normal pelican image post, this one created the html control system that animates the distance the wing travels from its pivot in time with the rotation of the wheel speed. Let's not pretend this is a solved problem and models are dumping about perfect pelicans on bikes one after another (or ever?).
Surely, you know someone makes the same post you did every time one is posted. Surly you see the answers and pushback since you are familiar with these posts. Genuine question, did you expect a different answer this time?
It's a great filter for people who take things far too seriously
It's tradition at this point. Based on the upvotes the comment receives, it looks like many readers find value in it.
I agree I'm of sick of this repetition. It's not even a good test it's so dumb.
I, for one, find it entertaining.
[dead]
Agreed! When I see any new model release and then this guy start running over with his stupid "hey guys look over here how this model made the pelicans-on-a-bicycle!" I mean, some are good, some are stupid and some are interesting. But that tells me exactly nothing about the model. It's just feel like this has become the Pete Davidson of the model evaluation. NO ONE CARES!
It's a lighthearted, fun, visual benchmark that's not part of the standard benchmarks; and at least traditionally, it was not something that the labs trained on so it was something of a measure of how well the intelligence of the model generalized. Part of the idea of LLMs is that they pick up general knowledge and reasoning ability, beyond any tasks that they are specifically trained for, from the vast quantity of data that they are trained on.
Of course, a while back there was a Gemini release that I believe specifically called out their ability to produce SVGs, for illustration and diagramming purposes. So it's not longer necessarily the case that the labs aren't training on generating SVGs, and in fact, there's a good chance that even if they're not doing so explicitly, the RLVR process might be generating tasks like that as there is more and more focus on frontend and design in the LLM space. So while they might not be specifically training for a pelican riding a bicycle, they may actually be training on SVG diagram quality.