What if they’re not pelicanmaxxing, but svgmaxxxing in general?
Because otherwise using a LLM to generate complex svgs is pretty niche and what I thought made this a good benchmark when it was new - generalized programming and spatial knowledge.
Obviously image gen in svg format is not a particularly hard problem if tackled directly on its own.
If they're optimizing for SVG generation, then that's an excellent outcome in my opinion. Vector images shouldn't be "pretty niche".
Gemini have absolutely been SVGmaxxing. They've openly talked about it.
Then it's great. A year ago I couldn't get any AI to draw a simple company logo in SVG or even convert from raster. No doubt the Pelicans put pressure on the labs to fix the awful SVG situation. Now I can even make a decent Peli in 3D.
SVG are really useful you can create images that don't have the AI look
Funnily enough, not that niche, because I have tried many times to do it as part of a wider project.
I agree, other formats, both textual and binary should be tested.
But that's a genuine worthwhile capability. It's like benchmarkmaxxing on a weightlifting competition by getting really strong.