alt Hacker NewsHow do models approach SVG generation? In one version, I imagine them actually trying to reason about them as an LLM. In another, I imagine something closer to a GAN.
Replies
akomtu • today at 1:18 AM
Image -> loose text description -> SVG. Did you notice that ChatGPT writes a verbose description of the image first?
I would be interested to hear more about that too. SVGs seem to be one of the biggest challenges- it has to write reasoned code rather than simply find averages of rasterized pixels. One thing I noticed is that none of the models chose to draw the frog in profile which would have made the jaw shape more prominent. To me that suggests the reasoning is very limited: "draw a frog" and "add feature X". The statistically average frog in the SVG training data is apparently front-facing. I tried the prompt in ChatGPT images (not SVG) and it produced a photo-realistic image of the frog in profile, showing the jaw clearly. Then when I asked it to convert the image to a cartoon vector it fell back to a generic front portrait template similar to those shown in the benchmark, nothing like the profile it had just produced.