I very firmly disagree. IMO image models are extremely under-used in this area.

Just for context, I ran design systems at Figma for 5 years / led their first prompt->design stream before quitting to make my own startup in this space.

Image models are trained on visual creativity, color usage, whitespace, etc - all the ingredients you need for good design principles. Where they struggle is often with information density and text rendering, which LLMs excel at. You can use the best parts of both of them though - use an image model for the initial design and an LLM to convert it to code.

One thing that's very telling is with openai's sites feature, in its chain of thought it actually renders out an image first as a design reference, then builds towards that.