Text being a challenge is a symptom of the bigger problem: most people have a hard time thinking spatially, and so struggle to communicate their ideas (and that's before you add on modeling vocabulary like "extrude", "chamfer", etc)

LLMs struggle because I think there's a lot of work to be done with translating colloquial speech. For example, someone might describe a creating a tube is fairly ambiguous language, even though they can see it in their head: "Draw a circle and go up 100mm, 5mm thick" as opposed to "Place a circle on the XY plane, offset the circle by 5mm, and extrude 100mm in the z-plane"

So there's OpenSCAD, which is basically programming the geometry parametrically. But... I'd liken it to generating an SVG of a pelican on a bicycle at the current levels of LLMs.

> LLMs seem to have a poor understanding of 3D space.

This is definitely my experience as well. However, in this situation it seems we are mostly working in "local" space, not "world" space wherein there are a lot of objects transformed relative to one another. There is also the massive benefit of having a fundamentally parametric representation of geometry.

I've been developing something similar around Unity, but I am not making competence in spatial domains a mandatory element. I am more interested in the LLM's ability to query scene objects, manage components, and fully own the scripting concerns behind everything.

Opus 4.5 seems to be a step above every other model in terms of creating SVGs. Before most models couldn't make something that looked half decent.

But I think this shows that these models can improve drastically on specific domains.

I think if three was some good datasets/mappings for spacial relation and CAD files -> text then a fine tune/model with this in its training data could improve the output a lot.

I assume this project is using a general LLM model with unique system prompt/context/MCP for this.

Curious what you think is the best interface for it? We thought about this ourselves and talked to some folks but it didn't seem there was a clear alternative to chat.

It's the star trek future way of interfacing with things. I don't know SOLIDWORKS at all. I'm a total noob at Fusion 360, but I've made a couple of things with it. Sketch and extrude. But what I can do is English. Using a combination of Claude and openSCAD and my knowledge of programming, I was able to make something that I could 3d print, without having to learn SOLIDWORKS. Same with Abelton for music. It's frustrating when Claude does the wrong thing, but where it shines is when you give it the skill to render the object to a png for it to look at the scad that it's generating, so it can iterate until it actually makes what you're looking for. It's the human out of the loop where leaps are being made.