From experience with Zena, this is not true at all. Opus, Fable, Gemini Flash and Pro all barely make any syntax mistakes after a little is in context, and those are caught extremely early.

The one thing I do see sometimes is that agents sometimes don't take advantage of added features, but that's partially because the Zena code base doesn't use them as much yet. I'm working on skills and linter-based suggestions to use better patterns.

This is absolutely not the case in my experience. I am building a very large embedded domain specific language for describing distributed systems. It looks like a small subset of Elixir, but with object-oriented syntax in a lot of places. (It’s called a choreography; there exist many other choreographic programming languages.)

Even though this programming language is absolutely nowhere in any large language model’s training set, they have so far done extremely well at extrapolating from the small set of examples I’ve given it when I need an agent to generate some tests or whatever for me.

I think starting with a familar typescript-like base language is a good approach to this. This should be familiar enough for LLMs for the most part as long as additional features can be explained in a succinct system promopt/skill.

This keep getting repeated. So were just stuck with whatever we have at the point of training the magic plagiarism machine?

The future is cooked

> bc agents will naturally be bad at it due to a lack of examples.

Can we please as a community stop parroting these false premises as a basis of every argument against doing anything new? It's plainly obvious to anybody that uses LLMs on a regular basis that it's not true.