This is absolutely not the case in my experience. I am building a very large embedded domain specific language for describing distributed systems. It looks like a small subset of Elixir, but with object-oriented syntax in a lot of places. (It’s called a choreography; there exist many other choreographic programming languages.)

Even though this programming language is absolutely nowhere in any large language model’s training set, they have so far done extremely well at extrapolating from the small set of examples I’ve given it when I need an agent to generate some tests or whatever for me.