alt Hacker News> less common DSLs like liquidsoap’s stream management DSL
seems to be on github since 2008 so definitely in the training data. i am not talking about less or more common. either "your dsl" would need to look something like someone elses dsl (at this point is it your dsl?) or you need some way to get your dsls examples in the training data for the llm, or feed it in the prompt.
> LLMs respond well to clear, simple structure
and what a "clear simple structure" for a dsl is also quite not mentioned. clear and simple would be quite subjective based on the domain, the article says let the llm go in a loop trying to figure out the dsl for you.
> checkpoints that have more structure than natural language
if llm is at any point in the structured generation part then either you have a deterministic validator/compiler or you are back to reading/reviewing it manually, what can you trust ?
Replies
Re: less common, I was just saying it doesn’t take millions off examples like PlanetUML.
> what can you trust
I wasn’t clear enough here — you’re responding to DSLs as an interface from non-deterministic LLMs to deterministic external systems.
What I meant was using DSLs as intermediate checkpoints in multi-LLM processing. If you just flow natural language through 5 LLM calls, the last one may be getting something very different from what it’s prompt is designed for.
But if you make the DSL a contract for handoff, results are much more stable.
Perfect and deterministic? No, of course not. Just an improvement and mitigation. But it’s served me well.
> or you need some way to get your dsls examples in the training data for the llm, or feed it in the prompt.
I'm really not sure this is true. Recently for work I've been making changes in a system written in tcl in the early 2000s, including a custom dsl that has never escaped this company. The LLMs can write it fine. It wrote it almost acceptably on first sight, an hour or two of targeted test cases to extract a one page functional spec of the subset we needed and that was it. They are quite good at finding and fitting to patterns, go figure.
For open source llms you can give your grammar to the llm to help it decide what valid next tokens it can generate.