alt Hacker NewsOk, so the proposed answer here is to define the spec in what is essentially code for another LLM to then interpret into different code?
I'm not sure if this does much more than a grillme skill and then poking an agent to do the work.
Ok, so the proposed answer here is to define the spec in what is essentially code for another LLM to then interpret into different code?
I'm not sure if this does much more than a grillme skill and then poking an agent to do the work.
The difference is accuracy, cost, time and more importantly consistency.
In case you noticed, a year ago, LLMs could not reliably count the number of 'R's in strawberry. Now they all do well. Guess how? Instead of training LLMs to do this, it was easier for them to write a small python script and run that. That solved the problem once and for all.
The same thing here, instead of just using LLMs and keep grilling repeatedly, there is a higher chance of getting a workable solution quickly. The best part about formal methods here is, it is self validating. It checks in seconds, what would have taken hours or even days with LLM only flow.