alt Hacker NewsThe non-determinism is one of the relevant features of this layer of abstraction! And one can learn to validate that the translation is being done properly. Some of the tools you have include writing extremely detailed specs, generating visualizations of the internals of the tool, or (perhaps) reading the code, though that becomes less feasible with volume.
Basically it turns out that code is full of incidental details and what you really want is to verify the important parts, while receiving a guarantee that the vast tail of incidentals is handled "reasonably."
Replies
Writing specs validates the translation is being done properly? Was this written by an LLM?
LLMs will still blithely ignore the specs and steering documents, apologize profusely for doing so after the fact, and tell you "I'll do better next time" which they might do once or twice. But after the context is cleared or a new session opened, the Dixie Flatline gets reset and it doesn't remember it screwed up, or that you told it not to.
This happened to a coworker of mine. Generally the response from one-shotted devs is a shrug of the shoulders and "wellp, them's the breaks! As long as it looks sensible from 10,000 feet up it's still a huge productivity win." But the devil, as they say, is in the details.
Writing Specs that are thousands of lines wrong without formalism, type and correctness checking is a major PITA. Wish we had a..compiler that can check our code..err I mean spec.