alt Hacker Newsone of the points is that LLMs are NOT deterministic so you can't run them yourself a thousand times and expect the same output.
one of the points is that LLMs are NOT deterministic so you can't run them yourself a thousand times and expect the same output.
That's true, but if the LLM is calling a tool to provide the base information to extrapolate, the results will not change even if the sentence structure is never the same.
Debian has an extensive toolchain to check things. Having a set of deterministic tools which feed LLM with data can be beneficial, though.
Again, I'm not defending the LLM here, just want to show how hard the problem is to begin with.