alt Hacker NewsThat's true, but if the LLM is calling a tool to provide the base information to extrapolate, the results will not change even if the sentence structure is never the same.
Debian has an extensive toolchain to check things. Having a set of deterministic tools which feed LLM with data can be beneficial, though.
Again, I'm not defending the LLM here, just want to show how hard the problem is to begin with.
And LLM "summarizes" the tool output as a completely unrelated and at times self-contradicting text.
If there are deterministic tools involved - just return their output as is. It's usually quite readable if needs a bit of adjusting