alt Hacker NewsLLMs are not inherently non-deterministic during inference. I don't believe non-determinism implies lack of abstraction. Abstraction is simply hiding detail to manage complexity.
LLMs are not inherently non-deterministic during inference. I don't believe non-determinism implies lack of abstraction. Abstraction is simply hiding detail to manage complexity.
Non-determinism is configurable at the level of the mathematical model, but current production systems do not support deterministic evaluation of LLMs.