alt Hacker NewsI haven't evaluated it yet, but I love the fact that the output is (at least claimed to be) deterministic. I can't trust an LLM to do the right thing after I deploy it to production, because their output is non-deterministic by design.
TERMy (or is it the NPC-forge) seems to be worth a try.
Replies
kouteiheika • yesterday at 7:00 PM
> because their output is non-deterministic by design.
It isn't. At least not by design, even though in practice it often can be. If you do greedy decoding (or use a preset seed) and deterministically compute everything (e.g. only use integer math) then it will be 100% always deterministic.
➕ show 1 reply
You can get determinostic output (mostly) by setting the temperature to zero. Using couple of other tricks you can get close to 100% of determinism with LLMs.