alt Hacker NewsIf you use a deterministic sampling strategy for the next token (e.g., always output the token with the highest probability) then a traditional LLM should be deterministic on the same hardware/software stack.
Replies
roywiggins • yesterday at 5:02 AM
Deterministic is one thing, but stable to small perturbations in the input is another.
➕ show 1 reply
Wouldn't seeding the RNG used to pick the next token be more configurable? How would changing the hardware/other software make a difference to what comes out of the model?