alt Hacker NewsIs there not a way to use a transformer that is deterministic in rounds as it is just maths? I thought it was temperature which randomized results but perhaps it is also something else.
Of course, the content returned is non-deterministic.
Is there not a way to use a transformer that is deterministic in rounds as it is just maths? I thought it was temperature which randomized results but perhaps it is also something else.
Of course, the content returned is non-deterministic.
If you specify the random seed and set temperature to zero, you'll always get the same response. As you said, it's just math.
That doesn't mean the response is correct though. You've just hard-locked the model to one possible response selection.
If you and I do this but we use different seeds, we still have no way to tell whose response will be better for every question or even a class of questions.
There will be responses where the model is outright wrong and we want to change seed or introduce some randomness by raising temperature, in the hope the model answers correctly next time.
Now we're back to running with a random seed and high temperature.