alt Hacker News> If you set it to 0 it makes the model deterministic.
No, it doesn't. It can help make the model more deterministic, but it does not guarantee it.
> If you set it to 0 it makes the model deterministic.
No, it doesn't. It can help make the model more deterministic, but it does not guarantee it.
The hardware can also add nondeterminism. GPUs reorder operations, leading to different results.
Vendors might also be running A/B testing or who knows what, even when you ask for a temperature of 0.
But, if you run a fixed model with temperature 0 on your local CPU, it will be deterministic (unless there are bugs).