logoalt Hacker News

erutoday at 5:33 AM2 repliesview on HN

No. That's wrong. LLMs don't output the highest probability taken: they do a random sampling.


Replies

storustoday at 5:40 AM

This was obviously a simplification which holds for zero temperature. Obviously top-p-sampling will add some randomness but the probability of unexpected longer sequences goes asymptotically to zero pretty quickly.

show 1 reply