logoalt Hacker News

octoberfranklintoday at 3:40 AM0 repliesview on HN

No, that's not how a Transformer works.

It gets the entire input all at once. Then it generates the output one token at a time.