logoalt Hacker News

Balinarestoday at 12:41 PM0 repliesview on HN

Isn't that exactly how draft models speed up inference, though? Validating a batch of tokens is significantly faster than generating them.