logoalt Hacker News

matthewolfeyesterday at 10:14 PM0 repliesview on HN

To echo the other replies, the tokenizer is definitely not the bottleneck. It just happens to be the first step in inference, so it's what I did first.