logoalt Hacker News

gavinsyancey • today at 4:57 PM • 0 replies • view on HN

So basically, the tokenizer is optimized for English and performs worse on other languages?