logoalt Hacker News

maxbondyesterday at 5:45 AM0 repliesview on HN

You can use a BPE variant like SentencePiece to identify these patterns rather than hard coding them.