logoalt Hacker News

theptipyesterday at 7:27 PM0 repliesview on HN

This is the right way of thinking end-to-end.

Tokenizer changes are one piece to understand for sure, but as you say, you need to evaluate $/task not $/token or #tokens/task alone.