logoalt Hacker News

seizethecheese • today at 6:23 PM • 1 reply • view on HN

> a model that's more token efficient but more expensive per token can be cheaper for a given task.

True, but I'm skeptical that this is now overly used to ignore cost per token. Your comment inspired me to do a quick investigation. Looks like in my data, 92% of cost per task can be predicted by cost per token https://micjm.com/blog/cost-per-task-vs-token/


Replies

danielmarkbruce • today at 6:34 PM

Definitely. Cost per task is nice in theory, pretty absurd in practice.

➕ show 1 reply