logoalt Hacker News

jacquesmtoday at 3:15 PM3 repliesview on HN

If you're paying per token then there is a big business incentive for the counterparty to burn tokens as much as possible.


Replies

dboontoday at 4:07 PM

Making a few pennies more from inference is not even on the radar of the labs making frontier models. The financial stakes are so much higher than that for them.

lkbmtoday at 4:23 PM

If I'll pay to get a fixed result, sure. I'd expect a Jevons paradox effect: if LLMs got me results twice as fast for the same cost, I'm going to use it more and end up paying more in total.

Maximizing the utility of your product for users is usually the winning strategy.

lxgrtoday at 3:20 PM

As long as there's no moat (and arguably current LLM inference APIs are far from having one), it arguably doesn't really matter what users pay by.

The only thing I care about are whether the answer helps me out and how much I paid for it, whether it took the model a million tokens or one to get to it.