It always makes me laugh when people say this, because its so utterly pointless. That percentage assumes literally no other costs exist besides the direct inference cost.

Even if they quit trying to make better models today, there are a mountain of recurring costs that will never go away. Retraining the models with new data, replacing/upgrading old hardware, enormous infrastructure costs related to maintaining the actual platforms, data collection costs, payroll...

I'm not aware of a single player in the LLM space actually turning a profit, even if they're only providing inference.