What happens when these companies decide to start increasing the cost/token, exceeding the cost of hiring an engineer or paying for an external service in the first place?
No, they would never...
What do you mean? They need to recover hundreds of billions in sunk costs?
Cost per intelligence-adjusted token has been in free-fall since GPT-3 landed five years ago [0]. I see no compelling reason to think we'll see that trend suddenly stop, much less reverse.
0. "The cost of achieving a given level of AI performance has fallen about 47% per quarter since 2023, or 13× per year." https://epoch.ai/publications/the-plunging-price-of-thought