SWE's are expensive; median salary is $133k (not counting health insurance, payroll taxes, etc). If you can shave off an hour of dev time with $40 in LLM credits, that's $26.50 cheaper than having them do it without.

I'm not entirely convinced it works out that way so far, but that's the theory.

Trying to bring down LLM costs is sort of a double-edged sword, because the dev needs to be cutting LLM costs by more than what you're paying them. If it takes them a day to bring costs down by $1 an invocation, then it takes almost 2 years to recoup the salary costs. It's worse because LLMs currently change so much I wouldn't be confident that their solution won't be broken before the 2 year period. Will we still be tool calling in 2 years, or will that be something new? Will thinking still be a thing, or will it be superceded by something else? I don't think anyone knows, even the frontier providers.