What's cute is that you're falling into the same trap of confusing accounting concepts.

Gross margin is revenue minus cost of revenue (COGS). COGS is basically the compute that serves inference (rented capacity, or depreciation on owned GPUs used for inference, plus expenses like support, etc.).

Model training is normally treated as R&D. Under US GAAP, R&D is expensed as it occurs and is below gross profit as an operating expense. It is not COGS.

You mention "depreciation" but depreciation applies to physical assets like GPUs. The GPU depreciation that belongs in gross margin is already in COGS. A trained model would be an intangible asset, so that is amortization, not depreciation. Even that only applies if the training cost was capitalized, and under GAAP it generally wouldn't be. The full cost is expensed when it's incurred. Nobody is "pretending models don't depreciate." Their cost is counted all at once but not in gross margin because that's simply not what gross margin measures.

Obviously, models lose value fast in an economic sense. They get replaced within a year or two and competition limits what they can earn. But that's an argument about whether the R&D expense is sustainable. It doesn't mean that gross margins are being misstated.

"Inference has high gross margins" and "training is extremely expensive" can both be true, which oddly seems to be something many people have trouble comprehending. But this is the crux of the questions that will determine whether these businesses are viable long-term:

1. Will training costs fall over time, or at least grow more slowly than revenue?

2. Does the gross profit a model earns from inference before it's replaced exceed what it cost to train?

3. Will inference margins hold up as competition and open-weight models push prices down?

You have a legitimate point about compute costs. If the deals with Google, Amazon or anyone else are priced below market, that directly inflates gross margin, because it lowers COGS. But we don't know those terms, so that's just speculation, and it's still a different argument from depreciation.

To reiterate what I wrote:

> The question is whether these companies ever get to the point where the profit from inference justifies (recoups) the investment made in training. If they have to keep investing vast sums in training new models, and/or competition causes the margins on inference to collapse, they're in trouble. But if training becomes less costly and inference margins don't compress too much, it's a totally different story.