alt Hacker News> Time on 4 year old H100 servers costs more now than when they were new (!!)
There are several confounding factors.
We’ve seen massive inflation since then. So some growth in cost was expected.
More importantly, the current Tech industry almost always starts by selling things at a loss. The increased cost could simply be the industry choosing to not subsidize that particular service anymore.
But also, I don’t think that’s a realistic comparison. Rented out GPUs are likely not a similar use profile as compute used for training LLMs. The latter is likely closer to the cryptocurrency GPUs that are running at full tilt 24/7.
And those things physically burn out.
> Rented out GPUs are likely not a similar use profile as compute used for training LLMs. The latter is likely closer to the cryptocurrency GPUs that are running at full tilt 24/7.
This is untrue.
H100's are used for training (well were, but are now outdated because B100/B200s are much faster).
Most of the reason people rent H100s is for smaller training runs.
If you are doing inference you usually buy managed capacity at Baseten or something, and that is often priced differently (although it comes down to an extra margin on longer term H100 prices basically).
Inference utilization is often actually higher than training now because so much effort has been spent on optimizing that stack.