alt Hacker NewsAssuming the DGX priced at $100K, and GLM-5.3 at $1.4/M output token, this has ROI of ~7 years at maximum utilization (unlikely) and ignoring operational expenses like electricity. Which makes it economically unreasonable unless you want privacy or something like that.
Replies
It’s useful in controlled environments where getting a commodity api approved will take years.
Eg I can put 2x of these workstations in an office handling banking/medical/secret/sensitive data and have my engineers stop complaining.
In each similar thread there are posts disputing the ROI - most often that's a minor positive side-effect of self-hosting. Independence, privacy and unchanging, uncensored models, that makes it already worth it.
It seems like hosting your frontier models don't make any kind of economic sense at this point given the exorbitant premiums on hardware in this space right now.
What will prices look like once production has caught up with demand though?
This is a lot of faster though. That could be worth something.
It seems particularly poor compared to larger deployments of other models like Kimi K3.1 Pro.
I've heard numbers between 10k-38k tokens per second per GB300 GPU.
Assuming constant pricing for tokens from online suppliers.
If you’re doing any work near and dear to your heart, that you think is gonna have some type of significance, some type of value, if you don’t care about privacy and are not doing anything about it upfront, don’t cry on the back end.
Maybe it won't be for long, if the price inflators Huang allegedly engaged in are relaxed (financing AI startups to put capacity pressure on pricing).
I am really getting tired of this point. Yeah no shit renting something instead of buying it probably saves money in the short term. Like… a house, or taking an uber everywhere, or renting a musical instrument instead of buying one.
There is an invaluable, priceless sense of of ownership of a thing instead of renting a thing.
If we thought of everything the way you’re pointing out the min-maxing value of LLMs nobody would own anything except for the oligarchs. You’re good with that?
Where your price is from? https://docs.z.ai/guides/overview/pricing Input tokens: $1.4 Output tokens: $4.4 Cached input: $0.26
The ROI on this hardware from my back-of-envelope calculation is (50M per day for output tokens, 500M (assuming 10x ratio, which again is conservative (probably 4x more realistically)) per day for input tokens, and assuming 90% / 10% mixing (10% output, and 90% input), you are doing $22 + $630 per day), that is less than a year to make the money (2 DGX Stations, $200k) back.
EDIT: actually read the article, it is kinda poorly written with no clear separation what's the performance for decode v.s. prefill, making us arguing about this much less meaningful.