alt Hacker NewsIt’s available on demand from a few cloud providers. Seems like the cheapest is AMD Developer Cloud (https://www.amd.com/en/developer/resources/cloud-access/amd-...) powered by Digital Ocean at $1.99/hour.
Edit: Now I think about it, this might be the cheapest way to run the DeepSeek V4 Flash 0731 on a dedicated inference server at original weights. I haven’t run mixed load benchmarks but I guess it’s possible to generate $3-$4 worth of tokens per hour and still maintain a usable per-user throughput.
Replies
WASDx • today at 11:17 AM
At 830tok/s * 1 hour that's almost 3M tokens which is just $0.54 worth of tokens at Deepseeks current output price.
➕ show 4 replies
langs • today at 11:33 AM
You need to optimize the KVCache part(save to disk to save compute) to achieve this goal.
If you have 2x DGX Spark it will run quite nicely. They cost only $8000 or so and use less power so you may be able to rent them cheaper than the MI300X.
I found an offer to rent two at $1.65 per hour https://spark.enverge.ai/#pricing
The MI300X will vastly outperform it for only a slightly higher price.