alt Hacker NewsDeepSeek just open-sourced its inference engine
Comments
That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.
That's strategic sabotage to say the least.
Ban all open weight models! Cried the VCs.
Supporting Huawei Ascend 950 NPUs with open source inference code has huge implications in the global AI hardware market. Imagine if all smaller labs now can build on top of this stack and add increasing support for more compatible hardware , software and models. This could seriously challenge NVidias death grip. Although inference is still only a part of all AI workloads and open sourcing training on Huawei chips would be even bigger.
20 merge requests in the last two hours. And all look somewhat reasonable on first glance.
I am happy for not maintaining a relevant open-source repo.
Relevant: Deepseek reports its Revenue per GPU per hour at off peak prices (38$) [reddit link] https://www.reddit.com/r/DeepSeek/comments/1xanexq/deepseek_...
Also cool (from the translated screenshot since original is in Chineese https://zhuanlan.zhihu.com/p/2095560295930508953)
> the DeepSeek V4.1 Flash inference system has established three goals
> 1-..... 2-..... 3. Supports retaining KVCache for more than 72 hours
This is awesome
Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.
The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).
Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
Even though their API price is cheap compared to OpenAI/Anthropic, I still can’t “afford” to use their API because the subscriptions from OAI/Anthropic provide far higher value and peace of mind—I know for sure how much I need to pay each month. I believe most developers fall into this category as well, where $20/$100/$200 from Anthropic/OAI provides more value.
This raises the question: why doesn’t DeepSeek provide a subscription service as well? Is it a better move to have a single pricing model like they currently do?
Let’s hope this is just the beginning of bringing the drama queens (Dario and Sam) closer to planet Earth.
Curious how the performance compares to vLLM or SGLang!
They say it was inspired by both so I am wondering what the performance gain is.
This month I changed my Claude Max Subscription to DeepSeek API. V4.1 Flash is fast and perfect for everything and refuses to almost nothing.
Hope DeepSeek starts month plans soon.
For comparison, rental prices seem to be ~$2.75/ per H-100-hour [1]. Meanwhile, Deepseek says "38 USD per hour per card”. They haven’t specified which card as far as I can tell.
[1] https://www.ft.com/content/a1815bc5-b5d5-47ff-bbbc-e1a689929...
Supported hardware: Huawei Ascend.
Deepseek moment.
If that's the thing that makes $0.00X cached inputs possible, then this is huge!
Can somebody explain like I'm five what this exactly is? How are providers like Fireworks or Novita providing inference today? Do they use some other inference engine software? If yes, what is the advantage of the DeepSeek inference engine?
I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
They open source it because they don't need it anymore. Probably they are doing something better with Huawei
Deepseek released just enough to stir the pot but not enough to cook your own from scratch.
Wake me up when we can clone, point it at a dozen H100s we own, and get a working production service.
Anthropic and OpenAI will do just fine.
[flagged]
A nuclear bomb
"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"
Source: https://zhuanlan.zhihu.com/p/2095560295930508953