logoalt Hacker News

theanonymousoneyesterday at 7:04 PM2 repliesview on HN

That's my questions as well. DeepSeek v4 0731 is served dirt cheap and it needs 10 times more RAM.


Replies

kmike84yesterday at 7:09 PM

DeepSeek needs more RAM for weights, Qwen requires more compute.

Also, DeepSeek's KV cache requires less RAM than Qwen's. In concurrent situations (on servers) you load model weights once, but you have different context in each parallel session. So, it can also need less RAM than Qwen to serve, even if it's a larger model.

petuyesterday at 7:28 PM

> and it needs 10 times more RAM.

More like 3-6.

Qwen 27B full quality is FP16. So 54GB. In practice most inference providers would serve FP8, so 27GB.

DeepSeek V4 Flash in full quality is mostly FP4. ~167GB official release.

So Deepseek has 140GB model size overhead... which is shared between 100s of users single inference node serves, so not even a gigabyte of VRAM per user.

Memory required for 200K of context per user:

  V4 Flash: 1GB. 
  Qwen 27B: 13GB.