alt Hacker NewsHm, that’s doubtful. I think even one instance of DS 4.1 Flash alone at ~500B parameters needs a full node (8 GPUs).
Replies
ytxm • yesterday at 9:45 PM
[dead]
Hm, that’s doubtful. I think even one instance of DS 4.1 Flash alone at ~500B parameters needs a full node (8 GPUs).
[dead]
4xh200 seem to be enough if you cpu offload the engrams (which is pretty typical and even expected).[1]
And these are quite old cards by now. But yeah "all of OpenRouters traffic" is gonna take more than a rack
https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4.1-Flash?hard...