alt Hacker NewsWhy is it so small, but expensive?
Open Router
Input /M $0.45
Output /M $3.20
Cache read /M $0.05
Throughput 27 tps
It would be a very nice model at 200-300 tps and if it was dirt cheap. What's the limiting factor of optimizing speed and price for inference providers?
Replies
freakynit • today at 7:06 PM
I read it somewhere recently that it's architecture does not allow serving as many concurrent requests as the deepseek models allow. Maybe that's why.
➕ show 1 reply
theanonymousone • today at 7:04 PM
That's my questions as well. DeepSeek v4 0731 is served dirt cheap and it needs 10 times more RAM.
➕ show 2 replies
thenews • today at 7:08 PM
more profits i guess, https://news.ycombinator.com/item?id=49323381
It's a dense model, so 27B active parameters to compute. Compare that to DeepSeek V4 Flash, which has only 13B active parameters (MoE).