alt Hacker NewsHow has it been and how’s the speed? I read online that the it’s ~200 TPS for PP and ~15 TPS for TG. Unfortunately for those speeds, it’s very very hard to use for agentic stuff.
How has it been and how’s the speed? I read online that the it’s ~200 TPS for PP and ~15 TPS for TG. Unfortunately for those speeds, it’s very very hard to use for agentic stuff.
That's pretty accurate to what I've seen, I'd definitely recommend a smaller model for active agentic use on that hardware.
It definitely seems to be the leader on 'general intelligence' on this hardware from my casual usage, but the newer Qwen or Gemma series models are much more usable speed-wise for agentic use, and often is as good or better than DS4 on that front.