Been running this on a few Asus GX10 machines with k3s on top, it’s been great. I’m running the new deepseek.
Thank you for your work!
What quant are you using, and what tps are you getting with K3?
What quant are you using, and what tps are you getting with K3?