alt Hacker NewsNot a SWE, but I do Freetoken (backend) + unsloth (front) to run qwen 3.6 35B A3B NVFP4 (~20GB) in 8GB VRAM + ~20GB RAM. Getting around 50 t/s on a 3070. I don't think I can even share through this site.
Not a SWE, but I do Freetoken (backend) + unsloth (front) to run qwen 3.6 35B A3B NVFP4 (~20GB) in 8GB VRAM + ~20GB RAM. Getting around 50 t/s on a 3070. I don't think I can even share through this site.
If not coding what do you use it for? Also what kind of performance do you tend to get out of it? I've been thinking about a similar setup but am skeptical it's worth the effort.