logoalt Hacker News

flatline • today at 8:39 PM • 1 reply • view on HN

I’m getting ~6.7 tok/s out of it on an a laptop RTX 4060 with 8GB VRAM — over twice the speed I can get out of qwen, and arguably fast enough to turn it loose on real work. For a minimal hardware setup, this is easily the best speed for performance I’ve found in a model. The nemotron models are the next best, they are much faster, much more verbose, and much less accurate.


Replies

6_7 • today at 8:47 PM

I'm so sorry you're getting how many tok/s?

➕ show 1 reply