logoalt Hacker News

anon373839today at 1:05 PM0 repliesview on HN

> albeit at a lower tok/s rate

Not always! I get 80-100 tok/s from Qwen 3.6 35B-A3B on a MacBook Pro thanks to MTP. With long contexts that dips to around 50-60. However, prefill is much slower than API models. So it becomes really, really, really critical to not have cache misses.