logoalt Hacker News

hypercube33 • yesterday at 10:54 PM • 0 replies • view on HN

Doesn't this depend on the model and quant as well vs. the hardware and its capability? Plus memory bandwidth, amount, and layout.

My Strix Halo now can run some models through the NPU and its roughly as fast as its GPU+CPU (on some of the models - some its slower)