alt Hacker NewsI want to propose a called AI Index, if you can do say 1000t/s on processing and 50t/s on generation on a min 25B model you can call your device AI9 and you have AI7 and AI5 and AI3 with 3 being 100t/s processing and 10t/s generation on a min 10B model. Values above are on a 100k context size minimum. OFFLINE.
So any effing claim of being an AI device doesn't end on the demo of best buy laptop where you remove a dog from a photo....
Or that you have "better noise removal" or "you look better on the cam" because that's called noise cancelling and instagram filter and it's being done for ages...
Replies
argee • yesterday at 10:23 PM
Can't you just look at memory bandwidth for generation?
➕ show 2 replies
Doesn't this depend on the model and quant as well vs. the hardware and its capability? Plus memory bandwidth, amount, and layout.
My Strix Halo now can run some models through the NPU and its roughly as fast as its GPU+CPU (on some of the models - some its slower)