alt Hacker NewsI wonder when we begin to see the dividends of all the NPU PCs come into play. AMD have been doing some good work with their NPU/iGPU hybrid inference kernels. If these larger models could be scaled down to run on NPUs, you'd see much better power advantages, compared to running them on the CPU.
Replies
throwa356262 • today at 6:48 PM
Are the NPUs really that powerful?
I was under the impression that they were primarily designed for low power use.
➕ show 1 reply
You can already run some models on the NPUs in the Rockchip RK3588 SBCs which are pretty abundant.
A claude 4.6 they are most certainly not, but if you get through the janky AF software ecosystem they can run small LLMs reasonably well with basically zero CPU/GPU usage