logoalt Hacker News

nobankailast Wednesday at 6:15 PM1 replyview on HN

Apple and Nvidia have arguably already diverged at the important forks in the road. Nvidia believes in complex GPU/streaming multiprocessor stacks - Apple relies on specialized hardware. Apple used to support a more complex software stack that could enable them to compete with Nvidia, but abandoned it shortly before the crypto craze to focus on NPU hardware.

...and then NPUs sorta did nothing. They run a few tiny models, maybe, but for any "serious" inference tasks Apple will automatically prioritize your 10x more powerful GPU hardware. Oftentimes the GPU is more efficient too, depending on the task.

So now Apple has a choice to make. They can either attempt to scale-up the NPU hardware and leave it on-device as dark silicon 99% of the time, or they can renovate their GPU hardware to support complex GPGPU operations and axe the NPU altogether. Right now it seems like Nvidia has the right idea, Apple just needs to find out how to scale it down as well as they can.


Replies

benttlast Wednesday at 11:17 PM

Their private cloud for AI reads on the surface like the answer. If they can establish privacy then they can push the work to the edges.

Again, I'm not saying the are. They're the only ones that could, though.

show 1 reply