alt Hacker NewsLooks more and more that Apple is pushing AI inference to the edges, with this and the Mac Mini and it's very suspiciously placed power button which says "No, you don't need to shut this computer off. It has important work to do! (For everyone else)"
Will be interesting to see how Apple and Nvidia's approaches to AI compare and contrast over the coming months/years.
Replies
Apple and Nvidia have arguably already diverged at the important forks in the road. Nvidia believes in complex GPU/streaming multiprocessor stacks - Apple relies on specialized hardware. Apple used to support a more complex software stack that could enable them to compete with Nvidia, but abandoned it shortly before the crypto craze to focus on NPU hardware.
...and then NPUs sorta did nothing. They run a few tiny models, maybe, but for any "serious" inference tasks Apple will automatically prioritize your 10x more powerful GPU hardware. Oftentimes the GPU is more efficient too, depending on the task.
So now Apple has a choice to make. They can either attempt to scale-up the NPU hardware and leave it on-device as dark silicon 99% of the time, or they can renovate their GPU hardware to support complex GPGPU operations and axe the NPU altogether. Right now it seems like Nvidia has the right idea, Apple just needs to find out how to scale it down as well as they can.
> It has important work to do! (For everyone else)
Where do you get that macOS is running code like this? I've never heard of shared compute on my computer for anything other than software I deliberately installed. I haven't paid that close attention to the past couple of OS propaganda films at the start of WWDC. Did I miss something?
Are you promoting some kind of conspiracy theory that Apple is using some people's devices to do inference for other people's devices?
There's absolutely zero evidence of that, and it would be easy to observe it happening. So why are you pushing some kind of totally false narrative?
I am hoping Apple at some point in the future will allow iphone owners who also own a Mac with an M series chip to run Apple Intelligence from the Mac to take advantage of faster compute and larger parameter models while still operating at the 'edge'. If the apple private cloud tech allows for a Tailscale like connectivity between your mac and all iphone/ipads on an icloud account I could see that as an additional hook to stay inside the apple ecosystem in addition to the benefits brought to bear by having a much more capable LLM to offload compute to without involving 3rd party AI companies. Perhaps bundle the feature with iCloud+ subscription?