alt Hacker News> Can't imagine a single task in my workflow that would benefit from AI.
You don't do anything involving realtime image, video, or sound processing? You don't want ML-powered denoising and other enhancements for your webcam, live captions/transcription for video, OCR allowing you to select and copy text out of any image, object and face recognition for your photo library enabling semantic search? I can agree that local LLMs aren't for everybody—especially the kind of models you can fit on a consumer machine that isn't very high-end—but NPUs aren't really meant for LLMs, anyways, and there are still other kinds of ML tasks.
> It's similar to performance/effiency cores. I don't need power efficiency and I'd actually buy CPU that doesn't make that distinction.
Do you insist that your CPU cores must be completely homogeneous? AMD, Intel, Qualcomm and Apple are all making at least some processors where the smaller CPU cores aren't optimized for power efficiency so much as maximizing total multi-core throughput with the available die area. It's a pretty straightforward consequence of Amdahl's Law that only a few of your CPU cores need the absolute highest single-thread performance, and if you have the option of replacing the rest with a significantly larger number of smaller cores that individually have most of the performance of the larger cores, you'll come out ahead.
Replies
I provide a data point
* You don't do anything involving realtime image, video, or sound processing?
I don't
* You don't want ML-powered denoising and other enhancements for your webcam,
Maybe, but I don't care. The webcam is good or bad as it stands.
* live captions/transcription for video,
YouTube has them. I don't need it for live calls.
* OCR allowing you to select and copy text out of any image,
Maybe
* object and face recognition for your photo library enabling semantic search?
Maybe but I think that most people have their photo library on a cloud service that does AI in the cloud. My photos are on a SSD attached to a little single board ARM machine at home, so no AI.
What I would like to be able to do is running the latest Sonnet locally.
In general I think that every provider will do their best to centralize AI in their servers, much like Adobe did for their suite and Microsoft did for Office so local AI will be marginal, maybe OCR, maybe not even blurring the room behind my back (the server could do it.)
There are alternatives to Adobe and Office, because I don't care about more than the very basic functionality: running Gimp and Libreoffice on my laptop costs zero. How much would it cost to run Sonnet with the same performances of Claude's free tier? Start with a new machine and add every piece of hardware. I bet that's not a trivial amount of money.
Is everyone a content creator these days?
Besides, most of what you mentioned doesn't run on NPU anyway. They are usually standard GPU workload.
> You don't do anything involving realtime image, video, or sound processing?
Nothing that's not already hardware accelerated by the GPU or trivial to do on CPU.
> You don't want ML-powered denoising and other enhancements for your webcam
Not really.
> live captions/transcription for video
Not really, since they're always bad. Maybe if it's really good, but I haven't seen that yet.
> OCR allowing you to select and copy text out of any image
Yet to see this implemented well, but it would be a nice QOL feature, but not one I'd care all that much about being absent.
> object and face recognition for your photo library enabling semantic search?
Maybe for my old vacation photos, but that's a solid 'eh'. Nice to have, wouldn't care if it wasn't there.