Local AI already is really useful. This has been true since Qwen Flash Next and Qwen 3.8 27B rolled out. The struggle with exactly 64GB of unified RAM is that you end up having to stream the engram table from SSD because you will need almost all of your 64GB to run QFN. A 27B dense model still runs a little slow, even on an M1 Max Mac. However if you have even an older gaming GPU with 12-16GB of RAM and enough system RAM you can run this model (QFN) at a very respectable token rate and intelligence level. It is a Sonnet 5 class model you can run at home. There are new quantization techniques that make these models very good at agentic tasks.

And now if you stretch you can run DeepSeek Flash 4.1.

And if you were willing to roll up your sleeves you could build your own AI lab at home and run this model and models like GLM Flash 5.3 and DeepSeek Flash V4.1 at lower quants. The budget GPUs (like V100) are kind of not budget now, but still <5K to acquire 128GB of fast VRAM that can compete with a 5090 and actually has a lot more interesting options available IMO.

That's only part of the issue - prefill is horrible prior to M5 class chips.

So it depends on what you're doing. Using it for a fun oracle without feeding it too much context per turn? Sure, an older Max chip is great.

As a coding agent doing meaningful work on a sizable project? Compared to a cheap as chips cloud option, it will take an order of magnitude more time to complete.

Puh i don't know, there might be plenty of new/better chips coming out for AI next year or in 2 years.

Isn't it also rumored that the M7 is reworked for AI?

And Xiaomi announced their AI Cube with 1.2t of bandwidth. Just because Nvidia is protecting its business by giving consumers either compute or bandwidth and Apple being something in between, doesn't mean we can't get both a lot better balanced.

I'm running qwen 3.8 flash next with strata (q3 quant) on laptop with 5090 and 128G RAM and OMP harness. At 90t/s speed it is pretty much good enough already.

Apple killed M6 Pro, M6 Ultra and M6 Max to focus on AI chip, so yeah that'd be a more sensible option.

Will local AI eventually pull the proverbial rug out from under the big providers and the huge sums invested in building out data centers?

>The generation doesn't mater as much to me as the memory bandwidth

I don't know man?

The generation kind of does matter. I'm not gonna lie, matrix multiplication units speed things up considerably. That hardware confers a material change in capability.

Maybe from M5 up, it doesn't matter? But you're definitely going to notice the difference between M1 and M5 for instance.

> I'd rather have a M1 Max 64GB over a M5 Pro 64GB.

I am one of those: bought the M1 Max with 64gb thinking in running VMs, afterward bought a lighter Air with less memory and ended up using the M1 Max as a kind of luxury for LLMs and multicore computing.

Considering the insane memory price surge it's going to take years (or an eternity) before most people will consider upspecing their machines to run local over paying $20 per month to have the honor to loan a slice of Altman's hardware.

The only hope we have is the AI bubble bursting, but that will undoubtedly come with its own set of macroeconomic surprises.

Still on my M1 Max 64GB from release date. Love it. I don't do local AI but this laptop runs so cool and fast I have no reason to upgrade. They really knocked it out of the park on the design.

It feels just as fast as day 1 and I've never had any speed issues. Say what you want about Apple and their pricing and ecosystem but I've been using their hardware for over 20 years now and it's always been top notch.

If you buy their support/AppleCare, your support and warranty is excellent too. They don't skimp. I've seen them go so far past the extra mile it's unreal.

..really useful on a Mac. FTFY