alt Hacker NewsWhat's different this time is that LLMs require orders of magnitude more memory than we could have ever used before. And I think it's getting obvious by now that inference will move to local compute, So everyone's laptops and phones would need a lot of memory.
Replies
> LLMs require orders of magnitude more memory than we could have ever used before
That's true. But will anyone have any money to pay for the LLMs? The AI market is currently being heavily subsidised by, ultimately, everyone else. But the economy in general is looking extremely dire at the moment, and that seems unlikely to last.
That's what people thought about a year ago, but with the amazing strides that smaller models have taken over the past year, people are now starting to question that.
> And I think it's getting obvious by now that inference will move to local compute
From the bottom of my heart, I hope that's the future.
I'm not convinced that will be the case. The AI companies don't want you to have local control. They want you to subscribe to a service that they can change at any time.
I grit my teeth when I type this, but (god help us) I think Apple is maybe the best (least bad?) hope here. They are the only big player with the hardware chops and without a current vested interest in getting you addicted to monthly AI subscriptions. I'm not saying this is highly likely... I'm just saying that out of the current major players, they're the ones with the ability and motivation to move in this direction in the near future.
The next best hope is probably just nVidia or AMD catering to the consumer market once again, after the AI bubble bursts or the datacenter market reaches saturation.
The outside hope is that some startup makes a business out of burning open weight LLMs to silicon like ChatJimmy... although ChatJimmy was bought up by AMD AFAIK.
The question the memory manufacturers have is, sure, we have all these orders out to the next three years... but will they actually be there to pay when the memory is delivered?
The memory manufacturers have been through bubbles before.
I don't deny that AI is "real", in that there is certainly something there, but I would hate to be the guy betting billions or even trillions of dollars that we're not in a demand bubble well in excess of what is justified by that tech at the moment and expanding capacity is a great idea. Of course, I would also hate to be the guy saying "no we shouldn't expand capacity" when it turns out that, yes, we should have, but the staggering profits being made in the meantime would cushion that blow pretty well.