alt Hacker NewsWhat intrigues me the most about AI progress, is not AGI or the model du jour by $AI_UNICORN, but rather what can be run locally. I remember having an amusing, but rather useless model in a beefy gaming PC that I had 6 years ago; and now, something that’s a hundred times better on my M5 laptop.
Should the market react to the memory shortage, the progress of the Apple silicon continue at the same pace, and what we’ll be able to run locally in 6 years will be very exciting. or frightening.
Also I don’t know what this means for the valuation of the AI companies. I remember asking about this very idea to one of their employees at an event and instead of answering he bailed out to grab a cocktail.
Replies
What you can run locally in consumer hardware is progressing pretty well.
If you get a not-quite-the-best gaming GPU like a 5080, you can run local models that are better than the state of the art from early 2025. Depending on what you want to do, you might have to switch models. The one size fits all huge models are still a data center thing.
Its a convenience thing. You can run a whole lot of stuff locally from wikipedia to social media/email/video servers whatever. Most people with a full time job and 2 kids dont do it cause who has time and energy to patch and maintain the ever growing complexity of this stuff. These systems will keep growing complex. That also means more bugs. Age old tradeoff between freedom and convenience.
Training AI models to drive valuation reminds me of high frequency trading
Things you are not supposed to talk about:
- There is no "moat" (lasting, easy-to-defend technological edge) in AI model businesses. There are just short-term advantages.
- An AI business is a capital-intensive business, just like old factories. Data centers are expensive, models are energy-hungry, and the hardware inside must be replaced every 3–4 years.
- Smaller, specialized models eat margins from below. Transcription, voice, or image detection do not need large models.
There is no reason to expect high margins like you can in traditional software business. Benefits of AI go mostly to consumers.
edit: There is potential for economies of scale. Few megacorps can strive for cost advantage when they achieve scale (Microsoft, Google, Amazon and Meta)