alt Hacker News8800 GTX in 2006. Cutting-edge, an insanely powered consumer card for the time. Theoretically around 0.3456 TFLOPS.
1080 GTX in 2016. Cutting-edge, an insanely powerful consumer card for the time. Theoretically around 8.87 to 8.9 TFLOPS.
5090 RTX in 2026. Cutting-edge, an insanely powerful consumer card for today. Theoretically around 104.8 TFLOPS.
In the same timeframe mobile processor CPU's went from 0.001 TFLOPS, to today's Apple's A19 Pro chip which delivers 2.074 TFLOPS.
That's _without_ getting into ASIC's, or purpose-built hardware like Taalas's model on silicon HC1, or generic AI dies like what they're planning with HC2 or Cerebras, which will massively compress the timeline.
Replies
Yeah but here you describing the opposite phenomenon. You're saying that the hardware is going to become cheaper and more powerful with the years, to the point a current State of the Art model from today will run on a normal consumer hardware in ten years. What people are trying to do now is the opposite, optimize the software as much as possible so that it does not need the best hardware but the normal one we currently have. As if we were trying to make a current AAA game to run smoothly on the 1080 GTX of your example.
Ok, now do memory capacity and bandwidth - the things that actually constraint local LLMs.
Isn't there such a thing as low hanging fruit?
Aren't we already approaching theoretical physical limits? We're at 2nm
Sadly while the FLOPS are increasing nicely, total graphics memory is stalled in consumer cards by comparison.
We will see such power and price now only when AI market crashes or China reaches node parity and goes after market share as currently the way they are buying out most of the latest node production the consumer prices will only be palatable to the very rich or we will need to be happy with older slower nodes
Just a note that I think the direction most people are paying attention to is memory bandwidth; thats the real bottleneck and “number go up” but also constraint people are designing around
Except the 499$ of a 1080 GTX inflation-adjusted only buys you a 5070 or 5070 Ti even by MSRP.
Speaking of ASICs - how likely is it that as models get better we'll see someone baking a whole model directly into the silicon? It's like having l0 cache.