alt Hacker Newsequilibrium in one or two more years on the consumer/prosumer side
think Apple M6 or M7 with a currently unforeseen denser memory style, 256gb RAM
a couple inference or cache improvements on the algorithmic side, using less ram for context windows and doubling token speed again
denser open source models, packing more experts for smaller active layers
it'll still be expensive but like $8,000 - $13,000 instead of $450,000 worth of B200s
Fairly certain that model sizes and computational requirements will grow as the price for LLM compute drops.