alt Hacker NewsDo you think you'll realistically need 3 TB RAM to run a sufficiently good model 1 to 2 years from now? I certainly don't. Considering what can already be done with 128 GB of relatively slow unified memory, imagine if efficiency improvements continue apace, the memory becomes 128 GB of HBM, the flash device becomes capable of sequential throughput matching today's DDR5, and such a system was affordable as a routine purchase for the average person.
The things in LLM “knows” are not infinite per its physical size. “Experts” don’t exist in a world where hardware capacity was a trivial issue.
in two years, yes I expect us to have much better models. Perhaps many times more efficient. But not 100 times.