alt Hacker NewsBut power hungry.
In that 5+ year timeline, the compute per watt could change by three orders of magnitude.
GPUs are to LLMs what CPUs are to gaming — not a good fit.
But power hungry.
In that 5+ year timeline, the compute per watt could change by three orders of magnitude.
GPUs are to LLMs what CPUs are to gaming — not a good fit.
A cursory estimate courtesy of ChatGPT suggests that there is a grand total of one order of magnitude or less of power efficiency improvement available compared to current Blackwell if the entire system’s power consumption outside the ALUs went all the way to zero.
If you want three orders of magnitude improvement, you probably need to find two of those orders of magnitude somewhere else: process improvements, different ALU design, model architecture changes, etc.