What a non-answer. And there should be perfect latency hiding, and there should always be predictable power-of-2 workloads, and all weights should always be homogenous, etc.
It's a reasonable answer. Consumer chips are idle 95% of the time, so idle power draw matters. AI datacenters are currently under capacity, so rather than optimizing for idle power draw, just have work to do.
It's a reasonable answer. Consumer chips are idle 95% of the time, so idle power draw matters. AI datacenters are currently under capacity, so rather than optimizing for idle power draw, just have work to do.