alt Hacker News> Power is only a small part, single digit, it will eventually matter but does not really today.
Sorta yes, sorta no.
A single 5090 consumes 450W - at Californian energy prices of $0.38 per kWh that's $0.17 per hour. And the card itself costs $4100 on amazon. So after 2.75 years running at full power 24/7 you'll have spent more on electricity than on the card. I would have thought most data centres being built today would have a design life longer than 3 years.
Of course you can throttle the cards to ~300W without losing too much performance. But also you need more than a single 24GB card to run most modern LLMs.
Replies
Anyone seriously building out AI infrastructure I presume is paying nowhere near $.38/kWh which is extortionate. Utility scale solar is closer to $.02-.03/kWh, then maybe around ~$.10/kWh for natural gas peaker plants.
GPU design life is commonly cited as about 3 years. The main driver is not physical degradation of the cards but the expectation they'll become obsolete with newer cards.
Also add electricity 50% on top for cooling the DC.
DC has wider margins than the 5090. by someone elses rough numbers (https://www.spheron.network/blog/gb300-nvl72-vs-gb200-nvl72-...) for GB300 rack:
> 132kW
> ~$3.7-4M
So about 300x the 5090's power but 1000x the price. Roughly 9 years for electricity to exceed price at $0.38 and datacenters will show up in areas with cheaper power than CA.