logoalt Hacker News

bayindirhtoday at 1:59 PM1 replyview on HN

Because the dynamics have shifted enormously inside the rack.

10 years ago, I was running 4 CPU servers with 48 cores and 128GB of RAM in 2U enclosures with a maximum power consumption of 500W or so. I was able to stick ~20 of them in a 42U rack, totaling 10kW.

A data center full of these can be cooled with CRACs and hot/cold aisles without much problem. This is still too much for a bog-standard server colocation operation, but for HPC, that was normal and manageable.

Now, a ~1U server houses 4 SOTA NVIDIA GPUs, 64 cores, magnitudes more RAM. This server alone uses ~3KW of power. This means you go anywhere between 30kW to 50kW per rack, and you have many racks.

Of course this means more power comes in, more heat comes out. This means more sophisticated infrastructure: bigger and beefier primary and secondary power systems, beefier cooling, more heat, more noise, in short "more of everything".

Of course when you cram this much energy and heat into a relatively small space, its effect on the environment will be much more pronounced.

Facebook's previous SOTA datacenter used water infused, HEPA filtered free flowing air accross the datacenter. Now, it's server level direct liquid cooling with extensive water treatment and oversight on coolant parameters.

Compare this having a hand warmer vs. coal ember in your hand. The latter needs a much more elaborate setup to prevent it burning you badly.


Replies

butvacuumtoday at 2:38 PM

Why are you implying all datacenters are GPU farms? You can't retrofit that kind of power density into existing buildings.

You can stuff GPU servers into existing buildings- but even with significant upgrades you end up with a lot of empty space on the floor that can't be used.

show 1 reply