logoalt Hacker News

WithinReasonyesterday at 3:19 PM2 repliesview on HN

1. Many places in the world don't ever need cooling

2. If servers are distributed then downtime is distributed, you can virtually guarantee that some of the servers over the world will be online so you can get effectively 100% uptime, something that is not possible in a data center

3. To serve tokens you need very little bandwidth, it's just text in and out

4. All of this is down to the HW and the SW itself, not the building. That is, the box that's being deployed.

5. Just switch to a different server until the problem is resolved, in this model there is no urgency. You just need redundancy which you can afford with how much cheaper this would be.


Replies

kube-systemyesterday at 3:34 PM

> 1. Many places in the world don't ever need cooling

And data centers also exist in cold places. But if you put 8kw of extra heat in someone's home that previously didn't need cooling, it might need it now.

> 2. If servers are distributed then downtime is distributed, you can virtually guarantee that some of the servers over the world will be online so you can get effectively 100% uptime,

You can! But running more servers with worse uptime is less efficient and requires more capital expense than running fewer servers with better uptime.

> something that is not possible in a data center

This is not only possible, this is how the large clouds are architected. This is what availability zones are for.

> 3. To serve tokens you need very little bandwidth, it's just text in and out

bandwidth is only one of the many connectivity advantages that datacenters provide... and LLMs are a bad choice to run residentially for other reasons, particularly power density

> 4. All of this is down to the HW and the SW itself, not the building.

Absolutely not -- basically all industry data protection standards have physical security standards. At least, any of the ones that matter.

> 5. Just switch to a different server until the problem is resolved, in this model there is no urgency.

That is true, there are data centers without 24/7 access. They tend to struggle to compete, though.

> You just need redundancy which you can afford with how much cheaper this would be.

Is it? Residential power and cooling costs more -- and that's the majority of the cost to colocate servers

show 1 reply
brianakeryesterday at 4:06 PM

If you don't need to manage cooling?

Then you are likely needing to manage humidity.