wizzwizz4 • yesterday at 4:10 PM

I'm not trying to act smug. Open Connect (Netflix's telco CDN system) doesn't seem like it'd take "massive DCs" to maintain: per [DG], each Open Connect Appliance (OCA) draws between 350W and 750W (depending on model) at peak use, plus cooling to keep it below 26°C (with peaks up to 40°C permitted). By its nature, the OCAs should be geographically distributed, not centralised. With the Open Connect network handling the most popular shows (i.e., the peak load), you'd only need to serve the long tail of the back-catalogue from elsewhere.

I don't know the true numbers, because I'm lazy, but [DG] says that storage OCAs can take 6.5 TB of traffic for a fill window, so we can take this as a lower bound for the storage. I'll use my earlier 1 PB overestimate for the total catalogue. Assuming that every OCA has the same data on it, this is (at least) 0.65% of the catalogue. Assuming a Zipf's law distribution of popularity for 20k titles, this means the Open Connect network can serve the top 130 titles:

  >>> range1 = lambda i: range(1, i+1)
  >>> sum(1/i for i in range1(130)) / sum(1/i for i in range1(20000))
  0.5198676299332914

which is over half of all Netflix traffic. In reality, this number is surely higher: different parts of the Open Connect network hold different titles depending on regional popularity, storage isn't 100% redundant, and so on: I wouldn't be surprised if it were closer to 70% or 80%.

Modify Netflix so you have to request an item from the back-catalogue a day in advance, or something, and the cost to serve the long tail of Netflix goes waaaay down. (Considering that Netflix used to work by sending you physical DVDs, which have more than a day's delay to them, I don't think this is unreasonable.) If you want to get costs down even further, combine this with some custom UDP-ish torrent protocol, and make it peer-to-peer among users (not just the Open Connect network).

But if you go too far in this direction, people will start asking why they're even paying Netflix in the first place. Netflix has to remain an infrastructure provider, not just a rentier, if they want to preserve their business model.

Or, pie-in-the-sky, if we implemented a worldwide pubsub multicast system, then the top 200 90-minute titles multicasted at 30-second intervals would cost Netflix the bandwidth of just 36000 simultaneous streams, and they could run all of Netflix off the hardware of 200 OCAs… which would fit in one (1) datacentre. Or I could assemble a custom rig that would fit in… ten racks, ballpark estimate, given generous application of duct tape: which probably means I've way underestimated the capacity of each OCA. (Serving unpopular titles on demand would be cheaper than sending out multicast broadcasts that nobody's subscribed to: it would actually be even cheaper than this estimate.) It's such a shame that the only internet-routeable protocols are TCP, unicast UDP, and ICMP.

[DG]: https://openconnect.netflix.com/deploymentguide.pdf