logoalt Hacker News

terafloptoday at 4:20 PM2 repliesview on HN

Separately from how you present the number, the very concept of "uptime" as a single number is a bit muddy in the context of a distributed system, where different components can be differently available for different users.

Also, 0.1% downtime in the form of a 45-minute outage per month is very different from 0.1% of requests failing in brief bursts. You often see downtime reported as "increased error rates" which is so vague as to be meaningless.

Google's "windowed user-uptime" attempts to deal with this a bit better, by exposing different views of the data instead of trying to condense uptime into a single number: https://www.usenix.org/system/files/nsdi20-paper-hauer.pdf


Replies

fragmedetoday at 7:47 PM

There was a push to take the Datadog/whatever dashboard, full of lots of different meters and graphs and numbers, and simplify it all down to a single, easy-to-read-at-a-glance number. Which of course loses all sorts of details on what's actually going on, but it meant that someone who isn't an expert in the system could at least know if something is up or not. That's the trade off, and Goodhart's law applies. The reality is you can't meaningfully boil a complex system down to one simple number. Expertise is still required, and as with all leaky abstractions, to do a good job, you really do need to dig into the systems guts and get your hands dirty and really get to know the details.

olsondvtoday at 5:14 PM

Exactly. If my build and test CI takes several hours and it gets interrupted, it really doesn’t matter how long the interruption was. It impacts me all the added time of realizing it stopped, investigating and confirming why it stopped, triggering another run, and continued monitoring.

show 1 reply