logoalt Hacker News

ZiiS • yesterday at 3:17 PM • 3 replies • view on HN

- we added a bug (yes, everyone does).

- our testing didn't cover it (again, this is expected occasionally).

- this cascaded to take down more than a billion people's service (WTF; this is not how you design systems).

- it took 15 hours to deploy the fix (not good, but at this scale maybe).

Feels like they are completely missing the point of where the problem is and how to fix it?


Replies

topaz0 • yesterday at 11:32 PM

Antitrust

VirusNewbie • yesterday at 10:53 PM

>- it took 15 hours to deploy the fix (not good, but at this scale maybe).

I've definitely been in similar scale war rooms and 15 hours is not acceptable.

lokar • yesterday at 3:20 PM

In terms of cascading failure and inability to rollback or fix quickly, MS has always been terrible.

➕ show 1 reply