alt Hacker NewsNot even tests but just some basic anomaly detection lol.
Like maybe if the bill amounts increase by like 10M% there should be someone that looks into it
Replies
While I didn't work on AWS, I did intern on the retail side of Amazon, and there's definitely this sort of monitoring in place. Surely somebody was paged. And even if not, this is "just" the cost explorer estimations, not what is ending up on folk's bills.
I learned about <https://en.wikipedia.org/wiki/2011_T%C5%8Dhoku_earthquake_an...> from alarms like this, as sales in Japan almost entirely stopped.
I've been told a tale of another incident where some customer ran some huge cpu-intensive workload that didn't do any networking. It caused various alarms to fire because it "looked like" a part of the network was idle (potentially indicating some sort of networking failure)
It's generally (in the broad sense) easy to add alarms for things going wrong, but in my experience anomaly detectors are just as likely to fire from other weird things like that happening.
Clearly some folks here have never worked at Amazon before. It's genuinely terrifying to see what most of the internet runs on.
I think billing is the only thing AWS doesn't really care about optimizing or putting enough tests to avoid anomalies lol.
They've already got anomaly detection: their users.
If only there was some way to get anomaly detection services [1] inside of AWS...
You overestimate how much people give shits at big techs like Amazon. When literally everything is driven with sticks instead of carrots, the work culture does not invite employees to proactively care about product quality.
You'd be better off letting the heart attacks happen and take the 3am on-call and be the hero instead. It would be good promo doc material, and being a hero is extremely good insurance against getting kicked out of the country (via the PIP->H1B grace period expiry mechanism).