Sure. But you might not even be realizing that something is just at the cusp if the load is spiky enough.

The art of large system design is to identify and avoid these kinds of chokepoints. And when something happens, propagate the "backpressure" up the stack to avoid queuing.

AWS got a fair share of similar outages, so the newer SDKs now try to not exacerbate these kinds of issues: https://docs.aws.amazon.com/sdkref/latest/guide/feature-retr...

The original AWS EBS outage is probably the canonical example: https://aws.amazon.com/message/65648/