logoalt Hacker News

ACCount37today at 8:03 AM1 replyview on HN

It is a reliability problem - because if the API ends up rejecting 30% of the incoming queries, no one cares if it's 500 Internal Server Error or 529 Overloaded.

Having your infrastructure at the knife's edge of load to capacity also means you have no redundancy when something fails.


Replies

cheriootoday at 8:11 AM

If this incident had happened during the day I could buy that argument. But it is happening in the dead of the night. Are there really that much people scheduling claude runs over-night, taking more capacity than day-time? (assume most claude users are west coast programmers)

My other guess is they are scheduling training runs and their capacity isolation is bad.

show 2 replies