The AI part does seem relevant because it enabled incredibly low-effort “social” engineering.

For what it’s worth I don’t think you can call this social engineering since there was no human on the other end, even though it appears similar.

The question is, if there were actual human support agents, would they have built additional safeguards to prevent social engineering in this manner?

My impression is that AI didn't replace static code in this place; it replaced a person, who (hopefully) would have been suspicious about sending an account recovery code for e.g. "obamawhitehouse" to e.g. "[email protected]"

This is not true. Well, it kinda is, but nobody will be stupid enough to hand-code an account recovery where you get to type any email address.

The reason it worked there is that the designers of the system didn't anticipate that the AI will agree to accept any email (maybe they even put guardrails against it in the system prompt, we don't know). It's more like social engineering than bad-security-code, except that like the sibling comment said an actual human will probably not approve that.

> This exact same flow could have been…statically coded.

But had never been until it was wrapped in a chatbot. It’s just about unheard of for a major site in the modern era, isn’t it? I think the AI factor is essentially essential. All but.

Drowning has essentially nothing to do with water and everything to do with a terribly designed ability to get air into your lungs.

If you'd do a retrospective and ignore how AI has shaped expectations and a company's culture to allow this to pass through into production, you'd be complicit/perpetuating what led to this debacle in the first place.

It's not the end of the world, and water isn't going anywhere, but saying AI has essentially nothing to do with it is just a bad take.