Kind of interesting that LLMs are basically being sold as having “human-like” reasoning capabilities, but in this case when “obamawhitehouse” asked to have it’s password reset sent to [email protected] the LLM didn’t question it and just triggered the process that happened to have a bug.

Humans support agents certainly fall prey to social engineering all the time, but I can’t think of a case where it was done on this scale so easily.

It probably could have been, but how likely is that compared to with the AI agent? I'd assume (and I'm ready to look like an idiot if I'm wrong) that the humans are trained to send the verification code to the email address on file, rather than any address the client asks them to. I'd certainly assume most of them are more afraid of the consequences than the AI is.

I think they’re blaming a tool function so as not to admit the overall agent process was shit.

But it’s irrelevant, outside of PR. We know at least THREE bad components to this process and they were constituent parts.