alt Hacker News>deterministic verification like this is not, and should never be, the LLM’s job.
But when humans handled it, this was not as much as a problem. That is, the humans did the job, because they recognized the need to do that job.
Sure sometimes accounts could get recovered if a human was tricked, but evidently it was easier to trick the LLM in masse than humans.
> But when humans handled it, this was not as much as a problem.
In fact it's arguably a feature. The ability of support staff to short-circuit nitpicky rules when there's an obvious external validation happening (e.g. you're on the phone with a user who's presenting ID in real time and correlating it with previous use of the account, etc...) makes for better data quality and happier customers.
Obviously, yes, you can then human-engineer an authentication breach. But that was very difficult, because people are "common-sense careful" in a way we haven't been able to tease out of AI yet.