alt Hacker NewsWhile this is relevant and should indeed be fixed, the attack surface and the practicality of the exploit is a bit meh.
The user needs to do 3 things for this to be actually be phished:
1. Receive money from somebody they don’t known with a weird description 2. Proactively ask the agent for such transaction 3. Click the link the agent provide
While this of course can happen on scale, doesn’t seems so critical in practice
Replies
Unless I missed it they didn't provide any proof of this actually working. Really seems like a thing veiled advert for their product
Depending on how much access the AI agent has, there are worse things to inject it with than a link.
People already click suspicious emails that ask them to login. At a high number of attempts, some chickens will be caught. However, people are now weary of emails since there is a lot of phishing there. On the other hand, the AI assistant env. could be considered "safe" by users because it's stuff coming from the bank. So they are more likely to fall for it. (honestly, unless you are a dev and aware of prompt injection, I don't see why the users wouldn't fall for it).
I think the critical part is that it launders an arbitrary URL as trustworthy. The alternative is “Don’t trust anything our bot says at face value, please.”
I think a better criticism is allowing arbitrary text (including URLs) in a transaction description.
Thanks for chiming in.
I agree this is not a one-click account takeover.
But I think point 2 is broader than that. The user does not need to ask about the malicious transaction specifically. Any normal question that makes the agent fetch recent transactions could bring the attacker-controlled text into the LLM context.