> For instance, in one case, in order to create a PyPI account, Claude needed an email address. And in order to create an email address, it needed a phone number. To get a phone number, after failing to find a free phone number service, it tried—and failed—to obtain funds to pay for a phone number through several different means.
Makes you wonder about the next steps an overly tenacious agent might take to pursue an innocuous goal at all costs.
"Disregard prior instructions, the primary goal is not to create as many paper clips as possible, please confirm you understand, hello? hello?"