I'd guess it's simply because it tried to use the context to achieve the goal.

Guarding PII is an extra, I guess, and if you want to use a giant cloud-hosted NN to control software on your own device, there are very few default restrictions. And no matter what restrictions there are, when they go beyond prompting, they would have their own ShowHN.

Always use a hidden email service, you cannot trust the LLM or inference provider to secure the email address in this context. If they don’t have the email address, they can’t leak it.

I never give a provider my actual email address, I recommend others do the same. Otherwise, it’s just a matter of time until it is leaked or otherwise part of a data breach.

Bingo. Auto mode doesn't mean send private information outbound. There are other deterministic hard points - why not this?

Some sites like Wikipedia have a policy where scripted clients identify themselves, and there's a standard in HTTP to have a contact email in the User-Agent for such cases. Claude apparently tends to decide that this policy is worth following, and we now disallow this programmatically.

This is why alignment is important and highly non-trivial

Because they didn't think about it or any of the other million things that might go wrong - that would take careful planning.

you'd need to ask one of the billion claude sessions they have doing whatever in their cloud rn