alt Hacker News> What's much more likely is that your US AI provider is promising not to train on your data but is doing so anyway.
There's genuinely no evidence of this for OpenAI and Anthropic. It's impossible to disprove, but I don't think it's likely because:
* They get enough volume from consumer subs with data training enabled anyway.
* If this was happening, it needs serious work at the scale OpenAI and Anthroppic, from data pipelines, to ablation experiments, to the actual data mix and traces going in all the telemetry/diagnosis of large-scale training runs.
* It would need to involve a team. Employees at these companies leave, there have been numerous whistleblowers, allegations, etc. Nothing on this front that I can find.
* It would damage enterprise trust permanently and be a company and reputation-ending thing. Now that these tools are used by everyone from state governments to the DoW, the exposure radius is massive, investors (many of whom are customers/users too; and often have their stakes in not just a single company but multiple) would not be happy. Piss off enough powerful people, and anyone can join Sam Bankman-Fried in prison.
* There's a myriad of enterprise customers and bespoke contracts. I can't get into details, but not all enterprises accept a 'trust me bro' clause.
Replies
> It's impossible to disprove, but I don't think it's likely
If something is impossible to disprove, we must assume it is happening from a threat modeling perspective.
I'm not saying that Anthropic and OpenAI are definitely dishonest, there is no proof of that. However, when you have
1) an extremely high financial incentive to be dishonest (trillions of dollars),
2) low-ish chance of being caught, especially if you launder the data through another model to remove identifying information,
3) the people in charge of said operations are generally agreed to be snakes,
then it should at least arouse suspicion. You may be right that get enough data from opt-ins that they don't need to do it. I do believe that they don't violate enterprise ZDR agreements, but for normal subscriptions I'm much less confident.
> * It would need to involve a team. Employees at these companies leave, there have been numerous whistleblowers, allegations, etc. Nothing on this front that I can find.
You have to also note the incentives going on here. Employees who have extremely valuable stock options, they have good reasons to not come public with anything that would damage the reputation of the company before they can cash out