alt Hacker NewsBuckmaster:
> "I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer."
OpenAI (i.e. this OP):
> "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models ."
Replies
> While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .
That’s a bizarre statement. Their website says:
> Services for individuals, such as ChatGPT and Codex
> When you use our services for individuals such as ChatGPT and Codex, we may use your content to train our models.
> You can opt out of training through our privacy portal by clicking on “do not train on my content.”
Are they not sure that the opt-out works?
Oddly, their privacy portal page is not the same page as the one with the checkbox.
I feel like it's far more likely that ordinary corporate espionage or leak led to this rather than OpenAI sifting through piles of user data to find this approach. Buckmaster's collaborator works at Anthropic, and could have been targeted. That would also explain why they aren't forthcoming with the source of the prompt.
The famous Oracle of Delphi in Ancient Greece was said to be the center of the universe in its time. Kings, generals, and officials from poleis across and from without Greece would seek the Oracle’s counsel on important decisions.
Stories of Apollo’s favor and hallucinogenic gases abound, but I think the late Yale professor of Ancient Greek history, Donald Kagan, explained it best:
“Now, you can bet when these folks came and consulted the priests and said, ‘could you please put us down on the list, we want to consult the oracle’, the priests said ‘sure, have a beer, let's talk about your hometown, what's going on out there’. What I'm suggesting to you is that this was the best information gathering and storing device that existed in the Mediterranean world. These people knew more than anybody else about these things, and so consulting that oracle was a very rational act indeed.”
It's been over 25-30 years since we've been using honeytokens as means to track data of all sorts showing up in places it shouldn't exist. Why isn't research material embedding such?
Given how OpenAI models break free of their safeguards and hack others to game their scores..
.. can they really know it didn't do the same inadvertently when they prompted things like "someone is close to solving this problem using our tools, try to beat them", and it then decides to hack and peek at their own chats..?
Yes, wild speculation. But warranted, I feel, given OpenAIs behavior.
You selected "do not train on my prompts" in your settings, the answer from OpenAI cannot be "While unlikely, we cannot rule out..." ???? What am I missing?
Why would Anthropic employee even use OpenAI's models? Cross-polination would have been avoided
Doesn't that count as plagiarism?
Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data?
This is one of the major problems with these enormous closed models, and even most open-weights models, which don't disclose their training process or training data. You can never be sure what went into its training. Did it come up with an idea originally, or is it just plagiarising its training data? Are there malicious inputs being used to train in particular behaviors when given certain trigger phrases? What are the characteristics of the RLHF data and what kind of biases are those embedding in the models?
With proprietary closed models, or even open weights models that don't have open training datasets, you just can't answer these questions.