alt Hacker NewsYou would have to have a very complete audit trail for the LLM and ensure the paper shows up nowhere in the dataset.
We have rare but not unheard of issues with academic fraud. LLMs fake data and lie at the drop of a hat
Replies
6stringmerc • last Wednesday at 12:55 PM
…because there are no consequences for AI. Humans understand shame, pain, and punishment. Until AI models develop this conditional reasoning as part of their process, to me, they’re grossly overestimated in capability and reliability.
> You would have to have a very complete audit trail for the LLM and ensure the paper shows up nowhere in the dataset.
We can do both known and novel reproductions. Like with both LLM training process and human learning, it's valuable to take it in two broad steps:
1) Internalize fully-worked examples, then learn to reproduce them from memory;
2) Train on solving problems for which you know the results but have to work out intermediate steps yourself (looking at the solution before solving the task)
And eventually:
3) Train on solving problems you don't know the answer to, have your solution evaluated by a teacher/judge (that knows the actual answers).
Even parroting existing papers is very valuable, especially early on, when the model is learning how papers and research looks like.