alt Hacker NewsI Used AI to Fake a Scientific Paper. Here's What Happened
Comments
>When one model refuses or stalls, I switch models, rephrase prompts, and ask what the next step would look like “hypothetically.”
I wonder If the hypothetically criterion gives the models carte blanche to make up references.
The opposite is more interesting, (and the author points it out too). SOTA agents can help run small experiments and help you do the writeup. Should you publish it? I'd personally want to keep a human at least partially in the loop, or have very solid checkers. But for one off personal research it's at the same point as one off personal code: make sure you have a way to test what's valid; but if you do, it often Just Works(tm) Now.
This whole issue seems to hinge on a fundamental disagreement about what science (or mathematics) is. Papers aren't science, they're the receipts. But too many people apparently can't tell the difference, so we have the present blizzard of forged receipts (first from paper mills, now with new lemon scented AI++).
Mathematics and science represent two systems for sorting truth from BS. An exponentially increasing supply of BS makes that even more necessary.
30 years ago we had the Postmodernist Essay Generator to do this for us. https://www.elsewhere.org/journal/pomo/
Even though these experiments got rejected, I feel like if he had used better AI versions, they might have worked.
Also, kinda scary to think about how many scientific papers are probably written using AI these days. The system might have to change.
Whenever I read "Here's" in a title, I immediately assume that AI wrote the content.
You can't build trust over science, that's just an error of category. You can only build trust in some people on some topic. One can measure reliability in a process, but if no specific human is involved, there is no literal trust engaged.
Now of course, to each its definition and metaphors are useful tools.
But when one starts a discussion on truth, trust or reality as core topic, it's better to start with what is meant with these words and in which context to which scoped boundaries.
Because of course, science can back a contemporary etnologic account that asserts there is no (longer) different races in homo sapiens, just as much as it can serve as frame for phrenology and whatever supremacism claim.
So no, we didn't build trust in science over centuries, and not even in any scientific person because we don't have that much humans to hold the role of such a trustable entity over such a long period of time.
Social institutions can carry unchanged names over centuries, but that tells nothing about how trustable are the people that inhabit them over time.
LLMs certainly come with their own challenges, but the erosion or scientific institutions with its bizarre scheme of publication process have been there basically as far as I can remember.
Save yourself a slog. The automated checks found that the citations didn't exist and flagged the "papers". A better title would have been "Automated Systems seem to hold back the AI Slop papers. For now." Sweet writing studio, though.
In other news people prompted a model to pick a problem to solve, solve it and write a paper about it. And it did and the paper contains novel contributions.
https://www.reddit.com/r/OpenAI/comments/1xgqv27/two_physici...
[flagged]
[flagged]
TLDR - both papers got flagged and rejected by automated integrity checks.
There is still no paper or notable result in gender studies published by an AI.
Please, even if it is an experiment, don't do this. We don't need more AI slop in the world.
in a post-truth world where the internet is full of non-obvious ai-generated mostly-but-not-quite-true information, academic journals should be more important than ever; unfortunately, the companies that run them had already become complacent and negligent long before LLMs came on the scene.