alt Hacker NewsIt does seem like its way too easy to do this kind of fraud.
The study seems extremely easy to replicate (no fancy equipment needed, just ask some people to proofread), and yet nobody did until now. Is there some way to change incentives to get people to try and replicate? Assign undergrads to replicate random research papers?
Replies
Replication is unpaid QA on someone else's product. Nobody gets tenure for finding the bug, hence the field has been meaning to get around to it for 20 years.
Why do we even accept anything as true without replication by default?
"easy to replicate (no fancy equipment needed, just ask some people to proofread)"
? This was a human subject study, that would cost $$ and there would be very little upside to confirmation and only a little notoriety with a failure to replicate (which by the way is not dispositive, people might need multiple failed replications to know who to believe, depending on the size of the replication). Maybe you didn't read the post: this was data fraud discovered by statistical irregularities. That is indeed becoming easy to spot with more widespread statistical literacy, at least for unsophisticated jobs like this from decades ago.
> Is there some way to change incentives to get people to try and replicate?
Sure. Pay them to do it.
> The study seems extremely easy to replicate (no fancy equipment needed, just ask some people to proofread), and yet nobody did until now
They were just putting it off.