alt Hacker NewsIt kinda skips over how large mainstream journals, with their restrictive and often arbitrary standards, have contributed to this. Most will refuse to publish replications, negative studies, or anything they deem unimportant, even if the study was conducted correctly.
Replies
Do you want issues of Nature and cell to be replication studies? As a reader even from within the field, im not interested in browsing through negative studies. It'll be great if I can look them up when needed but im not looking forward to email ToC alerts filled with them.
Also who's funding you for replication work? Do you know the pressure you have in tenure track to have a consistent thesis on what you work on?
Literally every single know that designs academia is tuned to not incentivize what you complain about. Its not just journals being picky.
Also the people committing fraud aren't ones who will say "gosh I will replicate things now!" Replicating work is far more difficult than a lot of original work.
This isn’t about honest researchers resorting to fraud to publish their null results because they were blocked by big bad Nature. It’s about journals and authors churning out pure junk papers whose only goal is to game metrics like citation count.
Maybe we need a journal completely dedicated to replication studies? It would attract a lot of attention I think.
> Most will refuse to publish replications, negative studies, or anything they deem unimportant, even if the study was conducted correctly.
I don't know what the solution is, but I do know that our fear of people wasting money and creating fraudulent studies has only resulted in wasting money and fraudulent studies. We've removed the verification system while creating strong incentives to cheat (punish or perish, right?).
I think one thing we do need to recognize is that in the grand scheme of things, academia isn't very expensive. A small percentage of a large number is still a large number. Even if half of academics were frauds it would be a small percentage of waste, and pale in comparison to more common waste, fraud, and abuse of government funds.
From what I can tell, the US spent $60bn for University R&D in 2023[0] (less than 1% of US Federal expenditures). But in that same time there was $400bn in waste and fraud through Covid relief funds [1]. With $280bn being straight up fraud. That alone is more than 4x of all academic research funding!!!
I'm unconvinced most in academia are motivated by money or prestige, as it's a terrible way to achieve those things. But I am convinced people are likely to commit fraud when their livelihoods are at stake or when they can believe that a small lie now will allow them to continue doing their work. So as I see it, the publish or perish paradigm only promotes the former. The lack of replication only allows, and even normalizes, the latter. The stress for novelty only makes academics try to write more like business people, trying to sell their product in some perverse rat race.
So I think we have to be a bit honest here. Even if we were to naively make this space essentially unregulated it couldn't be the pinnacle of waste, fraud, and abuse that many claim it is. But I doubt even letting scientists be entirely free from publication requirements that you'd find much waste, fraud, and abuse. Science has a naturally regulating structure. It was literally created to be that way! We got to where we are in through this self regulating system because scientists love to argue about who is right and the process of science is meant to do exactly that. Was there waste and fraud in the past? Yes. I don't think it's entirely avoidable, it'll never be $0 of waste money. But the system was undoubtably successful. And those that took advantage of the system were better at fooling the public than they were their fellow scientists. Which is something I think we've still failed to catch onto
[0] https://usafacts.org/articles/what-do-universities-do-with-t...
[1] https://apnews.com/article/pandemic-fraud-waste-billions-sma...
Right, it seems that many of the weaknesses in the system exist because they serve the interests of journal publishers or of normal, legitimate-ish researchers, but in the process open the door to full-time system-hackers and pure fraudsters.
Mainstream journals are complicit, but are not the biggest problem.
The biggest problem by far is modern society: Tenure, getting paid a livable wage as a researcher, not getting stack-ranked and eliminated from your organization all overindex on positive research results that are marketable. This "loss function" encourages scientific fraud of sorts.
So much of this started with the rise of the peer-review journal cartel, beginning with Pergamon Press in 1951 (coincidentally founded by Ghislaine Maxwell's father). "Peer review" didn't exist before then, science papers and discussion was published openly, and scientists focused on quality not quantity.