OSF- Reproducibility Project tries to replicate the results published in three psychological journals

Posted: April 23rd, 2012 | Author: Sven | Filed under: journals, Opinion | Tags: academic publishing, fraud, psychology, Replication | 1 Comment »

“If you’re a psychologist, the news has to make you a little nervous…”. With this statement Tom Bartlett introduced his article  “Is Psychology About to Come Undone?” in the Chronicle of Higher Education.

The source of his fears is the Reproducibility Project  – a group of researchers that aim to replicate every study within the three journals Psychological Science, the Journal of Personality and Social Psychology and the Journal of Experimental Psychology: Learning, Memory, and Cognition published in the year 2008.

The project is part of Open Science Framework (OSF), a group that is interested in increasing the alignment between scientific values and scientific practices. Despite developing some tools and infrastructure projects its stated mission is to “estimate the reproducibility of published psychological science.”

For Bartlett the works of the Reproducibility Project  is a more polite way of saying “We want to see how much of what gets published turns out to be bunk” – and I think he’s right with his interpretation.

But in general Bartlett dosn’t find it useful, that OSF is replicating the published results: ” It’s time-consuming and doesn’t do much for your career to replicate other researchers’ findings. Journal editors aren’t exactly jazzed about publishing replications. And potentially undermining someone else’s research is not a good way to make friends.”

In my opinion these aren’t valid arguments at all. If science is a principle, researchers have to take care that scientific findings aren’t nonsense or even fabricated. What if fabricated or fraudulent results shape therapies and the way how humans are medicated? Would the replication of results then still be a waste of time? Not a good way to make friends?  I don’t think so – in the contrary I think it is a useful approach to replicate the results of a scientific disciplines in a few journals from time to time. And it would be very interesting to see how well the peer-review process in the journals performs.

In general it would be a step in the right direction: Scientists have to think about the worth of reproducible research and how they might support it. Incentives for the fabrication of results as well as for sloppiness are minimized and a whole profession gets reminded that the fundamental principles of science are more than just a annoying part in discussions.

In my opinion verifying the results of research is a fundamental goal of science and should not only be the mission of a single replication project. I think it would be a much better model if scientific organisations or professions are checking the results of published research themselves – because fraud, fabrication and bad documentation of research undermines research much more, than every replication group might do.

Readers interested in discussing this topic should also read the large comments section at the end of Bartletts article.

Photo: Benjamin-Thorn / pixelio.de

---