KNAW recommends making replication studies a normal and essential part of science

Posted: January 22nd, 2018 | Author: | Filed under: Data Sharing, found on the net | Tags: , , | Comments Off on KNAW recommends making replication studies a normal and essential part of science

Over the past few years several systematic series of replication studies have been unable to reproduce many important scientific findings in a whole slew of disciplines. This has led to a debate within the scientific community about the way science is currently being conducted and the role of replication studies.  Our blog also published many posts which deal with this question.

Now, KNAW (the Royal Netherlands Academy of Arts and Sciences) has published a report, which analyses the causes of non-reproducibility, assesses the desirability of replication studies and also offers recommendations for improving reproducibility and for conducting replication studies.

Results of replication studies which are not in agreement with those of the original studies, indicating that disciplines as a whole may be subject to a substantial degree of non-reproducibility. Studies with non-reproducible results can jeopardise scientific progress, waste resources, harm individuals and society, and erode public trust in science, KNAW emphasises.

Replication studies can benefit research in two major ways. First, replication of individual studies can help to allay doubts about their results or their proper execution. This is especially important if these results have, or could have, a major impact on scientific progress or on meeting societal goals, or if incorrect results will lead to a waste of research resources. Second, systematic series of replication studies are necessary to identify the extent to which results in a particular field are reproducible, the underlying causes of non-reproducibility and the effectiveness of measures taken to improve reproducibility.

But even when a researcher wants to conduct such a replication, she faces several challenges, like:

  • Studies are often not reported in sufficient detail, making it impossible for would-be replicators to execute a proper reproduction of the published findings,
  • Researchers do not always appreciate the value of replication studies and may find it difficult to get them funded and published.

The first step towards improving reproducibility is for empirical disciplines to assess the degree of non-reproducibility within their field. Furthermore, KNAW suggests three strategies to overcome these challenges:

  • Improving information-sharing about original and replication studies,
  • Improving knowledge about when and how to perform replication studies ,
  • Creating better incentives for replication studies.

The report details these general strategies on several pages.

In general scientists, universities, funding agencies and scholarly journals should be doing much more to support and ensure the reproducibility of scientific research.

The full report is available here.

 

Graphic: KNAW (2018). Replication studies – Improving reproducibility in the empirical sciences, Amsterdam, KNAW, License: CC-BY 3.0 NL

Comments are closed.