KNAW recommends making replication studies a normal and essential part of science

Over the past few years several systematic series of replication studies have been unable to reproduce many important scientific findings in a whole slew of disciplines. This has led to a debate within the scientific community about the way science is currently being conducted and the role of replication studies.  Our blog also published many posts which deal with this question.

Now, KNAW (the Royal Netherlands Academy of Arts and Sciences) has published a report, which analyses the causes of non-reproducibility, assesses the desirability of replication studies and also offers recommendations for improving reproducibility and for conducting replication studies. Read the rest of this entry »

German Research Foundation (DFG) publishes Statement on Replicability

The German Research Foundation (DFG) has currently released a statement on the replicability of research results.

Interestingly (at least for me), the five-pager first starts with a broader definition of what replicable research is NOT.

Of course, replication is a very important method for testing empirical knowledge claims based on experimental and quantitative research in medicine, the natural, life, engineering, social and behavioural sciences, as well as the humanities.

But, according to DFG, there are limitations:

• Replicability is not a universal criterion for scientific knowledge.
• Ascertaining the replicability or non-replicability of a scientific result is itself a scientific result. As such, it is not final but subject to methodological scepticism and further investigation.
• Non-replicability is not a universal proof by falsification.
• Non-replicability is not a universal indicator of poor science.

‘Well, an unorthodox starting point for a paper on reproducible research‘ – so, at least, were my thoughts when I read the first page of the statement. Wouldn’t it be more common to first depict the important aspects of reproducible research and to suggest measures to support it, instead of rowing back at the beginning of such a statement? Read the rest of this entry »

Article: The Economics of Reproducibility in Preclinical Research

In 2012, an estimated 114.8 billion $ in the US were spent on life sciences research. Roughly half of it is spent on preclinical research, with government sources providing the majority of funding – approximatly 38 billion US$.

Now, three researchers calculate the costs of irreproducible research in preclinical research near 28 billion $ – only for the United States alone. That is the conclusion of a study published in PLoS Biology a few days ago.

In the opinion of the study’s authors, the giant amount of 28 billion $ accumulates, because low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development.

Read the rest of this entry »

Biomedical Sciences: Journals unite to forster reproducible research

When it comes to the topic of replication, it always is a good idea to consult the webpages of the nature journal. Yesterday, for instance, the journal reported that a group of editors representing more than 30 major journals, representatives from funding agencies as well as scientific leaders discussed principles and guidelines for preclinical biomedical research in June 2014.

The gathering was convened by the US National Institutes of Health, Nature and Science.

The attendees agreed on a common set of principles and guidelines in reporting preclinical research that list proposed journal policies and author reporting requirements in order to promote transparency and reproducibility. Read the rest of this entry »

New Nature Special: Challenges in irreproducible Research

nature has published a new special issue on challenges in irreproducible research. The journal addresses the challenges and barriers of reproducibel research:

No research paper can ever be considered to be the final word, and the replication and corroboration of research results is key to the scientific process. In studying complex entities, especially animals and human beings, the complexity of the system and of the techniques can all too easily lead to results that seem robust in the lab, and valid to editors and referees of journals, but which do not stand the test of further studies. Nature has published a series of articles about the worrying extent to which research results have been found wanting in this respect. The editors of Nature and the Nature life sciences research journals have also taken substantive steps to put our own houses in order, in improving the transparency and robustness of what we publish. Journals, research laboratories and institutions and funders all have an interest in tackling issues of irreproducibility. We hope that the articles contained in this collection will help.

All articles within this issue are available free of charge. The table of contents is available here.

Graphic: pasukaru76, www.flickr.com