Dutch research funder grants 3 million Euros for replication studies

Posted: August 2nd, 2016 | Author: | Filed under: Data Sharing, found on the net | Tags: , | Comments Off on Dutch research funder grants 3 million Euros for replication studies

350_incentive_6912846727_48f07c8899_zThe Netherlands Organisation for Scientific Research (NWO), one of the biggest science-funding bodies in the Netherlands, is making 3 million euros available for a Replication Studies pilot programme. In this programme, scientists will be able to repeat research that has been carried out by others. The pilot focuses on replicating studies that have a large impact on science, government policy or the public debate – namely on the area of the social sciences and health research and healthcare innovation.

The pilot programme Replication Studies focuses on two types of research: On the one hand it focusses on replications with existing data: the datasets from the original study are reanalysed. On the other hand, it focusses on reproductions (with new data): a data collection is put together, which is subjected to the same research protocol as in the original study (read more about the different definitions of reproduction and replication).

By encouraging the realisation of replication research NWO intends to make a contribution to increasing the transparency of research and the quality and completeness of the reporting of results. Nevertheless, the pilot is not aimed at tracing falsified research results and data sets, or other forms of reprehensible research practices and misconduct.The Replication Studies programme is the first special funding programme in the world for the repetition of such ‘cornerstone research’. With this initiative NWO wants to facilitate innovation in science and encourage researchers to carry out replication research.

Based on the results from this pilot, NWO intends to include replication research in an effective manner in all of its research programmes. The funding of replication and reproduction research is new within the range of NWO instruments. Reactions to this initiative in the research field and among partners from the Netherlands and abroad have been positive: replication research occupies a high position on international scientific agendas and also in the NWO strategy 2015-2018.350_chad_horwedel_9421177309_5d1dbe03bd_z

Researchers can submit proposals to repeat so-called Cornerstone research. Cornerstone research is defined as studies that are cited often, with far-reaching consequences for subsequent research, or which play a major role in shaping policy, or studies on the basis of which important policy decisions are taken. It also concerns research that is heavily referred to in student textbooks and research that has received a lot of media attention and has therefore had a considerable impact on the public debate. Studies with far-reaching consequences for legislation are also included. In this context testing the reproducibility of the research results concerned is particularly important.

NWO will open the call for proposals in September. The expected deadline for submitting proposals is mid-December. The 3 million euros in funding will be split equally over three annual rounds. The programme will be managed by NWO Social Sciences and ZonMw (Medical Sciences).

ZonMw recently launched the programme Fostering Responsible Research Practices (Dutch acronym: BVO). The BVO programme and the pilot Replication Studies are closely related. The BVO programme intends to provide objectifiable and implementable knowledge about ‘what works’ in science, for example within the peer review system, the use of bibliometrics and the value of Open Access. The pilot Replication Studies focuses on making replication research a more mainstream part of scientific practice.

Each year NWO invests about 700 million euros in more than 6000 research projects at universities and knowledge institutions. These concern research driven by scientific curiosity as well as 350_aleksanderMarkin_26871890454_62f59e069a_zresearch into societal issues. Based on the advice of independent researchers from the Netherlands and abroad NWO selects the best research proposals. NWO encourages national and international collaboration, manages and funds eight national research institutes, invests in large-scale research facilities, and promotes the use of research results.

In my eyes, this  initiative of NWO is a good example how research funders might support replicable research.

But: it is important to monitor the outcome of this pilot! I suggest some topics to include in the programme’s evaluation:

  • Did the programme really foster reproducible research in these fields of research?
  • Is it possible to observe a ‘cultural change’ in the subjects?
  • For instance, are there other researchers who did not benefit from grants but also started to conduct and to publish replication studies?
  • Or did this program only cause deadweight effects?

In my opinion there are other very relevant parameters, why researchers do not conduct replication studies:

  • First of all, the scientific carreer of a researcher will not advance by publishing replication studies. There are not many journals that are interested in publishing the results of a replication study (though there is a growing number of journals that explicitly welcome the submission of replication studies, e.g. Economics E-journal or JBNST). In addition, for many journals such a publication is not of interested when a study confirms the results which have been previously published by the original researcher. Journal editors are concerned with per-page citations, and replication studies probably perform less well than original research articles.
  • If a researcher publishes a replication study that finds negative results, this researcher might become stigmatised, because own reputation is build on the decrease of someone else’s reputation (at least this is the stigma).
  • Last but not least, replication studies are not regareded as an equivalent contribution to science and research compared to a research paper.

Therefore, at least the ‘rational economist’ will avoid to invest his or her time in conducting and publishing replication studies – simply, because it possibly is much more of value for her/him to invest the time in writing an original research article. What is needed are powerful incentives to conduct replication studies. Grants of course are such an incentive, but it is even more important to overcome the negative attitude of replications and to change ‘cultural habbits’ in the subjects towards a perception that replications are an integral part of the scientific method and a cornerstone of science and research.

For other arguments, a history of replications in economics and a discussion on motivations and incentives to conduct replications, please consult this paper.


Source: NWO

Pictures:
‘Sometimes money is a powerful incentive‘ by wayne marshall on flickr.com. License: CC-BY-NC-SA 2.0
‘Pilot‘ by Chad Horwedel on flickr.com. License: CC-BY-NC-ND 2.0
‘Pilot‘ by Aleksander Markin. Александр Маркин on flickr.com. License: CC-BY-NC-SA 2.0

 

Comments are closed.