German Research Foundation (DFG) publishes Statement on Replicability

Posted: April 26th, 2017 | Author: | Filed under: found on the net, German, Opinion | Tags: , | Comments Off on German Research Foundation (DFG) publishes Statement on Replicability

The German Research Foundation (DFG) has currently released a statement on the replicability of research results.

Interestingly (at least for me), the five-pager first starts with a broader definition of what replicable research is NOT.

Of course, replication is a very important method for testing empirical knowledge claims based on experimental and quantitative research in medicine, the natural, life, engineering, social and behavioural sciences, as well as the humanities.

But, according to DFG, there are limitations:

  • Replicability is not a universal criterion for scientific knowledge.
  • Ascertaining the replicability or non-replicability of a scientific result is itself a scientific result. As such, it is not final but subject to methodological scepticism and further investigation.
  • Non-replicability is not a universal proof by falsification.
  • Non-replicability is not a universal indicator of poor science.

‘Well, an unorthodox starting point for a paper on reproducible research‘ – so, at least, were my thoughts when I read the first page of the statement. Wouldn’t it be more common to first depict the important aspects of reproducible research and to suggest measures to support it, instead of rowing back at the beginning of such a statement?

Also the following positions of the DFG begin with two bulletpoints on the scientific and public debate on reproducible research, which the DFG describes as ‘often simplistically‘.

Starting with bulletpoint three, the research funder takes up the core of the challenge. DFG states, that ‘in their totality, the reported cases of non-replicability are a cause for concern‘ and the replication crisis ‘indicates a quality problem in research‘. Therefore ‘researchers, research institutions and research organisations must take this quality issue very seriously. It endangers the performance of science as well as society’s confidence in it‘.

The DFG also points out that along with individual misconduct, there are also structural reasons for the quality problem in research. Such structural resasons include, but are not limited to the weight of quantitatively-parametrising control, evaluation and gratification systems in science and research, because these instruments create an ‘increased (and continually increasing) pressure to compete and to accelerate results.’ This pressure is manifest in decisions about career moves, financial support, location of publication, and institutional structural trends.

The DFG also names another very central aspect: The preparation, execution, evaluation, description and publication of experimental and empirical-quantitative research requires time, opportunity, funding and staff. And within the publish or perish culture sufficient time and money are scarce resources. Therefore competitive and time pressures threaten, rather than support, high-quality research. The DFG also critically recognises that this ‘structural framework […] can all too easily be misunderstood as an invitation to quick-and-dirty research practices‘.

Hence, the German research funder also acknowledges its own responsibility. This is a reason why the DFG announces some changes in their policies:

  • The main criterion for scientific judgement will be the quality of publications rather than their quantity or location,
  • Replication as a method for testing experimental and empirical quantitative research results must be systematically strengthened,
  • The funder will facilitate and support processes of subject-specific investigation of questions concerning the replicability of research results; this also includes the development of subject-specific criteria for funding replication studies as well as the funding itself,
  • It will continue to pay particular attention to questions of research data management and current challenges that emerge from digitalisation,
  • The development of infrastructure and methodological tools as well as their use for this purpose will be promoted.

To conclude, the DFG calls on academic publishers, scientific institutions and ethics commissions, as well as lawmakers and scientific policymakers, to do everything in their power to combat the structural reasons for replication difficulties.

Despite my scepticism, when I read the first pages of the statement, I am quite pleased the DFG released such a statement. Apparently it has not been an easy task to publish these positions and it seems so that not every researcher is happy to publicly announce this important topic. Possibily, one does not want to tread on the toes of some researchers…

Nevertheless it is very important to find such a statement published by one of the most important research funders in Germany. It will facititate the discussions on reproducible research and might also have an impact on other players like journals and editorial offices to foster reproducible research.

 

 

Comments are closed.