The replication crisis in economics – and how we might answer it
Posted: September 21st, 2017 | Author: Sven | Filed under: Conference, Workshop | Tags: journals, replication studies | Comments Off on The replication crisis in economics – and how we might answer it
On September 8, 2017 the ZBW Leibniz Information Center for Economics hosted the workshop “Replications in Empirical Economics – Ways out of the Crisis” at the Annual Conference of the Verein für Socialpolitik in Vienna, Austria. Thirty participants and four speakers engaged in lively and stimulating discussions about replications and the publication of replications in Economics.
This is a cross-post of the weblog of the replication network.
“Next year, this topic should not be discussed in a pre-conference workshop but in the opening plenum of the conference!” This statement by a young researcher not only concluded the workshop but also gave bright prospects to replications in Economics.
Hilmar Schneider, Director of the Institute of Labor Economics (IZA, Germany), made an enthusiastic plea for replications in economics and sharply criticized the credibility of much economic research. He argued that without replication research, knowledge can only grow horizontally rather than vertically. According to Schneider, the core problem lies in the pressure for novelty and original research that is exerted by the academic culture and the scientific publication system. As a consequence, pseudo-innovations are preferred to research that may actually have an impact on society. Arbitrary findings that lack verification are insufficient to properly inform and support policy decisions.
Christiane Joerk, Program Director from the German Research Foundation (DFG), summarized the DFG Statement on the Replicability of Research Results. The DFG already financially supports projects that strengthen the infrastructure for replications (e.g. IREE – see below) and will also fund replication studies in the future.
“Is it possible to publish replication studies in the International Journal for Re-Views in Empirical Economics using a pseudonym?” This question reflects the situation of young researchers, who are caught between good scientific practice and the current academic culture. Alarmingly, good scientific practice and academic culture apparently pose a conflict to researchers.
The wish for the anonymous publication of replication studies also reflects the bad reputation of replication studies in economics. On the one hand, most researchers state that replication studies are a critical part of scientific progress and indispensable for good scientific practice. On the other hand, as documented by Maren Duvendack from the University of East Anglia (UK), researchers who do replications are seen to be engaged in “bullying” and “persecution”, and referred to as “research parasites”. Together with her colleagues Richard Palmer Jones and Bob Reed she analyzed the publication market for replications in economics. The authors found that published replications in economics have been increasing since the mid-seventies. However the absolute number is still alarmingly low (with a peak of 22 published replications in economics in 2012). Compared to other disciplines such as psychology and political science, economics lags behind with respect to replication efforts. According to Duvendack, a key obstacle could be the lack of publication outlets for replication studies in economics.
As a partial solution to the replication crisis, Martina Grunow, Project Manager at the ZBW Leibniz Information Center for Economics (Germany), concluded the talks with her presentation of the International Journal for Re-Views in Empirical Economics (IREE). She summarized the incentive problems for authors to conduct replication studies and as a result a lack of publication possibilities for this kind of research. IREE is a peer-reviewed and open-access e-journal which is dedicated to the publication of replication studies in empirical economics. Replication studies are published without regard to their results. IREE-papers are made citable with DOIs and are internationally disseminated via EconStor and RePEc. As the project is funded by the German Research Foundation (DFG) and the ZBW Leibniz Information Center for Economics, IREE does not charge any publication fees. By providing a publication platform for replication studies, IREE aims to encourage the credibility of economics research based on robust and replicable findings.
With the prospect of publishing replication studies, doctoral students participating in the workshop expressed the view that replications should become a mandatory part of their dissertation research. They argued that by doing so, replication research would receive deserved credibility and not be buried under the pressure for pseudo-innovations. This is especially important at the beginning of their academic careers. The participants also agreed on the importance of teaching the value of replications to students.
This post has been written by Dr. Martina Grunow: Martina is Managing Editor of the International Journal for Re-Views in Empirical Economics (IREE) and is an associate researcher at the Canadian Centre for Health Economics (CCHE).
