Goettingen University: Project fosters Replication in Economics

Posted: June 3rd, 2014 | Author: Sven | Filed under: Projects | Tags: economics, replication studies | 1 Comment »

There are many good reasons why we should replicate scientific findings. In his article “Open Access Economics Journals and the Market for Reproducible Economic Research“, the economist B.D. McCullough (2009) lists some of the reasons why replicable research is crucial for science:

„[…]replication ensures that the method used to produce the results is known. Whether the results are correct or not is another matter, but unless everyone knows how the results were produced, their correctness cannot be assessed. Replicable research is subject to the scientific principle of verification; non-replicable research cannot be verified. Second, and more importantly, replicable research speeds scientific progress. We are all familiar with Newton’s quote, ‘If I have seen a little further it is by standing on the shoulders of giants.’ […] Third, researchers will have an incentive to avoid sloppiness. […] Fourth, the incidence of fraud will decrease.“ (p.118)

More recently, the case of the US-economists Rogoff and Reinhart clearly illustrated the need for replications to be much more common in science and scientific education. But…

For this scrutiny to take place, the data and methodology (in the form of executable computer code) must be accessible, and qualified researchers must be willing to undertake the work. In practice it is rare for scientists to make their data and code accessible, and it is rare for scientists to replicate one another’s work, in part because it can be so difficult to get the data. (McCullough and Ross McKitrick, 2009, p.6)

Therefore it is great to see projects dealing with these challenges. One of these projects is the “replication in economics” project at Goettingen University. Replication in economics is run by Jan H. Höffler and Thomas Kneib. In a blog post the two researchers state:

In recent years, we have been teaching replication to students at all levels (ranging from undergraduates to Ph.D. candidates) and have set up a large global network to support the idea of replication by students. In our experience, it is unfortunately not possible to replicate all results exactly (although in most cases this does not change the main story of a paper.) One particular issue is that it is rarely transparent how data sets used for the analyses were obtained from raw data. […] Teaching and providing access to information will raise awareness for the need for replications, provide a basis for research about the reasons why replications so often fail and how this can be changed, and educate future generations of economists about how to make research replicable.

Because of these issues they began to start replication in economics, a wiki that serves as a database of empirical studies, where students and other researchers interested in replicating existing work are able to find suitable data (and further information) for doing so.

Because incentives to conduct a replication study are rare, the two researchers offer a special service: Replication results can be published as replication working papers of the University of Göttingen’s Center for Statistics.

In my opinion this is a very useful and commendable initiative – I would be glad to get to know even more examples of such projects. I you are aware of any other initiatives doing anything similar, please leave a reply in the comments section!

Thank you!

By the way – for our German-speaking readers: In the video below Thomas Kneip introduces the topic of reproducible research:

 

Picture: “Medical Research Miracle” by Mike Licht, NotionsCapital.com on flickr.com. License: CC-BY 2.0

---