Why Replications & Data Availability matter: The Case of Rogoff and Reinhart

Posted: April 18th, 2013 | Author: Sven | Filed under: found on the net | Tags: Data Sharing, economics, Linking Data and Publications | 1 Comment »

Yesterday several blogs, journals and news agencies spread the news: The US star economists Kenneth Rogoff and Carmen Reinhart are in trouble, because a paper of them -which had and has a considerable influence on worldwide economic policy- contains at least one serious error.

Their paper “Growth in a Time of Debt” was published in the flagship of economic research – the American Economic Review (AER) with also has a data availability policy that was evaluated in the course of our research project.
Important political assumptions and decisions were based on the research of Rogoff and Reinhart:  Olli Rehm, European Commissioner for Economic and Monetary Affairs and the Euro and vice president of the European Commission and also the US-Republican Paul Ryan argued with the findings of Rogoff and Reinhart to justify austerity economics.

The Rogoff/Reinhart paper draws a link between high levels of debt and negative average economic growth – an argumentation that has been used by right-leaning politicians to justify austerity budgets: slashing government expenditure and reducing budget deficits in a bid to curtail the growth of debt. The two US-economists claimed that the economic growth of a state considerably diminishes if the ratio between debts and economic performance is higher than 90%. According to the two economists the average economic growth is around 2.8% if the debt-to-GDP ratio is between 60% and 90%. In cases where the debt-to-GDP ratio exceeds the 90% mark the economic growth drops to minus 0.1%.

A new paper, written by three economists Herdon, Ash and Pollin of the University of Massachusetts, suggests that the data used by Rogoff and Reinhart itself is in error. The three economists tried to reproduce the Reinhart and Rogoff result with their own data, but didn’t succeed.  Therefore they asked for the original spreadsheets and checked these files and investigated how Reinhart and Rogoff’s data was constructed.

The results they achieved suggest that  the pro-austerity paper was flawed and that the two US economists were sloppy in handling their data.

Once again this case sheds light to a general problem in economics:

In my opinion this problem is not that the researchers made a mistake in their analyses. The central issue is that the data and code to prove the results were not available and attached – despite the data availability policy of the AER. (The explanation of the Review is that the Rogoff/Reinhart paper was a conference player – and for those conference papers the data policy is not enforced.)
If data and code would have been available the mistake of Rogoff and Reinhart could have been detected much earlier and the implications for both economic research and public policies would not have been so severe.

Especially in economics the availability of data and code is crucial due to the importance of the results the profession achieves and their implications for the political debate.
This case therefore is another strong argument to mandate the availability for data and code for economic research.

More detailed information on the case of Reinert and Rogoff is available on rortybomb.

The paper of Herdon, Ash and Pollin is available here. The data and code files upon which the results are based is also available. Last but not least a text document that describes the files in the code and data archive is available in addition.

#Blogroll#

• Nicole Janz wrote an article on replication and the Reinert/Rogoff-case from the perspective of a political scientist.
• Velichka Dimitrova used the Reinert/Rogoff-Case to argue for open data in economics and beyond.

Chart: Rogoff/Reinhart – http://www.nber.org/papers/w15639.pdf

---